Studios and news libraries around the world have a large number of old film archives that are waiting for a new lease of life. These old movies are recorded on films, which are susceptible to degradation such as gouges, scratches and the accumulation of dirt. The result is a variety of artefacts that make the old movies look their age. The deterioration in old movies can be stopped by adopting digital film archiving technology, but defects that are already present in the films will be inherited into the digital storage. Restoration of degraded motion pictures is a labour-intensive and costly undertaking. A much publicised event is the restoration work of Disney’s 1937 masterpiece – Snow White and the Seven Dwarfs, which was re-released in 1993 [1]. It would be rewarding to reproduce the old movies with as much fidelity to the original negatives as possible so that the movies can be re-released in other media such as DVD, cable-TV, TV broadcast stations, and video-on-demand. Therefore, a motion picture restoration system that can automatically remove the artefacts in video archives will be of useful to the entertainment and broadcast industries. Besides using such a restoration system to restore old video, the system can also serve as a pre-processor for video encoding in VCD or DVD-mastering application. This is because video containing noise will reduce the efficiency of the video encoder.
A computerised motion picture restoration software – Revival has been successfully developed at Nanyang Technological University of Singapore. Revival automatically restores old, noise-corrupted motion pictures and transfers the restored pictures to new storage media with very minimal human supervision. In addition, Revival provides an ergonomically designed user-interface and openness allowing users to easily integrate Revival into their system (such as telecine transfer, MPEG encoding, and tape re-mastering) to meet specific production requirements. Figure 1 shows the stages of how such an automated system is realised. The restoration process begins with the conversion of the degraded video (stored in Betacam tape) into its digital form with the aid of a real-time, high-resolution video digitiser. Alternatively, digital video stored in D1 format can also be read directly into a Digital-Disk-Recorder (DDR) for processing. The statistical redundancy in motion picture provides us a way to mathematically model the image region at the vicinity of these artefacts so that meaningful information may be used to fill in the corrupted image regions, resulting in a restored image frame.
The crux of our picture restoration system is a novel mathematical model that accurately detects, locates and restores the distorted image regions without causing distortions to other image areas [2, 3]. Our system has the important advantage over existing global filtering strategies that tend to blur sharp edges or homogenise highly textured regions in both the distorted and uncorrupted image regions.
Revival is a high precision noise-removal software that focuses on restoring old movies having scratches and dirt-and-sparkle noise. The first release of Revival is implemented on an industrial standard platform, UNIX and OSF/MOTIF. Currently Revival can run on Silicon Graphics workstations with DDR. Recognising the enormous impact on user productivity, a friendly user-interface and streamlined workflow are developed. The restoration system can be operated by computer novices and can speed up the throughput of post-production experts.
2. SYSTEM AND USER-INTERFACE
The Revival project is to develop a resolution-independent motion picture restoration system for broadcasters, movie producers, media preservationists, and post-production specialists, with the emphasis on intuitive graphical user-interface and streamlined workflow.
The workflow is designed to provide the user with flexibility to restore any specified set of segments of the motion picture. For example, a movie might consist of a few batches (segments) that require restoration work, as shown in Figure 2; the user can specify the extend of the segment by entering the start-and-end frame or in-and-out time code. In addition, REVIVAL also provides users with EDL input to better integrate REVIVAL into the users’ existing workflow.
Two main architectural components combine to make Revival a powerful picture restoration system. The first is the Blotch, Dirt-and-Sparkle Removal Algorithm, which allows users to automatically restore the motion picture that is corrupted by temporally isolated noise. The Blotch, Dirt-and-Sparkle Removal menu is shown in Figure 3. The user has the flexibility to fine-tune three key parameters to optimise performance. The first parameter reflects the amount of motion that is exhibited in the particular segment. The second parameter allows the user to select the degree of noise that the sequence exhibits. Although the default values of these two parameters are provided, the fine-tuning will allow more efficient processing to be carried out on resource-limited workstation. The third options provide the user with a choice to select explicitly the restoration-quality and processing-speed trade-off.
The second main architectural component is the persistent-scratch removal algorithm. Vertical scratches require special treatment due to the repetition of the scratches at the same or nearby location in adjacent frames. An adaptive scratch-removal algorithm has been developed in Revival that efficiently and effectively conceals vertical scratches in old motion picture. The adaptive process also ensures that only areas of the image that are suffering from scratches are processed, leaving the rest of the image as before. Most common types of vertical scratches, including positive and negative scratches, are effectively handled. The user can fine-tune parameters as shown in Figure 4, to enable the best possible concealment. A major advantage of the scratch removal process in Revival is that vertical scratches are automatically detected; the user need not therefore have to specify the position of the scratches. Figure 4 also shows other options for users to specify the presence of subtitles, further protecting the crispness and clarity of the subtitles. In addition, impulsive or salt-and-pepper noise can be handled with the speckle removal option.
The aim of Revival is to be image-resolution independent thus allowing restoration work to be carried out on various types of formats such as CCIR 601 PAL/NTSC or HDTV resolution. The user can select the different image formats from the input/output menu as shown in Figure 5. One unique feature of Revival is the ability to automatically handle artefacts due to the 3:2 pull-down telecine transfer. Revival also allows the seamless connection of the restoration system to the Digital VTR (such as D1) or Betacam VTR via a third party DDR, thus providing a total solution to motion picture restoration.
The computerised motion picture restoration system has been tested on different image sequences containing different noise processes such as pepper noise, variable size blotches and line scratches. The restoration quality of the algorithm on naturally degraded image sequence is shown in Figures 6a, 6b, 7a, 7b, 8a and 8b. The video was first digitised from a Betacam tape before applying the algorithm onto the image sequences. The original sequences are heavily blotched and have been restored effectively by the system. It can be seen from these figures that the noise-level of the image sequence is significantly reduced while the crispness and sharpness of the original image sequence are retained. The results are in contrast with most of the existing image-restoration algorithms which blur the edges of moving objects in the vicinity of occluded and uncovered image regions. The video restoration system described here can successfully suppress mixed-noise processes and recover lost signals in both the covered and uncovered regions in image sequences. The layout of the complete computerised video restoration system is shown in Figure 9.
