UW Hub Lawn - Test Images - Full Image
Alissa Harrison
The purpose of this project was to create a program that could automatically stitch together a sequence of images given minimal cues. This panorama can then be viewable online using a third-party software tool. There are three steps to stitch the images: first, all of the images must be warped to cylindrical coordinates with simulated camera lens distortion; second, the images are registered using the Lucas-Kanade algorithm; finally, the images are composed together using a linear blending technique. Warping images is done to project the images on to the original plane the pictures came from, i.e. a cylinder. After warping radial distortion is applied to correct for the inherent radial distortion of the lens. In these two steps, we assure that the pictures can be lined up properly, where there exists only a simple linear translation between each pair (and furthermore, if the pictures were taken on a completely level surface, there would only be a horizontal translation between each pair). The Lucas-Kanade motion estimation then works the assumption that the intensity of a particular pixel will be consistent throughout the motion. We can make this assumption with images if the exposure is consistent across the sequence of pictures. However, there is significant limitation with the approximated optical flow equation that the Lucas-Kanade algorithm uses, it only works for small motion. To work around this, the program implements a Gaussian pyramid that sub-samples images to successively lower resolutions, running Lucas-Kanade on each level from coarse to fine resolution, so that at each level, a better approximation is made. Finally, the images are composed using the amount of translation calculated by the Lucas-Kanade algorithm on to one final image canvas. To make a consistent look across the panorama, images are blended which eliminates the lines from slight exposure differences or misalignments. Since pictures can rarely be taken in a completely level environment, they will tend to "drift" in a certain direction throughout the composition. In this step, there is also a compensation by a matrix transformation for this problem.
The program is a command line based and can be directed best by the use of simple batch files. The executable and a sample zip containing a set of images with a batch file will show the user a successful example of running the program. For the program to work, it requires the user to know the focal length of the camera for the particular image size and lens distortion and images with a 4:3 or 3:4 aspect and saved as a 24-bit TGA format.
The code originally is copyrighted by Robert Szeliski, however I implemented the meat of the functions to warp images, perform Lucas-Kanade, and blend images.
The program takes a variety of parameters, including focal length and radial distortion for the first step of warping images. In all my artifacts except the handheld (Fig. 4), I used focal lengths and distortion factors listed for the particular camera I used. In the situation of the handheld artifact I experimented with some different focal lengths and radial distortion in attempt to limit ghosting effects (seeing faint duplicates of the scene). Because I did not rotate the camera precisely around it's optical axis, the images did not line up as well in the composition which lead to ghosting when using the linear blending. I found a larger radial distortion improved the registration somewhat.
There were also the parameters for the number of levels and iterations of the Lucas-Kanade step. Here I experimented a little, of course too many levels will not work since the resolution will eventually sub-sample to values less than 1 but too little will also lead to poor estimates of motion and hence bad registration. For most of my images with overlaps around 200-250 pixels, 4 to 5 levels worked best. As for iterations, again, too little could cause bad registration but too much was wasted computations. Ten iterations was usually far more than necessary, 4-6 usually lead to the optimal solution.
In blending, I found that a blend width window close to the amount of overlap was suitable.
Finally, I used Photoshop to adjust color balances. In some cases where I had extreme exposure differences, I found that making an adjusting in Photoshop first and then stitching lead to must better alignments. For just the sake of "prettying" up the images, I did take out some ghosting artifacts from moving objects in the scene (like people or boats) using Photoshop.
Fig. 1
UW Hub Lawn - Test Images - Full Image
Fig. 2
Gasworks park - View of downtown Seattle - Full
Image
Fig. 3
Gasworks park - "Consume Less and Live More"-
Full Image
Gasworks park - Inside the factory - Full
Image
Fig. 5
