Computing A 3D Model From A Single 2D Image

The normal image was a picture of Frank Lloyd Wright's "Fallingwater" house:

Vanishing Points

First, the vanishing lines were drawn (red for X-vanishings points, green for Y, and blue for Z):

You'll notice that the Y-vanishing points don't follow edges along the Y-axis.  This is because the true Y-vanishing point (marked by an orange square) was too close to some of the polygon corners, and the result was that the corners were believed to be very close to "infinity" in the Y direction.  So the solution was to simply force the Y-vanishing point out further by drawing only 2 lines and having them intersect where I wanted the new vanishing point to be.

Homography

Once the vanishing points were computed, the reference homography needed to be calculated, and a reference height needed to be chosen.  The following points where used, red for the reference homography points, and green for the reference height.

The reference coordinates were (with the origin in the lower left, x increasing to the right, y increasing to the left, z increasing upwards): (0,0,0) (2,0,0) (2,1,0) (0,1,0).  The reference height coordinate was (0,1,1).

Finding 3D points

With a reference plane and a reference, it was a matter of patience to find a good collection of 3D points to use as polygon vertexes.  The points were discovered from the reference plane by using one of the two following facts:

1) The new point had the same z coordinate as a known 3D point, so computing its x and y coordinates was a simple homography.

2) The new point had the same x and y coordinates as a known 3D point, so I used the reference height to find the z coordinate.

Here are all the 3D points:

Polygons

Once the 3D points have been found, it was a simple matter of creating polygons out of them.  Then, using a homography calculated for each polygon, its texture map could be extracted from the original image.  Putting all this together in a VRML file produces a texture-mapped model of this 2D image.  Click here to see it!

Here are some views from new viewpoints:

And some of the texture maps: