Photosynth – reconstructed three-dimensional space with photos

Posted on April 10, 2008
Filed Under Photo | 3 Comments

Photosynth is an amazing new technology from Microsoft Live Labs that will change forever the way you think about digital photos. It takes a large collection of photos of a place or an object, analyzes them for similarities, and displays them in a reconstructed three-dimensional space.

With Photosynth you can:

• Walk or fly through a scene to see photos from any angle.
• Seamlessly zoom in or out of a photo whether it’s megapixels or gigapixels in size.
• See where pictures were taken in relation to one another.
• Find similar photos to the one you’re currently viewing.
• Send a collection – or a particular view of one – to a friend

A Photosynth experience begins with nothing more than a bunch of digital photos. They might all have been taken by one person, or they might be a mixture of images from many different cameras, shooting conditions, dates, times of day, resolutions, and so on. Each photo is processed by computer vision algorithms to extract hundreds of distinctive features, like the corner of a window frame or a door handle. Photos that share features are then linked together in a web. When the same feature is found in multiple images, its 3D position can be calculated. It’s similar to depth perception – what your brain does to perceive the 3D positions of things in your field of view based on their images in both of your eyes. Photosynth’s 3D model is just the cloud of points showing where those features are in space.
Your brain knows that your eyes are about two inches apart. But when Photosynth does its magic, it doesn’t know where the cameras were, or which way they were pointing. Fortunately, when there are many cameras, and many features in common, the algorithms behind Photosynth can figure out not only where the features are in 3D, but where all of the cameras would have to have been, and which way they were aimed, consistent with the features they “saw”.

The Photosynth client shows you the 3D point cloud, but more importantly, it also shows you the original pictures overlaid on the model. Imagine a slide projector placed at each original camera position, aimed how the camera was, and projecting the picture that camera took. A screen is placed in the 3D environment at an appropriate distance from the projector. As you move around in the Photosynth environment, projectors turn on and off, giving you a changing perspective on a world built entirely out of the original photos.
Photosynth provides an immersive interface where you can “walk up” to any object in a photograph…or place yourself where the photographer was when the picture was taken.
Photosynth provides easy controls for exploring a collection of photos in 3D with a click of a mouse. Imagine being able to see your favorite band from either the front row or the bleachers, walking around that new motorbike you’ve been thinking of buying, or flying over every nook and cranny of a famous landmark.
In a future version of Photosynth, you’ll be able to see your own photographs displayed in 3D, forever changing the way you think of taking – and viewing – photos. Add the ability to see your pictures photosynthed together with those taken by other people and you’ll start to see the endless possibilities that Photosynth promises to provide.

Comments