Flying taxis made possible thanks to new 3D photo manipulation software

Photo manipulation software, while becoming ever more powerful, have always been limited to two-dimensional edits. While it is possible, for example, to move individual objects in a photo and have their backgrounds intelligently restored from the surrounding information, it is not possible to interact with these objects in a three-dimensional way.

At least, that's the status quo. A team of researchers at the Carnegie Mellon University in Pittsburgh, PA is about to change that, though. Their newly developed technology lets you not only move and copy objects in a photograph (with subsequent background restoration of course,) but also makes it possible to interact with objects in three dimensions.

Think, for example, of a still life photograph featuring various fruit in a bowl. The software developed at CMU would make it possible to pick individual fruit out of the bowl, move them around the frame, and even turn them around, exposing their opposite side which hasn't originally been photographed by the camera.

As much as this sounds like witchcraft, it's actually just a pretty clever software trick. Too bad, and here we thought some real magic was going on there. By making use of stock 3D models of all kinds of generic (or specific) objects, the software is able to interpolate what a given object in a photo would look like from the other side.

In order to make the appearance realistic, the software analyzes the lighting in the photo as well as the object's texture and color. This way, the research team was able to freely interact with all kinds of objects in photographs and achieve realistic-looking results. However, the software is limited in what it can do by the availability of 3D models of the objects a user wishes to manipulate. In the video below, you can see a demonstration of what is possible with the software so far.

As fascinating as this technology is, it raises a question that is as old as photography itself: can we trust a photograph to be a truthful representation of reality? Considering the possibilities that arise from the CMU research team's photo manipulation technology, the answer clearly is no. We would have no means to tell whether the objects depicted in a photograph have actually been captured by the camera, or whether they have been moved around or even inserted into the photo after its capture. (The same, by the way, goes for paintings, which can also theoretically be manipulated with the software.)

So will we soon have to doubt each and every photo we see? Probably not. For one, it is entirely unclear whether and when the technology will be available to the general public. Then, its operation requires quite a bit of skill and attention to detail, and involves more than just clicking on an object and then turning it around and dragging it to another position. We could theorectically pixel-peep and potentially observe errors in the 3D rendering in the manipulated objects.

Also, think about this: how much can we trust a photograph anyway? Photoshop is already making extensive manipulations possible, and as we all know, the fashion industry and portrait photographers make extensive use of that -- as do China and North Korea for propaganda purposes. And even if a photo hasn't been digitally manipulated, there still is the possibility that a scene is entirely staged.

So this new kind of 3D interaction with 2D objects is probably just the next logical step in the evolution of photo manipulation techniques.

(via DIY Photography)