Efros worked with Jun-Yan Zhu, UC Berkeley computer science graduate student and the paper’s lead author, and Yong Jae Lee, former UC Berkeley postdoctoral researcher, to develop the system, which they have dubbed AverageExplorer.
The researchers provided examples of potential applications of this system, such as in online shopping, where a consumer may want to quickly home in on two-inch wedge heels in the perfect shade of red. Or perhaps media analysts would like to see Stephen Colbert’s typical body posture when the face of President Barack Obama appears in the graphic over his shoulder.
Lee, now an assistant professor in computer sciences at UC Davis, said the system could also be used to help improve the ability of computer vision systems to distinguish key features in an image, such as the tires on a car or the eyes on a face. When users mark those features on an average image, the entire collection of images is automatically annotated as well.
“In computer vision, annotations are used to train a system to detect objects, so you might mark the eyes, nose and mouth to teach the computer what a human face looks like,” said Lee. “Lots of data is needed to accurately train the system, so reducing the amount of effort and time to do this is critical. Instead of annotating each image individually, with AverageExplorer, we only need to annotate the average image, and the system will automatically propagate the annotations to the image collection.”
The researchers were inspired by artists like James Salavon, who has created average images from hundreds of photos of kids with Santa, newlyweds or baseball players to illustrate a concept. Average images can provide interesting insights, such as the convention in Western culture for brides to wear white and stand to the right of the groom in formal portraits, or for youth baseball players to get down on one knee in their official photo.
Many of the manual steps Salavon used to sort and align his images are now automated through the UC Berkeley tool.
Funding from Google, Adobe and the Office of Naval Research helped support this work.
- AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections (ACM Transactions on Graphics, SIGGRAPH paper, August 2014)