A new automated computer program could teach itself everything there is to know about a visual concept.
The program, dubbed Learning Everything about Anything (LEVAN), searches through millions of internet images and books using an algorithm to learn every variation of the concept in question, a University of Washington news release reported.
The program would display lists for users to browse, in order to gain a deeper understanding of the concept.
"It is all about discovering associations between textual and visual data," Ali Farhadi, a UW assistant professor of computer science and engineering, said in the news release. "The program learns to tightly couple rich sets of phrases with pixels in images. This means that it can recognize instances of specific concepts when it sees them."
The program analyzes the content of images found on the web using their pixel arrangement as well as phrases and captions.
Users can look at a library of 175 concepts that range from "'airline' to 'window,' and include 'beautiful," "breakfast,' 'shiny,' 'cancer,' 'innovation,' 'skateboarding,' 'robot,' and the researchers' first-ever input, 'horse,' the news release reported.
If the concept one is looking for is not on the list it can be submitted and the program will generate a long list of subcategories.
For examples, a search for the word "dog" would bring up photos of "'Chihuahua dog,' 'black dog,' 'swimming dog,' 'scruffy dog,' 'greyhound dog.' But also 'dog nose,' 'dog bowl,' 'sad dog,' 'ugliest dog,' 'hot dog' and even 'down dog,' as in the yoga pose," the news release reported.
The program scours the books and images on the topic and filters out words that aren't visual, such as "my horse." It would retrieve all the images from a visual phrase such as "jumping horse."
"Major information resources such as dictionaries and encyclopedias are moving toward the direction of showing users visual information because it is easier to comprehend and much faster to browse through concepts. However, they have limited coverage as they are often manually curated. The new program needs no human supervision, and thus can automatically learn the visual knowledge for any concept," said Santosh Divvala, a research scientist at the Allen Institute for Artificial Intelligence and an affiliate scientist at UW in computer science and engineering.
WATCH:
