Get news updates from WGBH
Thu January 3, 2013
Innovation Hub 1/5/13: ZipCar. Let's Make a List
Rental car giant Avis is acquiring Cambridge-based ZipCar for $491 million. What will this mean for ZipCar and its devotees?
Let's Make a List
- Cynthia Rudin, assistant professor at MIT's Sloan School of Management.
- Ben Letham, Ph.D. student at MIT.
You might think that Google is the perfect way to comb the Internet for information. After all, it beat out search engine competitors like AltaVista, Ask Jeeves, and Dogpile. And you’d be right — Google is fantastic at finding you specific information, like the phone number of your favorite sandwich shop down the street, or the year of Abraham Lincoln’s inauguration.
But there’s one thing that Google is not very good at: creating lists. Say you wanted a list of traditional Thanksgiving foods. If you type “Thanksgiving foods” into Google, you’ll get recipes for stuffing, turkey, and mashed potatoes. But you won’t get a comprehensive list that breaks down the traditional American holiday table from pumpkin pie to cranberry sauce.
What’s in a List?
Cynthia Rudin, an assistant professor at MIT’s Sloan School of Management, noticed Google’s shortcoming, and she decided to fill in the gap.
“I kept getting stuck in traffic jams in Boston,” she remembers. “And these are traffic jams I could have avoided because they were due to planned events where roads were closed … But I didn’t know about these events in advance because there’s no single complete list of events in Boston. There are only a bunch of incomplete lists on different websites.”
Rudin set out to generate a complete list. She started with familiar events, such as the Boston Arts Festival, and developed an algorithm that allowed her to find similar items. But once the project was finished, Rudin discovered she hadn’t just created a list of Boston events, but a general tool — her algorithm could grow lists of anything on the internet.
“You just seed it with a few examples of whatever you like,” Rudin explains, “and it grows more examples of things like that.”
Where Search Engines Fail
Why are these sorts of lists helpful? Ben Letham, a Ph.D. student at MIT who helped Rudin build the algorithm, asks you to imagine a familiar scenario: you’re looking for the best Thai restaurant in your neighborhood. You enter a search into Google and then scroll through the results, sometimes opening dozens of tabs in your browser as you try to weigh your options — and you still might be missing something.
“Our algorithm actually visits all the websites for you,” Letham says. “So you just type in your [search] and it will visit the website for you and then automatically pull out the useful information and merge it in a nice, seamless way.”
Since making their first list, Letham and Rudin have grown lists of everything from the apps available for your smartphone to quintessential Jewish foods — Rudin’s favorite example. For that list, Letham and Rudin fed their algorithm examples of challah and knishes, and it generated a list that included matzo balls, kugel, and latkes.
“It really seems to capture the diversity and the essence of Jewish cooking,” says Rudin. “We get the information from all over the internet, like recipe sites, and blogs, and restaurant menus.”
The Future of Internet Lists
Although Rudin and Letham have released the details of their algorithm to the public, the list-building tool itself isn’t available — yet. But the two are hopeful that an innovative Internet company, perhaps even Google itself, will help them release it to the public.
“We don’t have the infrastructure and the capital to make this available to the public ourselves,” Letham laments. “But we’ve made all the information about how to do this publicly available, and I hope that Google, or Bing, or another search engine would work with us to implement this.”
In the meantime, they’re willing to share.
“Information wants to be free,” Letham says. “The more we share, the better everything will be for everybody.”
BOSTON PUBLIC RADIO