Not unexpectedly, the recent announcement of A*STAR’s $100k competition drew quite a lot of flak from all over.
USD$100,000 for a search engine? Laughable, critics say. And they’re right – if that’s indeed what the competition is about.
But that’s not really the case, is it? Looking over the details, the competition isn’t about coming up with a full-blown, optimised search engine. It’s merely a scavenger hunt within a large (but limited) data space, and prior to the actual hunt, participants will have a period of three months to develop the algorithms needed for the hunt itself. They’ll have some sample data and examples to help them along, according to the website:
“…The organizers will release a set of development data, sample queries & answers and recurrent voice segment annotation & evaluation tools…”
So at the end of the preliminary round, based on what they’ve been given, each team will have developed a set of search tools. Then, during the qualifying race, the teams will pitch their algorithms against one another on a similar, but previously unseen, set of data. Given the time frame, I’m guessing that the search tasks would be rather simplistic, for instance, identifying a particular car license plate in a ten hour long video of parking lot CCTV footage, and being able to do it via a method faster than, say, having all ten persons in the team each watching a tenth of the video.
The rules for the qualifying race itself:
The race on 20 August 2008 will be run as follows:
The race will comprise a total of four challenges.
- The media for all four challenges will be made available to the teams in July-August 2008.
- Solving a challenge will provide the teams a clue which will lead them to subsequent challenges, until the fourth challenge and the final destination.
- Each challenge will offer the teams a choice between 2 Search Tasks. The teams must choose and attempt one. If they are unable to solve the initial chosen Search Task or wish to switch Search Tasks mid-way, they may do so without penalty. The time spent on the failed/incomplete task will, however, be counted as part of their overall performance.
- Teams will be judged based on the time taken to reach the final destination
Sounds familiar? It should – it’s just The Amazing Race in a different guise. Hey, even the prize money is the same – for the Asian edition at least – USD$100,000!
Now, how all of this morphed into a “search for the ultimate holy grail”, well, it’s not too hard to figure out. The process was probably something like this:
- Researchers come up with a list of outstanding problems yet to be solved
- Administration layer 1, not quite understanding the technical details, rewrites problems in own words
- Administration layer 2 thinks it’s a great idea to run this as a Grand Challenge (ala DARPA and Ansari)
- Administration layer 3 throws in hype and buzzwords
- Administration layers 4-99 does the same
- Administration layer 100 ends up doing a bloated press release
- The press sexes it up even further because it needs to sell stories; original researchers go “wha…?”
Don’t even get me started on the sorry state of scientific journalism in this country.
In any case, my bet’s on the team that uses the stealth Mechanical Turk.
Posted by Wolf
Posted by Wolf
Posted by Wolf