Predicting crises from the news stream
One of the technology capabilities we have in our system is looking for patterns in streams of data coming from the web. We use it to tag events, detect events and as a result pull interesting documents out of the web for our clients.
But we’re not alone in this kind of work, obviously. Wired has an article this week about how the EUs joint research team has developed the EMM: European Media Monitor which is designed to detect patterns in the stream of data.
So what patterns does EMM find? Besides sending SMS and email news alerts to eurocrats and regular people alike, EMM counts the number of stories on a given topic and looks for the names of people and places to create geotagged “clusters” for given events, like food riots in Haiti or political unrest in Zimbabwe. Burgeoning clusters and increasing numbers of stories indicate a topic of growing importance or severity. Right now EMM looks for plain old violence; project manager Erik van der Goot is tweaking the software to pick up natural and humanitarian disasters, too. “That has crisis-room applications, where you have a bunch of people trying to monitor a situation,” Van der Goot says. “We map a cluster of news reports on a screen in the front of the room — they love that.”
Clustering technology is being used in ever wider applications to find what’s interesting from the petabytes of data we now see on the web. We’re just one cool application of it, but the technology space itself is fascinating and fast paced.
