Mid Term Report

This project has changed significantly over the last five weeks. Initially I grew interested in analyzing Twitter data because of issues of censorship in many countries around the world. I thought looking at the January revolution in Egypt would be an interesting way to do so, especially since internet access was cut off at one point during the event. By gathering tweets from that period I thought a good view of what was going on and how it changed as the government tried to censor the citizenry I could examine issues of censorship in Egypt.

As the term progressed I realized that I was too far out from the event to gather tweets from that period easily. The Twitter API only allows access so far back in the public time line. So, I started by gathering more current tweets about Egypt. These are currently available on the data page of this site. Many of these tweets turned out to be in Arabic, which makes working with them in Processing difficult as it does not seem to support Arabic text by default.

Last Sunday my project took a very distinct turn. Osama Bin Laden was killed and Twitter exploded talking about it. I had already been working with Processing to gather tweets as an alternative to accessing the API via a web browser (as I did initially). I modified Robotgrrl’s Simple Twitter for Processing program to gather more tweets, limit them to English, and then write the username, date, and tweet to a text file. So, I was part way to gathering mass data from Twitter programmatically.

I did not think to start my program running during President Obama’s address to the nation. However, immediately after I set to work gathering data. I rewrote my program to access the Twitter time line a total of 1000 times with a half second pause between each call. This was done with a simple for loop. I was able to gather a total of 73,200 tweets. Unfortunately Twitter timed out my access before the full 1000 cycles of data collection so I was unable to obtain the full 100,000. However, I now have plenty of data to analyze. Having this current event to analyze makes this project much more timely, even if it is a divergence from the original plan.

My next step in this process is to begin keyword analysis of the event. After my presentation on Tuesday Stacy suggested some tools for conducting keyword analysis that will not require me to write a program from scratch in Processing. I will be looking at AndConc tomorrow and exploring its features. Hopefully it will output in a format that Processing is able to work with, perhaps as an array or hashmap.

After doing keyword analysis I want to work on a way to turn the data generated into a three dimensional map of the event. Ultimately I would like to develop a means to create the 3D image and print it in 3D. Having a physical representation of events in the world based on the immediate reaction of thousands of people around the world could be a powerful way to examine events. This has turned into as much of an art project as a research project and ultimately I would like to create enough of these to be part of a gallery showing.