in Visualization

Visualizing web site interlinkage using Processing

Processing is an open project initiated by Ben Fry (of MIT fame) and Casey Reas (UCLA Design). From the processing.org web page:

Processing is an open source programming language and environment for people who want to program images, animation, and sound. Processing is developed by artists and designers as an alternative to proprietary software tools in the same domain.

I have been dabbling with it for a while and have come to like it’s simplicity. The downside is that it sometimes demands a lot from your computer’s processing power.

How it works

The link data for this sketch was collected with a Python script. 250 government web sites were crawled and links to other web sites on the list were recorded. This generated some 2000 links which were fed to the Processing sketch.

Web site links animation snapshot.

The resulting visualization applet can be viewed here. Please note that it will require some processing power to get it running smoothly. You can also view a screenshot of the sketch in action.

Every site starts out as a small disc at the bottom of the screen. More incoming links makes the site grow. Larger sites float to the top of the screen. Smaller sites move out of the way for larger sites. Hovering with your mouse over a site highlights outgoing links from that site to other sites.

Findings

The next step is to try PyOpenGL and see if speed can be improved when drawing hese simple 2D primitives. Initial tests looks promising.

Write a Comment here on the real web

Comment

  1. This was an interesting visualization. What did you use to collect the links?

  2. Drew, I used a simple web crawling script in Python. Links were recorded in the format from-url;to-url and then fed to Processing (which has limited support for dealing with text files).