Hackers Market Research: Screen Scraping + Visualization

After learning how to scrape websites I have been playing around with ways to use the information and came across an interesting application: visualizing keywords. Using Wordle, you can take content scraped from your competitors websites, run it through the visualizer and see the most common words used on the sites.

This could have some useful applications for copywriting, SEO and avoiding the most overused jargon (todays winner: CRM).

The Method

I ran a scRUBYt! script to gather all the text from these ten sales software websites (home and product pages):

Salesforce, Right90, PivitolCRM, Sage, Netsuite, Entellium, Goldmine, Maximizer, Softfront, and Aplicor.

You can see the script and the full Wordle cloud.