http://www.peterkrantz.com/simplecrawler/wiki/ 2008-09-06T01:39:17+00:00 http://www.peterkrantz.com/simplecrawler/wiki/ http://www.peterkrantz.com/simplecrawler/wiki/lib/images/favicon.ico text/html 2008-01-26T15:27:04+00:00 Peter Krantz http://www.peterkrantz.com/simplecrawler/wiki/principles Basic Principles of using SimpleCrawler Using SC involves the following steps: 1. Require the simplecrawler library require 'simplecrawler' 2. Create an instance of the SimpleCrawler::Crawler object Pass the website address as a parameter if you like. text/html 2008-01-26T13:37:55+00:00 Peter Krantz http://www.peterkrantz.com/simplecrawler/wiki/examples/accessibility-report Site Accessibility Report with Raakt and Ruport In this example SimpleCrawler can be used to crawl a website, test each page with the Ruby Accessibility Analysis Kit and format the result with Ruport. Please note that you must have a working installation of the Ruby programming language and the RubyGems package manager to use this code. text/html 2007-08-27T17:45:34+00:00 Peter Krantz http://www.peterkrantz.com/simplecrawler/wiki/start With SimpleCrawler (SC) basic web crawling becomes easy in Ruby. Use SimpleCrawler as the foundation for your own crawling needs. SC is inspired by code in an article by Scott Nedderman (which didn't work properly for me). A minimal example (crawl a website and print page titles): text/html 2007-08-27T17:36:47+00:00 Peter Krantz http://www.peterkrantz.com/simplecrawler/wiki/examples/find-pdf-documents Find all PDF documents on a site This is an example of how SimpleCrawler can be used to find documents of a specific type on a website. In this case the site in the command line argument is crawled and URIs for documents of the type “application/pdf” is returned.