An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 54033 pageviews (an average of 24 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
08/29/201422
08/28/201419
08/27/201425
08/26/201427
08/25/201433
08/24/201429
08/23/201421
08/22/201423
08/21/201422
08/20/201475
08/19/201423
08/18/201427
08/17/201431
08/16/201423
08/15/201425
08/14/201431
08/13/201427
08/12/201420
08/11/201433
08/10/201421
08/09/201424
08/08/201418
08/07/201423
08/06/201421
08/05/201416
08/04/201416
08/03/201423
08/02/201415
08/01/201411
07/31/201419
07/30/201420

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct19649
imported from google analytics10839
news.ycombinator.com4772
www.google.com4395
www.poynter.org3776
lethain.com2887
www.stumbleupon.com1375
www.reddit.com1094
paradox1x.org1085
www.quora.com483
stackoverflow.com465
schoolofdata.org342
dev.lethain.com270
webscraping.com235
feeds.delicious.com112
www.mozenda.com110
www.codeproject.com107
www.delicious.com96
news.ycombinator.org85
sitescraper.net75
ask.metafilter.com56
es.schoolofdata.org43
hackerne.ws41
wiki.greasespot.net40
twitter.com36
searchyc.com32
mail.python.org32
www.instapaper.com31
realtime.michaelhart.me30
hckrnews.com25
www.hnsearch.com22

All Rights Reserved, Will Larson 2007 - 2014.