An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 66886 pageviews (an average of 23 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
05/27/20167
05/26/20167
05/25/20164
05/24/201613
05/23/201613
05/22/201616
05/21/201619
05/20/201617
05/19/201610
05/18/201621
05/17/201612
05/16/201614
05/15/201626
05/14/201617
05/13/20168
05/12/201611
05/11/201618
05/10/201615
05/09/201621
05/08/201618
05/07/201617
05/06/201612
05/05/201623
05/04/201618
05/03/201613
05/02/201616
05/01/201626
04/30/201617
04/29/201617
04/28/201618
04/27/201622

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct26959
imported from google analytics10839
www.google.com5408
lethain.com5050
news.ycombinator.com4798
www.poynter.org4419
www.stumbleupon.com1387
www.reddit.com1108
paradox1x.org1086
www.quora.com864
schoolofdata.org607
stackoverflow.com535
webscraping.com331
dev.lethain.com290
es.schoolofdata.org194
feeds.delicious.com166
www.codeproject.com121
www.mozenda.com110
thiagomarzagao.com103
www.delicious.com96
ask.metafilter.com91
news.ycombinator.org85
sitescraper.net75
wiki.greasespot.net60
hackerne.ws41
twitter.com36
www.bing.com32
searchyc.com32
mail.python.org32
eg.okfn.org32
www.instapaper.com31

All Rights Reserved, Will Larson 2007 - 2016.