An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 66440 pageviews (an average of 23 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
04/28/201618
04/27/201622
04/26/201629
04/25/201623
04/24/201616
04/23/201612
04/22/201614
04/21/201613
04/20/201620
04/19/201614
04/18/201617
04/17/201610
04/16/201613
04/15/201613
04/14/201617
04/13/201619
04/12/201628
04/11/201616
04/10/201620
04/09/201620
04/08/201616
04/07/201617
04/06/201625
04/05/201624
04/04/201615
04/03/201621
04/02/201611
04/01/201617
03/31/201616
03/30/201624
03/29/201621

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct26666
imported from google analytics10839
www.google.com5349
lethain.com5025
news.ycombinator.com4796
www.poynter.org4380
www.stumbleupon.com1387
www.reddit.com1108
paradox1x.org1086
www.quora.com864
schoolofdata.org597
stackoverflow.com533
webscraping.com331
dev.lethain.com290
es.schoolofdata.org188
feeds.delicious.com166
www.codeproject.com121
www.mozenda.com110
thiagomarzagao.com101
www.delicious.com96
ask.metafilter.com91
news.ycombinator.org85
sitescraper.net75
wiki.greasespot.net60
hackerne.ws41
twitter.com36
www.bing.com32
searchyc.com32
mail.python.org32
eg.okfn.org32
www.instapaper.com31

All Rights Reserved, Will Larson 2007 - 2016.