An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 60445 pageviews (an average of 24 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
06/01/20154
05/31/201512
05/30/201517
05/29/20156
05/28/20157
05/27/201516
05/26/201515
05/25/201515
05/24/201512
05/23/201511
05/22/201511
05/21/201520
05/20/201518
05/19/201520
05/18/201513
05/17/201514
05/16/201515
05/15/201519
05/14/201523
05/13/201530
05/12/201527
05/11/201516
05/10/201521
05/09/201519
05/08/201516
05/07/201514
05/06/201520
05/05/201528
05/04/201523
05/03/201517
05/02/201519

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct22809
imported from google analytics10839
www.google.com4822
news.ycombinator.com4777
lethain.com4322
www.poynter.org4043
www.stumbleupon.com1386
www.reddit.com1104
paradox1x.org1086
www.quora.com794
stackoverflow.com502
schoolofdata.org449
webscraping.com296
dev.lethain.com275
feeds.delicious.com166
www.codeproject.com118
www.mozenda.com110
es.schoolofdata.org102
www.delicious.com96
news.ycombinator.org85
ask.metafilter.com77
sitescraper.net75
wiki.greasespot.net54
thiagomarzagao.com54
hackerne.ws41
twitter.com36
searchyc.com32
mail.python.org32
www.instapaper.com31
realtime.michaelhart.me30
www.bing.com25

All Rights Reserved, Will Larson 2007 - 2015.