An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 55597 pageviews (an average of 24 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
10/23/20148
10/22/201426
10/21/201425
10/20/201442
10/19/201431
10/18/201427
10/17/201432
10/16/201428
10/15/201426
10/14/201435
10/13/201431
10/12/201431
10/11/201421
10/10/201430
10/09/201424
10/08/201419
10/07/201432
10/06/201423
10/05/201417
10/04/201424
10/03/201425
10/02/201424
10/01/201445
09/30/201434
09/29/201429
09/28/201443
09/27/201439
09/26/201433
09/25/201445
09/24/201429
09/23/201435

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct20433
imported from google analytics10839
news.ycombinator.com4773
www.google.com4469
www.poynter.org3854
lethain.com3097
www.stumbleupon.com1377
www.reddit.com1096
paradox1x.org1085
www.quora.com543
stackoverflow.com471
schoolofdata.org358
dev.lethain.com271
webscraping.com247
feeds.delicious.com127
www.mozenda.com110
www.codeproject.com110
www.delicious.com96
news.ycombinator.org85
sitescraper.net75
ask.metafilter.com60
es.schoolofdata.org54
wiki.greasespot.net44
hackerne.ws41
twitter.com36
searchyc.com32
mail.python.org32
www.instapaper.com31
realtime.michaelhart.me30
hckrnews.com25
www.hnsearch.com22

All Rights Reserved, Will Larson 2007 - 2014.