An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 54929 pageviews (an average of 24 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
09/29/201421
09/28/201443
09/27/201439
09/26/201433
09/25/201445
09/24/201429
09/23/201435
09/22/201428
09/21/201439
09/20/201436
09/19/201430
09/18/201429
09/17/201431
09/16/201419
09/15/201436
09/14/201431
09/13/201418
09/12/201420
09/11/201428
09/10/201427
09/09/201434
09/08/201423
09/07/201418
09/06/201421
09/05/201414
09/04/201423
09/03/201427
09/02/201424
09/01/201433
08/31/201428
08/30/201428

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct20039
imported from google analytics10839
news.ycombinator.com4772
www.google.com4428
www.poynter.org3826
lethain.com2977
www.stumbleupon.com1376
www.reddit.com1095
paradox1x.org1085
www.quora.com520
stackoverflow.com466
schoolofdata.org354
dev.lethain.com270
webscraping.com245
feeds.delicious.com119
www.mozenda.com110
www.codeproject.com109
www.delicious.com96
news.ycombinator.org85
sitescraper.net75
ask.metafilter.com59
es.schoolofdata.org48
wiki.greasespot.net43
hackerne.ws41
twitter.com36
searchyc.com32
mail.python.org32
www.instapaper.com31
realtime.michaelhart.me30
hckrnews.com25
www.hnsearch.com22

All Rights Reserved, Will Larson 2007 - 2014.