An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 51421 pageviews (an average of 24 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
04/15/201412
04/14/201422
04/13/201415
04/12/201418
04/11/201419
04/10/201419
04/09/201424
04/08/201424
04/07/201420
04/06/201417
04/05/20148
04/04/201415
04/03/201417
04/02/201420
04/01/201429
03/31/201420
03/30/201416
03/29/201420
03/28/201419
03/27/201415
03/26/201423
03/25/201426
03/24/201425
03/23/201424
03/22/201424
03/21/201420
03/20/201429
03/19/201415
03/18/201415
03/17/201419
03/16/201424

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct18286
imported from google analytics10839
news.ycombinator.com4772
www.google.com4262
www.poynter.org3374
lethain.com2664
www.stumbleupon.com1370
www.reddit.com1088
paradox1x.org1082
stackoverflow.com443
www.quora.com429
dev.lethain.com269
schoolofdata.org266
webscraping.com201
www.mozenda.com107
www.delicious.com96
www.codeproject.com91
news.ycombinator.org85
sitescraper.net75
feeds.delicious.com70
ask.metafilter.com51
hackerne.ws41
wiki.greasespot.net37
twitter.com36
searchyc.com32
mail.python.org32
www.instapaper.com31
realtime.michaelhart.me30
hckrnews.com25
es.schoolofdata.org25
www.hnsearch.com22

All Rights Reserved, Will Larson 2007 - 2014.