An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 64941 pageviews (an average of 23 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
02/06/20169
02/05/201617
02/04/201620
02/03/201617
02/02/201615
02/01/201618
01/31/201629
01/30/201614
01/29/201613
01/28/201628
01/27/201614
01/26/201615
01/25/201643
01/24/201625
01/23/201621
01/22/201620
01/21/201624
01/20/201621
01/19/201618
01/18/201634
01/17/201628
01/16/201613
01/15/201610
01/14/201621
01/13/201631
01/12/201628
01/11/201627
01/10/201636
01/09/201619
01/08/201617
01/07/201637

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct25757
imported from google analytics10839
www.google.com5202
lethain.com4829
news.ycombinator.com4790
www.poynter.org4270
www.stumbleupon.com1387
www.reddit.com1107
paradox1x.org1086
www.quora.com861
schoolofdata.org565
stackoverflow.com531
webscraping.com331
dev.lethain.com282
feeds.delicious.com166
es.schoolofdata.org160
www.codeproject.com121
www.mozenda.com110
www.delicious.com96
thiagomarzagao.com90
ask.metafilter.com90
news.ycombinator.org85
sitescraper.net75
wiki.greasespot.net59
hackerne.ws41
twitter.com36
www.bing.com32
searchyc.com32
mail.python.org32
www.instapaper.com31
realtime.michaelhart.me30

All Rights Reserved, Will Larson 2007 - 2016.