An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 57888 pageviews (an average of 24 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
01/25/20153
01/24/201526
01/23/201513
01/22/201529
01/21/201522
01/20/201516
01/19/201526
01/18/201528
01/17/201518
01/16/201518
01/15/201522
01/14/201518
01/13/201524
01/12/201519
01/11/201520
01/10/201517
01/09/201521
01/08/201516
01/07/201516
01/06/201517
01/05/201518
01/04/201515
01/03/201517
01/02/201514
01/01/201516
12/31/201419
12/30/201421
12/29/201412
12/28/201428
12/27/201423
12/26/201419

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct21491
imported from google analytics10839
news.ycombinator.com4776
www.google.com4625
www.poynter.org3926
lethain.com3825
www.stumbleupon.com1380
www.reddit.com1099
paradox1x.org1086
www.quora.com626
stackoverflow.com489
schoolofdata.org383
dev.lethain.com274
webscraping.com263
feeds.delicious.com155
www.codeproject.com112
www.mozenda.com110
www.delicious.com96
news.ycombinator.org85
sitescraper.net75
es.schoolofdata.org75
ask.metafilter.com68
wiki.greasespot.net50
hackerne.ws41
twitter.com36
searchyc.com32
mail.python.org32
www.instapaper.com31
realtime.michaelhart.me30
www.bing.com25
hckrnews.com25

All Rights Reserved, Will Larson 2007 - 2015.