An Introduction to Compassionate Screen Scraping

08/10/2008

Page Summary

One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Page Statistics

An Introduction to Compassionate Screen Scraping has received 56463 pageviews (an average of 24 views per day since publication).

Pageviews for Recent Days

Show daily pageviews for trailing window.

DateViews
11/23/201420
11/22/201427
11/21/201434
11/20/201427
11/19/201440
11/18/201430
11/17/201441
11/16/201436
11/15/201424
11/14/201421
11/13/201432
11/12/201431
11/11/201425
11/10/201423
11/09/201441
11/08/201415
11/07/201420
11/06/201432
11/05/201421
11/04/201427
11/03/201426
11/02/201421
11/01/201415
10/31/201437
10/30/201423
10/29/201416
10/28/201432
10/27/201436
10/26/201426
10/25/201426
10/24/201432

Page Referrers

Top referrers for this page. Show up to 50 referrers with at least 10 pageviews.

ReferViews
direct20814
imported from google analytics10839
news.ycombinator.com4774
www.google.com4527
www.poynter.org3888
lethain.com3363
www.stumbleupon.com1378
www.reddit.com1099
paradox1x.org1085
www.quora.com589
stackoverflow.com481
schoolofdata.org364
dev.lethain.com272
webscraping.com252
feeds.delicious.com137
www.codeproject.com111
www.mozenda.com110
www.delicious.com96
news.ycombinator.org85
sitescraper.net75
es.schoolofdata.org65
ask.metafilter.com61
wiki.greasespot.net47
hackerne.ws41
twitter.com36
searchyc.com32
mail.python.org32
www.instapaper.com31
realtime.michaelhart.me30
hckrnews.com25
www.hnsearch.com22

All Rights Reserved, Will Larson 2007 - 2014.