Using Flot.js with the Digg Streaming API
Last year Digg released our Streaming API, but there still aren’t too many tutorials out there, especially on using it for JSONP. I’ve also wanted to experiment with more dynamic tutorials which run JavaScript examples in-line with the article. So, time to scratch some itches.
In this tutorial we’ll look at using Flot, jQuery and the Digg Streaming API to create dynamic graphs.
Install & Setup Flot
First we’ll need to download Flot (which comes with a copy of jQuery).
curl http://flot.googlecode.com/files/flot-0.7.tar.gz > flot-0.7.tar.gz
tar -xvzf flot-0.7.tar.gz
Next, create an HTML page in the same directory which these contents:
:::html
<html>
<head>
<title>Flot & Digg Streaming API</title>
</head>
<body>
<div id="placeholder" style="width:600px;height:300px"></div>
<!--[if lte IE 8]>
<script type="text/javascript" src="excanvas.min.js"></script>
<![endif]-->
<script type="text/javascript" src="flot/jquery.min.js"></script>
<script type="text/javascript" src="flot/jquery.flot.min.js"></script>
<script>
$(document).ready(function() {
var data = [[[0,0],[1,1]], [[0,1],[1,0]]]
$.plot($("#placeholder"), data, { yaxis: { max: 1 } });
});
</script>
</body>
</html>
If everything has downloaded correctly, you should see this example:
Digg Streaming API
The Digg Streaming API makes it possible to see all Diggs, comments and submission occuring on Digg.com in real-time (generally a second or two after they occur, but occasionally a bit longer).
It is available at
http://services.digg.com/2.0/stream
and takes a variety of parameters. For our purposes we’ll want to use its JSONP
capabilities to perform requests from across a different domain. To do so we’ll need to specify format=javascript
as well as callback=some_function. All put together the URL will look like:
http://services.digg.com/2.0/stream?types=comment
&format=javascript
&callback=streaming_callback
With that knowledge in mind, let’s create a simple script which retrieves recent comments and renders the them. Let’s use this HTML fragment as the template for rendering comments:
:::html
<div id="digg-comment">
<span id="digg-author">some author</span>
<p id="digg-body">some text</span>
</div>
Performing JSONP requests to the Digg Streaming API requires a bit of extra tact,
it isn’t enough to simply use $.getJSON("http://..."), you’ll actually need
to write a simple wrapper looking like:
:::javascript
var get_stream = function(callback_func, types) {
if (types == undefined) {
types = "digg,submission,comment";
}
var domain = "services.digg.com",
format = "format=javascript",
types = "types=comment";
var url = "http://"+domain+"/2.0/stream?"+types+"&"+format;
$.ajax({url:url,
dataType:"jsonp",
crossDomain:true,
cache:true,
success:callback_func});
};
More specifically the $.getJSON utility function for JSONP won’t work because it attempts to pass extra parameters
which the Digg Streamin API doesn’t know about and will reject (specifically jQuery passes the _ parameter
with the current time as its value as an attempt to bypass caching).
Armed with get_stream we can whip together the code for populating the HTML comment
pretty quickly.
:::javascript
var get_comment_stream(callback_func) {
get_stream(callback_func, "comment");
}
var stream_callback = function(data) {
var fullname = (data.user.fullname) ? data.user.fullname : data.user.name,
txt = data.text.substring(0, 45) +
"<a href=\"" +
data.item.href +
"\">…</a>",
author_str="<a href=\"http://digg.com/" +
data.user.name+"\">" +
fullname +
"</a>";
$("#digg-author").html(author_str);
$("#digg-body").html(txt);
// schedule a request for retrieving the next comment
setInterval("get_comment_stream(stream_callback)", 100);
}
$(document).ready(function() {
get_comment_stream(stream_callback);
}
Once you put those pieces together, it looks a bit like this (well, the version running here has a ten second delay between fetches to avoid wasting too much of your bandwidth, and also stops after sixty requests):
some comment
Okay, so now that we’ve put together an example of using the Streaming API and have Flot.js installed, let’s take a stab at using them together.
Dynamicly Updating Flot
For our example of updating Flot with real-time data from the Streaming API, let’s create a chart which will track the number of submissions, comments and Diggs each minute over time.
As usual, the HTML will be trivial.
:::html
<div class="js-example" id="exampletwo"></div>
The JavaScript will be a bit more complex this time, but the JSONP
aspect will be handled by reusing the get_stream function defined
above.
:::javascript
var chart_data_by_minute,
chart_data,
options = {
lines: { show: true },
points: { show: true },
xaxis: { tickDecimals: 0, tickSize: 1 },
yaxis: { tickDecimals: 0, tickSize: 1 },
legend: { backgroundOpacity: 0, position: "se" }
},
graph_callback = function(data) {
var curr_minute = (new Date()).getMinutes();
curr_data = (chart_data_by_minute[curr_minute]) ?
chart_data_by_minute[curr_minute] :
{'submission':0, 'digg':0, 'comment':0},
minutes = [],
series = { "submission":[], "comment":[], "digg":[] };
// update historical data with latest event
curr_data[data.type]++;
chart_data_by_minute[curr_minute] = curr_data;
// find all minutes in historical data and sort them
$.each(chart_data_by_minute, function(key, val) {
minutes.push(parseInt(key));
});
minutes.sort();
// generate event-type series for flot
$.each(minutes, function(index, minute) {
$.each(["submission", "comment", "digg"], function(index, x) {
series[x].push([minute, chart_data_by_minute[minute][x]]);
})});
// the real magic is here, where we recreate the series
// every time we get a new event and rerender the graph
chart_data = [{label:"submission", data:series["submission"]},
{label:"comment", data:series["comment"]},
{label:"digg", data:series["digg"]}];
$.plot($("#exampletwo"), chart_data, options);
// schedule a request for retrieving the next comment
setInterval("get_stream(graph_callback)", 100);
};
// get the party started
$(document).ready(function() {
chart_data_by_minute = {};
chart_data = [];
get_stream(graph_callback);
});
Put it together, and you have a real-time graph of comments, Diggs and submissions occuring right now on Digg.
Waiting a
Note that this really isn’t an accurate reflection of the total quantity of activity happening on Digg because of the way we’re doing the intervals and also because we’re missing events. The single-threaded and callback nature of JSONP, along with the lack of support for cursors and bulk response type for all activity since the last cursor mean that you’ll need to investigate establishing a persistent to the Digg Streaming API if you really want to see everything. (If you’re interested, take a look at this repository on Github for Python examples.)
Certainly not the prettiest of graphs, or the most useful of examples, but working directly with JavaScript examples embedded in the tutorial was certainly an interesting experience, and hopefully slightly more useful than simply seeing screenshots.
As always, hope this was useful, I apologize for any awkwardness in the new format, and let me know your feedback!