Parsing Yahoo Pipes JSON Feeds with jQuery
Posted: Sat 07/07/2007 by ramiroIn this article I describe how you can parse and display Yahoo Pipes data returned as a JSON feed. Yahoo Pipes can be used as a data aggregator that lets you mashup online data sources, such as RSS feeds. To learn more about Yahoo Pipes and how to create them you can watch this video on how Yahoo Pipes work.
This first thing I did was to create a Drupal News Pipe that aggregates 5 Drupal related RSS feeds. This Yahoo Pipe is derived from Harry Slaughter's pipe called Drupal News but uses less and partly different data sources and does not exclude feeds based on word filters.
When a pipe is created and published you can access it on the pipe's page, subscribe to the RSS feed of the pipe or get the data as a JSON (JavaScript Object Notation) string. Since I wanted to display the feed data using JavaScript, JSON was the obvious output format choice.
Code Snippets and Explanations
Now let's start with the actual code. To display the feed data I created a very simple HTML page as shown below called index.html:
<html>
<head>
<title></title>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="pipes.js"></script>
</head>
<body>
<div id="pipes-feed-content"></div>
</body>
</html>There are 3 important things to note. The inclusion of the jQuery JavaScript library, the inclusion of the pipes.js file explained below, and a div with the id pipes-feed-content used as a container for displaying data returned from the aggregated data sources.
To access the JSON data I wrote an even simpler PHP script that retrieves and prints the JSON string using the function file_get_contents() that requires the PHP fopen wrappers to be enabled. Some ISPs disable them, so if you can't use this example code on your server this is likely to be the reason. An alternative would be to use PHP's cURL functions, which require libcurl to be installed:
<?php
print file_get_contents(
"http://pipes.yahoo.com/pipes/pipe.run?_id=LFtDHHMs3BG_Un2LJhOy0Q&_render=json"
);
?>This PHP script is called via an AJAX request in the pipes.js file, that we will examine now. This code is a little longer but thanks to jQuery 50 lines of JavaScript code suffice to parse and display the feed data.
/* global variables */
var descList = new Array();
var itemList = "";
$(document).ready(function() {
var content = "";
$.getJSON("pipes.php",
function(json){
if(json.count > 0) {
content = output_feed_items(json);
} else {
content = "The request did not return results.";
}
$("#pipes-feed-content").html(content);
}
);
});I defined two global variables: descList is an array, where each array field stores the description of one feed item; itemList contains a list of "linked" feed headings. When the document is loaded the statements in the $(document).ready(function()) will be executed. $.getJSON performs an asynchronous request to the PHP script. The second argument is a callback function that will be executed with the returned data as it's argument.
As mentioned the PHP script returns a JSON string available via the json variable. To test whether the request returned any feed items the value of json.count is checked. If there are feed items output_feed_items() is called with the JSON string, else the content variable is set to an appropriate message.
function output_feed_items(json) {
document.title = json.value.title;
var heading = '<h3>' + json.value.title + '</h3>';
for (i=0;i<json.count;i++) {
itemList += make_feed_item(json.value.items[i], i);
descList.push(make_feed_desc(json.value.items[i], i));
}
return heading + itemList;
}After setting the current document's title and heading to the title of the Yahoo Pipe each feed item is iterated in a for loop. To fill the itemList variable the make_feed_item() function is called with the current feed item, and item id and then the return value of the make_feed_desc() function called with the same parameters is appended to the end of the descList array.
function make_feed_item(item, item_id) {
return '<h4 id="heading-' + item_id + '">' +
'<a href="#heading-' + item_id +
'" onclick="toggle_feed_desc(' + item_id + ');">' +
item.title + '</a></h4>';
}This function simply returns the feed item title wrapped in h4 tags, "linking" to a JavaScript function call of the toggle_feed_desc() function, passing the feed item id. The heading also gets an id to make it simple to show and hide the description in the toggle_feed_desc() function.
function make_feed_desc(item, item_id) {
var desc_info = '<span="item-submitted">Published: ' +
item.pubDate + '</span>';
desc_info += ' - <a href="' + item.link + '">Link to Article</a>';
var desc_info = '<div class="item-info">' + desc_info + '</div>';
return '<div id="desc-' + item_id + '">' +
desc_info + item.description + '</div>';
}The make_feed_desc() function returns the feed item description including the publication date, the link to the original article, and the feed's description, which can be a teaser or a full body depending on the RSS settings of the feed's originator. The div containing all this markup also gets an id to easily make it accessible to jQuery's DOM parsing functionality.
function toggle_feed_desc(item_id) {
var heading = '#heading-' + item_id;
var item_div = 'div#desc-' + item_id;
if ($(item_div).html()) {
$(item_div).remove();
} else {
$(heading).after(descList[item_id]);
}
} Last but not least the toggle_feed_desc() function is called with the feed item id when a visitor clicks on a heading link. It checks whether the corresponding description is currently displayed, if so it will be removed from the DOM tree. If not the description will be displayed after the appropriate heading by calling jQuers's after() function on the clicked heading with the HTML output contained in the value of the descList array field with the same id.
Summary
This little tutorial has shown how JSON data returned by the Yahoo Pipes Web Services API can be easily parsed and displayed using the powerful JavaScript library jQuery, some HTML, and even less PHP.
Links
Complete JavaScript Code
/* global variables */
var descList = new Array();
var itemList = "";
$(document).ready(function() {
var content = "";
$.getJSON(script_path,
function(json){
if(json.count > 0) {
content = output_feed_items(json);
} else {
content = "The request did not return results.";
}
$("#pipes-feed-content").html(content);
}
);
});
function output_feed_items(json) {
document.title = json.value.title;
var heading = '<h3>' + json.value.title + '</h3>';
for (i=0;i<json.count;i++) {
itemList += make_feed_item(json.value.items[i], i);
descList.push(make_feed_desc(json.value.items[i], i));
}
return heading + itemList;
}
function make_feed_item(item, item_id) {
return '<h4 id="heading-' + item_id + '">' +
'<a href="#heading-' + item_id +
'" onclick="toggle_feed_desc(' + item_id + ');">' +
item.title + '</a></h4>';
}
function make_feed_desc(item, item_id) {
var desc_info = '<span="item-submitted">Published: ' +
item.pubDate + '</span>';
desc_info += ' - <a href="' + item.link + '">Link to Article</a>';
var desc_info = '<div class="item-info">' + desc_info + '</div>';
return '<div id="desc-' + item_id + '">' +
desc_info + item.description + '</div>';
}
function toggle_feed_desc(item_id) {
var heading = '#heading-' + item_id;
var item_div = 'div#desc-' + item_id;
if ($(item_div).html()) {
$(item_div).remove();
} else {
$(heading).after(descList[item_id]);
}
}- Login to post comments
Yes, of course the feed is updated. Though it may be possible that some caching takes place on the side of Yahoo! Pipes.
This looks like a different problem. Maybe this forum thread can help you http://bugs.php.net/bug.php?id=11058
Yes take a look at the Drupal news page on this site, where the pipe output is displayed.
Your link will return the feed as a JSON string.