View source for How to set up a data stream

__NOTOC__
Note: I've copied the following information from an article I was preparing for one of my blogs. The approach is wrong, because it starts from examples that are much too specific and detailed, whereas it should start from more general principles.

==Howard Rheingold's chart of Information Filters==
Internet pundit [http://en.wikipedia.org/wiki/Howard_Rheingold Howard Rheingold] has drawn together instructions from many sources on how to set up Internet information filters. He presents these various sources [http://cmapspublic3.ihmc.us/rid=1GGG55974-H3VQRB-J1D/Infotention%20Filters.cmap as a concept map]. Let us work our way through Howard's diagram, starting with Intelligence Dashboards.

==Intelligence Dashboards==
In Howard's chart, the first link under "Intelligence Dashboards" goes to [http://www.metafluence.com/integrating-netvibes-pipes-aiderss-dapper-for-an-intelligence-dashboard Justin Kirster's instructions] for creating a Dashboard. (The method Justin explains was in fact [http://gobigalways.com/my-secret-is-a-dashboard-for-each-side-of-the-firewall/#comment-58 developed by Sam Laurence)]. Justin finds it easiest to work in this order:
*Create Dapper feeds
*Aggregate and manipulate feeds with Pipes
*Run outputs through AideRSS
*Import feeds and configure modules in Netvibes

Let us take look at these one at a time.

==Dapper==
Justin's Dashboard relies heavily on [http://en.wikipedia.org/wiki/Rss RSS], which is a [http://en.wikipedia.org/wiki/Web_feed web feed]  format which informs you when someone has posted new content to the Internet. However, there is a lot of data on the Internet that does not come as RSS feeds. Dapper helps to process unstructured information from websites (in HTML format), to clean it and transform it into structured XML.

The use of Dapper is [http://www.readwriteweb.com/archives/dapper_quest_to_unlock_web_data.php described in a post] by Alex Iskold. If you go to the [http://www.dapper.net Dapper website], it is at first unclear where you should click to find the data collection tool. Go to the bottom right of the page where you will find a link to the [http://www.dapper.net/dapp-factory.jsp Dapp Factory]. That link opens up a form in which you can enter the [http://en.wikipedia.org/wiki/Url URL] of the website you are interested in.

==Pipes==

==AideRSS==

==Netvibes==

==Bonus: Marsh Gardiner's Persistent Data Searches==
In answer to a question by Rheingold, Marsh Gardiner posted instructions on [http://blog.earth2marsh.com/2009/08/easy-way-to-watch-web.html how to create persistent data searches] using Google.

[[Category:Draft]]