Buster McLeod: Using Bloglines to manage my online presence

« Outsourcing the blog | Main | My two favorite conferences overlap »

Wednesday, 16 February 2005

Using Bloglines to manage my online presence

I'm still tweaking the way the new erikkregator works, especially since the timestamps get a little out of whack coming from all of the different sources. But, because I want to have something to link to to explain what I did, this is the order of thoughts in my head that led me to build what I built (one more time):

1) I used to try to keep all of my stuff in a single blog. Blog as defined by a single authoritative database table displayed in reverse chronological order on a single URL.

2) This setup worked for a lot of things. Movable Type was nice because it allowed multiple blogs to co-exist semi-nicely. For example, I set up a blog for camera phone pictures, one for quick links, one for longer posts, etc. Then, I pulled in flickr photos and 43 Things posts via post-to-blog features on the respective sites when desired, and pulled in books with All Consuming's javascript include, and pulled in upcoming through a Movable Type RSS plugin.

3) A lot of duct tape was required, as while it's awesome that most sites (that I'm interested in) have opened up their data to allow sharing of some sort, every site allowed sharing in a slightly different manner, and required me to step out of my flow on those sites to make sure that I got my stuff collected back on my blog as well.

4) I realized that Bloglines had become my own personal internet. I wasn't using my blogroll anymore, nor was I actually visiting sites much... instead I was reading most of my content through Bloglines folders that subscribed to the RSS feeds of sites that I was interested in.

5) Almost every bloggy site has an RSS feed now.

6) Bloglines is good for keeping up with other people, but I realized that somewhere along the line I had begun to use it to keep up with myself as well. In a way, I was using it to keep track of my own work around the web so that I could find it again when I wanted.

7) Bloglines has an API, aka a web service that lets me retrieve content from my account via a script rather than through the browser.

8) I haven't played around with the Bloglines web service yet.

9) I consolidated all of my personal feeds (many from Movable Type, others from places like Ta-da List, Flickr, Live Journal, and 43 Things) into one Bloglines folder.

10) I build a quick Ruby script to pull in entries from that server, in chronological order. I store these entries in a local database, and display them with my favorite web application framework in the world, Ruby On Rails.

11) RSS is already normalized. Bloglines also does some cleaning up for me. What I get back from Bloglines is beautiful content that looks the same no matter which site I originally posted it on. Since everything goes through this filter, there's really no need to have access to the database of content directly. I therefore decided to move all of them off my server and to make Bloglines the mesh net through which I collect all content.

12) I exported my Movable Type blog (already incarnation number three million since I started writing 6 years ago) over to Typepad. I moved all of my Morale-O-Meter links over there too and decided to start using del.icio.us a lot more religiously. I love del.icio.us. I moved all of my photos over to Flickr, and then added a few more sites that I had never been able to integrate into my blog like Live Journal and Ta-da Lists.

13) Now, I don't have to worry about backups, performance of my crazy little scripts, maintaining my email server to correctly handle posts via email, or really any of the stuff that I was never very good at to begin with. I love end results, and that's what I get when people who are good at what I want to do are doing the things I want to do better than I ever could.

I like thinking about the layers of computers between you and me. I write something somewhere, it gets stored in a database, then represented as RSS. Then it gets picked up by Bloglines, normalized, stored in a database, and represented again as RSS. Then it gets picked up by my script (which runs every 15 minutes), parsed, and stored in another database, and represented as RSS. Then it gets picked up either by your browser, or by Feedburner, where it again gets parsed, stored, and redisplayed before being parsed, stored, and redisplayed in your favorite RSS reader. Depending on how you are reading this, that's between 9 and 14 transformations along the way. And that's not even counting the parsing, storing, interpreting, and comprehension layers in my brain before I write the thoughts down, and in your brain after you read the words. It's amazing that we can communicate at all.

I'm at least 3.5 times happier as a result of all this fun. And too boot I don't have to worry about comment spam during this fun honeymoon period before the spammers all find my new homes. I'm enjoying it while it lasts.

01:42 AM in Weblog | Permalink

Comments

This is one seriously cool mash-up you've put together. I for one would be very interested to know the details of your Ruby script and Rails setup. If you feel like sharing, please post more details.

Posted by: Carrick Mundell | Wednesday, 16 February 2005 at 10:02 AM

Bloglines web services are pretty basic. Here's some code showing how I retrieved entries from a particular folder (I had to run a different method to find the ID of my folder):

require 'net/http'
require "base64"
require "rexml/document"

# Bloglines info
my_id = '[email protected]'
my_pwd = 'my-password'
auth = "Basic " + ["#{my_id}:#{my_pwd}"].pack("m").strip

site = 'rpc.bloglines.com'
port = 80

req = Net::HTTP.new( site, port )
raw_xml = ""

d = Time.now.to_i - 86400
req.get( "/getitems?s=7138308&n=0&d=#{d}", 'Authorization' => auth ) do |r|
break unless r
raw_xml += r.to_s
end

Then I parse the XML using REXML::Document, and store the pieces I want in the database.

The Rails setup basically has a model for Entry that has a title, description, link, created_on, bloglines_id, and guid. I have one controller (front_controller) for displaying the entries, and another (report_controller) for building the daily and monthly reports. Is that what you wanted to know? I'd be glad to go into more detail if it would help... how new are you to Ruby and Rails?

Posted by: Erik Benson | Wednesday, 16 February 2005 at 10:42 AM

This is great stuff Erik. For those of us who are either not developers or too lazy to put enough effort in to be developers (I think I fall into both camps), do you think the process you outlined above could be wrapped up into an easy-to-deploy 'web service'? Maybe this would need to be hosted in order to make it easy for people to use - it may even be a candidate for a Robot Co-op service? (ie host it on the Robots server). It's such a neat way to bundle a whole lot of content 'buckets' into one stream of content, that it would be really cool if lots of people could use it. Just an idea and I'm not sure how feasible or even desirable it is. Either way, your use of the Bloglines API is admirable.

Posted by: Richard MacManus | Wednesday, 16 February 2005 at 11:47 AM

Thanks Richard! I did think about making this a bit easier to share with others... the right solution didn't jump out at me but I think it would be a fun experiment. In a sense, bloglines is already doing all of the hosting necessary... it's just a matter of displaying what they have. The only problem is that the web service isn't always zippy, and therefore it's better to cache the results and only check every once in a while (he probably wouldn't like me hitting them on every page load anyway).

I'll think about it a bit more and see if there's some kind of answer. Kinja ( http://kinja.com ) is one possibility... they basically do the same thing, but after fiddling with them a bit I didn't quite like the way they excerpt everything.

Posted by: Erik Benson | Wednesday, 16 February 2005 at 05:10 PM

Good one. It would also be a nice value-add for 43 Things - a way for people to add their 43 Things content into their weblogs. The killer value for users would be the ability to group *everything* though - blog, 43 Things, delicious, flickr, allconsuming, etc.

ps have you see what this guy is doing with Bloglines Web Services:
http://joshtyler.com/test/myblogs.html

It's called Chameleon and there are some interesting personalisation featues he's managed to build.

Posted by: Richard MacManus | Thursday, 17 February 2005 at 12:56 AM

Thanks for the specifics, and it's enough to know that I don't know enough of what you're doing to implement this on my own. I'm with Richard on the idea of making this into some kind of web service. It strikes me as a personal feed splicer, like FeedBurner on steroids with a bloggy front-end. The concept of a personal aggregator that looks like a web site is very appealing. You've sparked my curiosity greatly.

Posted by: Carrick Mundell | Thursday, 17 February 2005 at 12:48 PM

Hey. Do you have an updated copy of your NaNo report card? The copy I have gave a link to get an update, but the link turned up a dead end.

Thanks.

Posted by: Carly | Saturday, 02 July 2005 at 10:48 PM

Soery please :(
Wrong categoryh...

iwll be caeeufl

Posted by: Traibpydrassy | Monday, 25 June 2007 at 05:29 PM

Hi.
Nice site bro.
http://www.viagraprice.org/>viagra

Posted by: superdoc | Wednesday, 27 June 2007 at 04:58 PM

Posted by: wrix | Wednesday, 27 June 2007 at 09:15 PM

the best sport nutrition
http://www.maxsport.ee

Протеины Гейнеры Креатин Аминокислоты

хотите стать сильнее читайте и узучайте

Cжигатели жира Витамины low carb protein proteiin creatiin creatin amino
best body nutrition cytogen universal Nutrition Weider biotech
peak perfect fitnes bodybuilding buy order try in forum gallery eff wbbf
Eesti Fitnessi Foderatsioon stack lava hard fast fat burner Whey Protein
Premium pro CytoPro Milk & Egg Protein Crash 5000 Hyper Mass 5000
Creatin Monohydrat XXL Booster + BCAA Amino 5000 Aminoliquid
Amino Liquid - SK L-Glutamin Kapseln L-Glutamin Pulver xxl booster creatin
L-Carnitin Kapseln L-Carnitin Liquid L-Carnitin Liquid Lecithin Granulat
protein valk gym box body
http://www.maxsport.ee>read more about...

Posted by: seiGera | Wednesday, 04 July 2007 at 02:26 PM

http://top-adult-web-cam-614sf1.spycamssite.info "> top adult web cam top adult web cam
http://top-web-cam-chat-614sf1.supercamsusa.info "> top web cam chat top web cam chat
http://top-live-web-cam-614sf1.hiddenlimocams.info "> top live web cam top live web cam
http://top-sex-web-cam-614sf1.spycamsgear.info "> top sex web cam top sex web cam
http://top-rated-web-cam-614sf1.thecamsnow.info "> top rated web cam top rated web cam
http://top-web-cam-chat-room-614sf1.camerascams.info "> top web cam chat room top web cam chat room

Posted by: MatiCamsf | Thursday, 05 July 2007 at 03:43 PM

Cool site!!!

http://live-girls-cam-614f1.hiddenlimocams.info "> live girls cam live girls cam
http://live-granny-cam-614f1.spycamssite.info "> live granny cam live granny cam
http://live-hot-cams-614f1.girlcamsworld.info "> live hot cams live hot cams
http://live-jail-cams-614f1.camerasera.info "> live jail cams live jail cams
http://live-nude-cam-614f1.spycamssite.info "> live nude cam live nude cam
http://live-nude-cams-614f1.camerasera.info "> live nude cams live nude cams

Posted by: KelyCamse | Thursday, 05 July 2007 at 05:23 PM

Cool site!!!

Posted by: KelyCamse | Thursday, 05 July 2007 at 05:24 PM

Thank you!
breakfree holiday specials http://om5408752nt.h16.ru ">3 bedroom holiday house cairns

Posted by: Binujero | Saturday, 07 July 2007 at 02:28 PM

Sponsor Results are sites that pay for placement in search results on terms that are relevant to their business.
http://virginspussy.infinites.net

Posted by: virginspussy | Wednesday, 11 July 2007 at 02:46 PM

The importance of utilizing personal http://lajasper.4sql.net>Tax Attorneys is not recognized by most business owners, not the same way they understand the importance of employing an office manager or accountant.

Posted by: mamAnnorm | Wednesday, 11 July 2007 at 10:55 PM

I agree.
http://pussylicking.hub.io

Posted by: pussylicking | Thursday, 12 July 2007 at 05:22 AM

Today the lottery is as integrated with our lives as the internet or chocolate. Although in this modern, fast paced world where we simply don't have time to mess around, we have chopped a full syllable off of the word, and just refer to it as lotto. Despite this questionable phonetic practice, lotto is part of the daily life in a large chunk of the world.

http://gente.chueca.com/world-lotto/lottery.html>lotery results

Posted by: hipoym | Tuesday, 17 July 2007 at 05:17 PM

It is match.

Posted by: hairypussy | Friday, 20 July 2007 at 03:23 AM

Interesting game about the submarine! Very long and with the not bad drawing.

Posted by: Maks | Friday, 20 July 2007 at 07:18 PM

Thomson holiday http://mycar.imess.net/page=137.html ">Thomson holiday

Posted by: Njwiyero | Monday, 23 July 2007 at 04:45 AM

http://www.hyip-catalog.com

I want to invest money online.
On a site written that it is the program which helps make money online.

You invest online?

It really works?

This site:
http://www.hyip-catalog.com

Posted by: acarturce | Wednesday, 25 July 2007 at 02:15 AM

Sie leben nur einmal - warum dann nicht was neues ausprobieren?

Preise die keine Konkurrenz kennen

- keine versteckte Kosten
- Kostenlose, arztliche Telefon-Beratung
- Bequem und diskret online bestellen.
- Diskrete Verpackung und Zahlung
- Kein peinlicher Arztbesuch erforderlich
- Kein langes Warten - Auslieferung innerhalb von 2-3 Tagen
- Visa verifizierter Onlineshop

Originalmedikamente
Cialis 10 Pack. 27,00 Euro
Viagra 10 Pack. 21,00 Euro

Nur fur kurze Zeit - vier Pillen umsonst erhalten

http://BestPharmSpecial.Com

http://bestpharmspecial.com/>BestPharmSpecial.COM