In the comment thread of a previous post on this site, Chris Taggart made an interesting criticism of my Facebooker Facebook API. It seems that, on a high level review of the code, he found it to be well-craft and well-tested. His criticism was that, unlike in RFacebook, I chose to implement a concrete set of Ruby classes to abstract the underlying workings of the Facebook XML API away from Facebooker’s users. RFacebook, on the other hands, employs a trick using Ruby’s method_missing, to catch undefined method calls and to transform those method calls into HTTP posts which conform to Facebook’s HTTP/XML endpoints. Here’s what an RFacebook call might look like (from the RFacebook Web site):
friendUIDs = fbsession.friends_get.uid_listfriendNames = []
friendsInfo = fbsession.users_getInfo(:uids => friendUIDs, :fields => ["first_name", "last_name"])
friendsInfo.user_list.each do |userInfo|
friendNames << userInfo.first_name + " " + userInfo.last_name
endThis results in two Facebook XML API calls. The two method calls on the fbsession object are undefined, so fbsession converts them to HTTP posts to the Facebook XML API methods called “facebook.friends.get” and “facebook.users.getInfo” respectively. Furthermore, when “uid_list”, “user_list”, “first_name”, and “last_name” are called, RFacebook again employs a method_missing trick to generate an XPath expression searching for the requested data in an HPricot DOM.
Here’s the equivalent Facebooker code:
friend_names = session.user.friends!(:first_name, :last_name).map do |friend|
"#{friend.first_name} #{friend.last_name}"
endIn this code, the same API calls happen as in the RFacebook example, only we don’t explicitly invoke them.
Chris’s (albeit tentative) concern was that, since Facebooker implements objects and their methods as first-class Ruby constructs, the API would somehow be more brittle and less resilient to potential future Facebook XML API changes.
I found it especially interesting that Chris chose this as his only point of criticism, since RFacebook’s method_missing trick and return of an HPricot XML parser object from each API call were two of the key reasons we decided to scrap RFacebook after writing an application with it and implement our own Facebook wrapper library.
As I started to formulate a response, instead of taking my own motivations for granted, I decided to take some time to question my own assumptions. This all led me to question (and to answer for myself) why we write this sort of wrapper library to begin with. What follows are my answers.
Perhaps the primary reason for building a concrete layer on top of a low-level API like Facebook’s is to make the code which uses it less brittle. The scenario Chris was concerned with was that Facebook might change their API, requiring Facebooker to be changed to match it. My point exactly, Chris. If Facebooker didn’t hide the implementation details of the XML API from its end-users, a change in the XML API would require every application which uses Facebooker to change. Magic missing method calls would have to be renamed or changed. If “facebook.users.getInfo” were to be changed to “facebook.user_info”, the calls to “session.users_getInfo()” would need to be changed to “session.user_info()”.
Sure, you could hack the API so that “users_getInfo()” and its parameters were munged into Facebook’s hypothetical new format, but then our API would pretend to be low-level and direct, which would lead to some opaque application code.
When I’m writing a Rails application controller or the controller for a desktop application (both of which I have been doing with Facebooker), I want to think in terms of the domain I’m modeling. In a Rails application, it’s commonly accepted that fat models and skinny controllers lead to well-factored, expressive, and maintainable code.
This is partially because, in the controller layer of an application like this, the code’s job is to (as Marcel Molina said to me recently) codify a dialogue between the domain objects in a system. In other words, when you’re developing an application about users, their friends, their affiliations, and the groups they belong to, the controller should deal with those concepts. It doesn’t make sense to deal with, say, a user, a database connection, the user’s friends, their XML representation, a shared photo album, and HTTP connectivity issues all in the same code.
My brain doesn’t want to jumble all of that together. When I’m trying to look at XML parsing code, domain logic gets in the way. When I’m trying to see how users and their friends interact, XPath is line noise.
Kent Beck refers to this in his Smalltalk Best Practice Patterns as a guiding principle of good API design, specifically having to do with how to decompose your code into methods. His rule of thumb is that one way you know it’s time to create a new method is when the code in one method mixes two or more levels of abstraction.
By abstracting the implementation details of XML and HTTP away from the high level of the API, code becomes more testable. In this case, it’s true for Facebooker itself, but more importantly, the applications that use Facebooker are more testable with a layer of abstraction between them and the XML and HTTP calls which are going on under the covers.
When you’re trying to test a Rails action, it’s much easier to create a User object and set its attributes (first_name, last_name, etc.) than to generate a DOM of sample data. Sure, Ruby being as dynamic as it is, you could create a stub at runtime and forego dealing with the library’s implementation of User altogether. But you shouldn’t have to.
It may seem like a nitpick, but as a Ruby programmer I want to use APIs that look like Ruby code. “fbsession.users_getInfo()” looks like PHP code to my eyes. It’s no surprise. Facebook is written in PHP, and its HTTP/XML API was designed by the same PHP programmers that created Facebook. Wrapping the Facebook API allows me to isolate and hide the PHP idioms, such that the code I use reads like “normal” Ruby code. You could argue that this is not a technical issue, and you’d be right. But I believe in the power of both consonance and dissonance in software development. Dissonant code stands out and alerts the reader that something strange and exceptional is taking place. Sometimes that strange and exceptional thing is just bad code. Sometimes it’s an unusual technique that should be highlighted.
When I see non-idiomatic Ruby code, it tells me one of two things. The first assumption I make is that whoever wrote the code is not a Ruby programmer. That’s usually the case. The second is that the programmer is employing a new or unusual technique to accomplish something which is difficult to do without that technique.
Having implemented a wrapper for the Facebook API, I don’t think there’s a need for dissonance. There’s nothing difficult or unusual going on at any level of Facebooker, so it stands to reason that there should be no dissonant section of the library and that it should all read as idiomatic Ruby code.
I love the tricks you can do with Ruby. method_missing, const_missing, autoloading, and their friends make really powerful things possible.
But they do so at a price. When something goes wrong in a piece of code that relies heavily on one of these tricks, it can be much much harder to track down. So the decision to use such a tool shouldn’t be taken lightly. These are power tools. Used effectively, really cool things can happen. Used incorrectly, you can easily find yourself limb-less and bloody. So when you decide to use one of these power tools, you have to ask yourself: is it worth the risk?
Where possible, especially in a library already dealing with something abstract and out of our control (e.g. someone else’s XML API running on their own HTTP servers of which we have no direct visibility), explicit and concrete are much better attributes than abstract and fuzzy. If you run into a problem or question with part of a library for which you have the source code, explicit method definitions and concrete APIs under the covers are much easier to navigate than parsed method names magically constructing arguments to HTTP posts.
In general, if you’re dealing with code which is out of your control, implicit and hidden from view, it’s helpful if the code which surrounds it balances the abstractness with a nice concrete anchor.
This is a summary of many of the usually-abstract ideas that go through my head when I’m creating APIs that wrap other APIs. I’ve been designing wrapper APIs like this for ages, so it’s been helpful for me to convert intuition into a set of hints. I hope these ideas are helpful to others. If nothing else, if you ever have to use an API I’ve written, you’ll at least know why it’s written the way it is.
Sorry, comments are closed for this article.
September 5th, 2007 at 03:36 AM
Great post, but where’s the facebooker code? :)
September 5th, 2007 at 03:46 AM
Theron, http://facebooker.rubyforge.org/svn/trunk/facebooker/
September 5th, 2007 at 05:08 AM
I was just about to start evaluating Ruby Facebook libraries this evening when I came across this superb post. Thanks, the delicious is easy now.
September 5th, 2007 at 05:09 AM
I may have meant ‘decision’ but I’m quite happy with how that turned out.
September 5th, 2007 at 05:13 AM
Chris: that is hilarious.
September 5th, 2007 at 05:43 AM
Well I wish I’d known about Facebooker last Wednesday. And after a few days muttering RFacebook’s approach, I’m with you on pretty much all points.
September 5th, 2007 at 07:41 AM
Greetings, Huh! Nicely written… I just finished building a Rails web-service and a concrete Ruby client API to talk to it at work, eschewing the generic auto-generated RESTafarian client-side interface for a concrete API. (To a very REST-y web service, though.) I didn’t consciously think about it, except to note to myself how badly all the auto-generated/method_missing-based REST APIs we’ve used turned out in the end, but all the points resonate strongly.
There’s a lot of power in being able to write down the knowledge that underpin the unconscious decisions that we make in working our craft.
Good article.
— Morgan
September 5th, 2007 at 10:21 AM
Chad
Thanks for picking up on my comment. Thought I should just come back to you on a couple of things, in part to clarify a few things, and in part because I do think there are a few things worth discussing here (though, as a relative novice, I may be wrong).
The first is that I’ve got nothing to do with RFacebook, and I wasn’t comparing your library with that
- like you, I had a problem wih the un-Rubyish language, and the approach it took. I’m also not a big fan of the catch-all method_missing approach -IMHO it makes applications behave strangely and makes bug-catching a bit of a nightmare.Also I mentioned in my comment, I’ve not had a proper review of the code (though it’s clear well-written and thought out, and, crucially, tested), but the use of concrete classes was the thing that stood out, not because it’s different to RFacebook, but because, well, it’s a different approach to a lot of Rails stuff, which (to a perhaps uninformed person like me) seems to be moving towards a wrapper approach rather than an API one. I’m not sure I’m articulating this well, but I hope you get what I mean.
Mostly that comment was written because I was surprised at the approach rather than dogmatically felt it was the wrong one, and wrote: “I wonder if there?s perhaps a danger there, both from changes in the Facebook API (I suspect they won?t give the notice that, say, Amazon will of API version deprecation), and from the fact that users need to understand a slightly different API from Facebook (and therefore will need to rely upon the community using this library rather than the wider Facebook community).”
I do, however, think there’s a worthwhile debate here
- in fact there are a couple. the least interesting is probably directly to do with a Facebook library. It seems that we both had big problems with RFacebook -you took one approach, and I took another, with a very lightweight library that’s about a 10th the amount of code of RFacebook, doesn’t need HPricot, and, mostly importantly, did what I needed (http://pushrod.wordpress.com/2007/08/14/rails-ruby-facebook-and-tests-my-own-itch-scratched/).The bigger (and IMHO more interesting) issues here are:
1. Big concrete library where everything is done for you Vs lightweight library which makes life easier but still requires some input. I don’t really want to debate this, partly because I’d lose any debate with you (I’m a self-taught novice who hasn’t read nearly as widely), and partly because I think the answer may be: it depends.
My own personal experience has partly been shaped through using libraries like Ruby/Amazon, which is a bit of a monster to understand, doesn’t always play well with others, is no longer being developed, and only works with an outdated version of the API. Because of this (and also a good experience with lightweight libraries such as will_paginate, when it came to finding a solution to Facebook functionality (and having rejected RFacebook) it seemed natural to write my own lightweight solution.
2. Use of external libraries. I remember hearing/reading some time ago a comment from I think one of the core Rails contribs about use of plugins, and they said they thought there should be a rule—never use a plugin you didn’t understand and couldn’t write yourself. I thought at the time it was a bit extreme, but it stuck with me, and I think now there’s a good deal of sense in it, at least as a goal.
The problem is with libraries, particularly gems and plugins (and I think big ones suffer more from this) is that very often people use them without thinking.
There’s a full-blown blog post to be written on this subject, but it’s been nagging at me for a while, and it’s probably subconsciously why I wrote my little Facebook library to be dropped into /lib and then included where necessary, rather than a plugin—because it’s then clear that it’s a library to wrap the Facebook API, to help programmers and clean things up a little, rather than acting as a black box.
When they call facebook.fb_users_get_info (to use the syntax from my tiny library) they need to get their hands a little dirtier as the response is a basic hash (that’s been produced from the raw XML response by XMLSimple), but the library’s so simple that they could easily parse it with Hpricot or something else, or add methods to return a more intelligent response. Plus, when it breaks
- as it surely will when Facebook change their API -the programmer will be changing their code to suit rather than somebody else’s (or conversely assuming that they will change it as speedily as they would like).I’m not saying that is necessarily better (and I’m certain someone with more Ruby fu could write a better lightweight library), just that it’s an alternative approach, and one that’s worth discussing. Cheers
Chris p.s. Would be great if you could edit comments p.p.s. Apologies for the mega length of comment. Nearly did it with a blog post, but not a big fan of conversations via two different blogs – they always see to end up needlessly confrontational
September 5th, 2007 at 11:45 AM
Great Article. Having written a similar wrapper API, one that does use method missing (but only for digging into the parsed XML). All other API “actions” are concrete.
Question: what is your intuition for wrapping APIs that have user created custom fields or objects? Where the objects can change at runtime?
- Bob
September 5th, 2007 at 12:25 PM
This makes me think that I’ve not been using enough abstraction in my rails application.
“Debugging Magical Incantations is Hard. Yes. I wish someone familiar with Ruby internals {w,c}ould write a hacked version of the interpreter so it saves the last k states for the last k method calls, and when an exception causes the interpreter to exit, it could drop out to an interactive mode. Then at least we might be able to have a ‘conversation’ with it about what it was doing (seriously Anglicized):
STRESSED_PROGRAMMER: Where did you get the nil object whose stack method you tried to call? HACKED_RUBY: from executing shrdlu.pickup(:pyramid, :purple). STRESSED_PROGRAMMER:”? Where did that come from? (It should be :green).
Obviously this would eat memory or disk and be geologically slow, but it might be faster than some of my debugging sessions. I don’t have enough familiarity with Ruby internals to think about attempting this.
September 5th, 2007 at 12:28 PM
Clearly that wasn’t intended to be a link. Can you fix that and delete this comment?
September 5th, 2007 at 01:33 PM
I downloaded the project from svn, in the readme it says to install do gem install facebooker I get this error when doing so ERROR: While executing gem … (Gem::GemNotFoundException) Could not find facebooker (> 0) in any repository I know I can compile it, I just prefer to use gems Thanks
September 5th, 2007 at 04:45 PM
My process for creating APIs/DSLs/pretty code is from a user’s perspective – what would I like to type in each time to activate this code? and combine that with all the syntactical idioms that I know I can implement in Ruby (method_missing, blocks, const_missing, dynamic method generation, etc). So, I would have ended up with Chad’s result – an API that I prefer to work with, rather than the API that is forced on my by Facebook’s PHP ppl merged into Ruby world.
But, I guess there’s nothing wrong with Chad’s code supporting a backdoor for tricky Facebook APIs that his pretty API might not yet support etc.
September 6th, 2007 at 12:10 AM
Thanks for the link, Chad!
September 9th, 2007 at 06:19 AM
I had the same problem as Clarence: got a “Could not find facebooker (> 0) in any repository” error when trying to install with RubyGems.
Tried the procedure at the link below, to no avail: http://armyofevilrobots.com/node/418
I’ll checkout from svn and read the tests/code, but any plans to put Rdoc docs, or other documentation, online?
Regards, John
September 9th, 2007 at 05:33 PM
Clarence and John, sorry the “gem install” insructions are meant for after we do a release. There’s no gem currently, though you can do:
rake dist:gem cd pkg gem install facebooker
September 11th, 2007 at 02:25 AM
Any idea when there’ll be a release?
September 13th, 2007 at 05:40 AM
I had a go at using the facebooker api from the subversion repository on Sourceforge, and here’s a word of warning… it’s not complete yet. Not by a long shot. Example: user properties are not being downloaded (even though the fields are there). I should have guessed when I looked at the unit tests and the only test there was for a user’s friends being present.
September 13th, 2007 at 02:55 PM
Daniel, it’s on RubyForge (correction for other readers). It’s true that Facebooker is not complete. That’s why there’s no release as of yet. I think you’ve run into a specific bug which isn’t actually due to it being incomplete. I’m on vacation so I haven’t had a chance to look into it yet. Another user reported the same problem. If you could post some code (maybe on the tracker on RubyForge) which exactly replicates the problem, it would be greatly appreciated.
September 21st, 2007 at 10:55 PM
Daniel or anyone else using (or attempting to use) Facebooker… any thoughts or examples that you want to share? I see there’s an examples folder with a desktop login example in the repository, I’m just looking for as much information as possible to help me in digging into Facebooker v. rFacebook. This discussion has been great so far.
September 23rd, 2007 at 09:26 PM
I’d like try your library Chad, but I’m concerned about the fact that a stable version still hasn’t been released. Are you still developing it? Will you be updating it frequently as changes are made to the Facebook platform?
September 24th, 2007 at 05:21 AM
George, Chad’s been off in Europe for the past week or two (attending RailsConf Europe, among other things). It’s still being developed and the release will happen when it’s ready (until then, you can always grab it and help out, as others have). As to future development… I’ll let Chad answer that.
September 24th, 2007 at 06:09 AM
Sorry, did not mean to sound accusing, but upon re-reading my post, it seems a bit on the harsh side!
I can’t remember exactly how it failed, I’m afraid. I think what happened is I retrieved a user object through what appeared to be the correct way (gathered from the unit tests), and found it had none of the expected properties set (e.g. first name, etc). When I then looked through the (very clean and readable) facebooker code, I couldn’t find any code that actually retrieved the user properties! (although now that I look through it again populate_from_hash! appears like it should be doing the job).
I’m afraid I don’t have the concept code anymore. I was evaluating facebooker and rfacebook and the code was changing rapidly and not checked in to any repository… Please don’t take my post as an accusation against facebooker, more as a warning that there are still (predictably) a few issues that will need to be ironed out. I definitely like the structure though – compared to rfacebook, which does feel very much like php, this is clean and neat and very easy to read.
I’ll definitely be giving it another try at some later date, and I’ll remember to keep my sample code then!
September 24th, 2007 at 11:42 AM
Hi all. The reason I haven’t yet been pushing adoption of Facebooker is that there is no release yet. The reason there is no release is that I want to run it through its paces more heavily first. The reason I haven’t run it through its paces is that I’ve been on a combined vacation/string of work and conferences in Europe for 3 weeks now (and won’t be home for over a week).
I heartily welcome contributions and feedback from people who are OK with being on the bleeding edge. If you’re not, then please use Facebooker as an example with which to frame this discussion on APIs (and not necessarily as software to use just yet).
Our company is doing Facebook development and will continue to evolve Facebooker as we do.
September 25th, 2007 at 04:55 AM
I actually prefer the RFacebook code because
1. it makes it crystal clear exactly which methods are doing network transactions, and network transactions are really, really expensive (especially into Facebook!).
2. it is easy to read Facebook’s documentation and see exactly how that applies to RFacebook. It isn’t as clear for Facebookr.