Is Serendipity the Heart of the WS-*/REST Debate?

Thanks to Technorati, I found this post by John Heintz. He’s checking out John Evdemon’se-book on SOA and has a problem with this overview:

SOA is an architectural approach to creating systems built from autonomous services. With SOA, integration becomes forethought rather than afterthought. This book introduces a set of architectural capabilities, and explores them in subsequent chapters.

To which John H. responds:

I, for one, would rather build on an architecture that promotes integration as an afterthought, so I don’t have to think about it before hand!!!

Yeah, I’d rather not have to think about integration before hand either. On the other hand, I want integration that actually works. It sounds like John H. is suggesting here that REST somehow eliminates the need to consider integration up front. It doesn’t. Consider this: if you’re building a Web 2.0 site then you are expected to expose everything in your site via APP, RSS and/or RESTful POX services. In other words, the Web 2.0 community expects you to have the forethought to enable integration. If you don’t, Marc Canter will call you out in front of Bill Gates and Tim O’Reilly.

This integration by afterthought approach seems to be big among RESTifarians. John H. links to a REST discussion post by Nick Gall advocating the principle of generality, “unexpected reuse” and “design for serendipity”. Money quote:

The Internet and the Web are paradigms of Serendipity-Oriented Architectures. Why? Largely because of their simple generality. It is my belief that generality is one of the major enablers of serendipity. So here I immodestly offer Gall’s General Principle of Serendipity: “Just as generality of knowledge is the key to serendipitous discovery, generality of purpose is the key to serendipitous (re)use.”

Serendipity means “the accidental discovery of something pleasant, valuable, or useful“. “Serendipitous reuse” sounds an awful lot like accidental reuse. Most enterprises have been there, done that and have nothing to show for their efforts or $$$ except the team t-shirt. Yet Tim Berners-Lee believes “Unexpected reuse is the value of the web” and Roy Fielding tells us to “Engineer for serendipity”. What gives?

First off, enterprises aren’t interested in unexpected or serendipitous reuse. They want their reuse to be systematic and predictable. The likelihood of serendipitous reuse is directly related to the number of potential reusers. But the number of potential reusers inside the enterprise is dramatically smaller than out on the public Internet. That brings the chance for serendipitous reuse inside the enterprise to nearly zero.

Second, enterprise systems aren’t exactly known for their “simple generality”. If Nick’s right that “generality of purpose is the key to serendipitous (re)use”, then enterprises might as well give up on serendipitous reuse right now. As I said last year, it’s a question of context. Context is specifics, the opposite of generality. Different business units have different business practices, different geographies have different laws, different markets have different competitors, etc. If an enterprise operates in multiple contexts – and most do - enterprise solutions have to take them into account. Those different contexts prevent you from building usable – much less reusable – general solutions.

Finally, I think the amount of serendipitous reuse in REST is overstated. If you build an app on the Facebook Platform, can you use it on MySpace? Nope. If you build an app that uses the Flickr services, will it work with Picasa Web Albums? Nope. Of course, there are exceptions – pretty much everyone supports the MetaWeblog API – but those exceptions seem few and far between to me. Furthermore, the bits that are getting reused – such as identifier, format and protocol – are infrastructure capabilities more suitable to reuse anyway. Serendipitously reusing infrastructure capabilities is much easier than serendipitously reusing business capabilities, REST or not.

The problems that stand in the way of reuse aren’t technology ones. Furthermore, the reuse problems face by enterprises are very different than ones faced by Web 2.0 companies. REST is a great approach, but it isn’t a one-size-fits-all technology solution that magically relegates integration and reuse to “afterthought” status. Serendipity is nice, when it happens. However, by definition it’s not something you can depend on.

Comments:

Serendipity derives from commonality; the more our components agree upon before they meet, the more they can accomplish without requiring a software change. You're right about Facebook and Flickr apps, but keep in mind that all it takes to create this problem is the slightest dependency of one component on anything specific to the implementation of the other. In the cases of Facebook/Flickr, those dependencies are solely on data. Had those APIs been WS-* APIs, the disagreement would have been far greater. Look at it this way; one could consider Googlebot as a client of both Facebook and Flickr, but it has no dependency on any specific implementation detail of either. It also handles other apps, including those yet to be developed, quite well too.
I've written a reply to your comments. My main points are: 1. I want my systems building effort to have the highest leverage possible. 2. REST add uniformity constraints to promote loose-coupling and integration. 3. A few more quotes from the book.
The book defines SOA as "A loosely-coupled architecture designed to meet the business needs of the organization." SOA does not necessarily require the use of Web Services or REST – these are, for most organizations, the simplest approach for implementing a loosely coupled solution architecture. There is no need for a REST/WS-* debate, simply use the approach best suited to the problem at hand. The Microsoft Robotics Studio (http://msdn2.microsoft.com/en-us/robotics/default.aspx) illustrates how both can be supported, combining REST with event notifications, relationships between components and more. "Some architectural styles are often portrayed as 'silver bullet' solutions for all forms of software. However, a good designer should select a style that matches the needs of the particular problem being solved." - From Roy Fielding's REST dissertation ("Architectural Styles and the Design of Network-based Software Architectures" - http://www.ics.uci.edu/~fielding/pubs/dissertation/software_arch.htm#sec_1_5)