Is Serendipity the Heart of the WS-*/REST Debate?

Thanks to Technorati, I found this post by John Heintz. He’s checking out John Evdemon’se-book on SOA and has a problem with this overview:

SOA is an architectural approach to creating systems built from autonomous services. With SOA, integration becomes forethought rather than afterthought. This book introduces a set of architectural capabilities, and explores them in subsequent chapters.

To which John H. responds:

I, for one, would rather build on an architecture that promotes integration as an afterthought, so I don’t have to think about it before hand!!!

Yeah, I’d rather not have to think about integration before hand either. On the other hand, I want integration that actually works. It sounds like John H. is suggesting here that REST somehow eliminates the need to consider integration up front. It doesn’t. Consider this: if you’re building a Web 2.0 site then you are expected to expose everything in your site via APP, RSS and/or RESTful POX services. In other words, the Web 2.0 community expects you to have the forethought to enable integration. If you don’t, Marc Canter will call you out in front of Bill Gates and Tim O’Reilly.

This integration by afterthought approach seems to be big among RESTifarians. John H. links to a REST discussion post by Nick Gall advocating the principle of generality, “unexpected reuse” and “design for serendipity”. Money quote:

The Internet and the Web are paradigms of Serendipity-Oriented Architectures. Why? Largely because of their simple generality. It is my belief that generality is one of the major enablers of serendipity. So here I immodestly offer Gall’s General Principle of Serendipity: “Just as generality of knowledge is the key to serendipitous discovery, generality of purpose is the key to serendipitous (re)use.”

Serendipity means “the accidental discovery of something pleasant, valuable, or useful“. “Serendipitous reuse” sounds an awful lot like accidental reuse. Most enterprises have been there, done that and have nothing to show for their efforts or $$$ except the team t-shirt. Yet Tim Berners-Lee believes “Unexpected reuse is the value of the web” and Roy Fielding tells us to “Engineer for serendipity”. What gives?

First off, enterprises aren’t interested in unexpected or serendipitous reuse. They want their reuse to be systematic and predictable. The likelihood of serendipitous reuse is directly related to the number of potential reusers. But the number of potential reusers inside the enterprise is dramatically smaller than out on the public Internet. That brings the chance for serendipitous reuse inside the enterprise to nearly zero.

Second, enterprise systems aren’t exactly known for their “simple generality”. If Nick’s right that “generality of purpose is the key to serendipitous (re)use”, then enterprises might as well give up on serendipitous reuse right now. As I said last year, it’s a question of context. Context is specifics, the opposite of generality. Different business units have different business practices, different geographies have different laws, different markets have different competitors, etc. If an enterprise operates in multiple contexts – and most do - enterprise solutions have to take them into account. Those different contexts prevent you from building usable – much less reusable – general solutions.

Finally, I think the amount of serendipitous reuse in REST is overstated. If you build an app on the Facebook Platform, can you use it on MySpace? Nope. If you build an app that uses the Flickr services, will it work with Picasa Web Albums? Nope. Of course, there are exceptions – pretty much everyone supports the MetaWeblog API – but those exceptions seem few and far between to me. Furthermore, the bits that are getting reused – such as identifier, format and protocol – are infrastructure capabilities more suitable to reuse anyway. Serendipitously reusing infrastructure capabilities is much easier than serendipitously reusing business capabilities, REST or not.

The problems that stand in the way of reuse aren’t technology ones. Furthermore, the reuse problems face by enterprises are very different than ones faced by Web 2.0 companies. REST is a great approach, but it isn’t a one-size-fits-all technology solution that magically relegates integration and reuse to “afterthought” status. Serendipity is nice, when it happens. However, by definition it’s not something you can depend on.

Afternoon Coffee 106

Lots of meetings today, so my coffee post is late…

  • The Big Newstm: Visual Studio 2008 and .NET Framework 3.5 Beta 2 is available for download. Soma and Scott have more. Silverlight 1.0 RC and the Silverlight Add-in for VS08 will apparently be available in a couple of days. Finally, there’s a go-live license for the framework, so you get a head-start deploying apps before VS08 and NETFX 3.5 RTM. Time to build out a new VPC image.
  • Next week, I’m attending the p&p Service Factory v3 Customization Workshop. I’m looking forward to playing with the new Service Factory drop, but I’m really interested in learning more about building factories. I wonder if they’re going to discuss their VS08 plans.
  • Nick Malik recently wrote about making “middle out SOA” work. I hate that term “middle-out”. It feels like we’re pinning our hopes on middle-out because we know top-down and bottom-up don’t work. My old boss John DeVadoss (who assures me he’ll be blogging regularly again “soon”) big vs. little SOA, with big SOA being “dead”. I like the term “little SOA” better than “middle-out SOA”, but just because big SOA is a big failure, doesn’t mean little SOA will make any headway.
  • There’s a new F# drop available. Don Syme has the details. Looks like they’ve got some interesting new constructs for async and parallel programing.
  • ABC announced yesterday that they are streaming HD on their website. So you can check out the season finale of Lost in HD for free. They embed commercials so it’s not really “for free”, but you don’t have to pay $3 an episode like you do on XBLM. I wonder if XBLM might offer this capability in the future? Certainly would increase my use of XBLM. (as would an all-you-can-eat pricing scheme)

Early Afternoon Coffee 105

  • My two sessions on Rome went very well. Sort of like what I did @ TechEd last month, but with a bit more kimono opening since it was an internal audience. Best things about doing these types of talks is the questions and post-session conversation. I’ve missed that since moving over to MSIT.
  • Late last week, I got my phone switched over to the new Office Communications Server 2007 beta. In my old office, I used the Office Communicator PBX phone integration features extensively. However, when we moved we got new IP phones that didn’t integrate with Communicator. So when a chance to get on the beta came along, I jumped. I’ll let you know my impressions after a few weeks, in the meantime you can read about Mark Deakin’s experience.
  • Matevz Gacnik figures out how to build a transactional web service that interacts with the new transactional file system in Vista and Server 08. Interesting, but personally I don’t believe in using transactional web services. The whole point of service orientation is to reduce the coupling between services. Trying two services (technically, a service consumer and provider) together in an atomic transaction seems like going in the wrong direction. Still, good on Matevz for digging into the transactional file system.
  • Udi Dahan gives us 6 simple steps to being a “top” IT consultant. I notice that getting well known, speaking and publishing are at the top of the list but actually being good at what you’re well known for comes in at #5 on the list. I’m sure Udi thinks that’s implicit in becoming a “top” consultant, but I’m not so sure.
  • Pat Helland thinks Normalization is for Sissies. Slide #6 has the key take away: “For God’s Sake, Don’t Normalize Immutable Data”.
  • Larry O’Brien bashes the new binary efficient XML working group and working draft. I agree 100% w/ Larry. These aren’t the droids we’re looking for.
  • John Evdemon points to a new e-book from my old team called SOA in the Real World. I flipped thru it (figuratively) and it appears to drill into the Foundations of Solution Architecture as well as provide real-world case studdies for each of the pillars recurring logical capabilities. Need to give it a deeper read.

Restating the Concurrency Problem

I’ll be honest, I recommended Herb Sutter’s concurrency series in today’s Morning Coffee because it a series on concurrency by Herb Sutter. In other words, I hadn’t actually read it yet, but I know how good Sutter’s stuff is. Now I have read it and I wanted to re-issue my recommendation, even more strongly. Go read it.

Interestingly enough, I like the article because it doesn’t provide an “answer” to the problem of concurrency. Rather, by providing a mental model, it essentially is a concise and precise restatement of the problem. Often, in the rush to find a solution to a problem, this step is skipped and it isn’t until the end that you realize that you misunderstood the original problem and what you built doesn’t match what you need.

I’ve often argued that this is also the key to selling in the enterprise. In my experience, whatever solution you’re selling is usually way too complicated to be understood by the people who have the purchasing power to buy it. So explaining how your solution works or how your solution solves the problem isn’t going to get you very far. However, the buyer does understand the problem at hand. Being able to demonstrate that you understand the fundamental nature of the problem and can communicate it back to them garners you a great deal of trust in the selling process. If you can show that you understand their problem, then you probably know how to fix it – even if the buyer doesn’t understand how your solution works.

One other quick thought on Sutter’s article. In discussing the use of concurrency to keep a GUI responsive (aka Pillar 1), he wrote the following:

Today, we typically express Pillar 1 by running the background work on its own thread or as a work item on a thread pool; the foreground task that wants to stay responsive is typically long-running and is usually a thread; and communication happens through message queues and message-like abstractions like futures (Java Future, .NET IAsyncResult). In coming years, we’ll get new tools and abstractions in this pillar, where potential candidates include active objects/services (objects that conceptually run on their own thread, and calling a method is an asynchronous message); channels of communication between two or more tasks; and contracts that let us explicitly express, enforce, and validate the expected order of messages. [emphasis added]

If we’re going to provide the ability to express, enforce and validate the expected order of messages between concurrent blocks of code, can we also do it for services across the network? WSDL is wholly deficient in this area. SSDL’s Communicating Sequential Processes (CSP) and Rules-based Protocol Frameworks are a good start.

Morning Coffee 99

  • Mladen Prajdic has a great post on handling a database in your unit tests. He mentions NDbUnit but seems mostly to favor SQL 2005′s database snapshot feature. He’s got sample code for creating and restoring a snapshot. (via DNK)
  • Microsoft Robotics Studio 1.5 released yesterday. Tandy Trower – GM of the Robotics group – has the details on what’s new.
  • Herb Sutter has a new column in Dr. Dobbs on concurrency. First up, “building a consistent mental model for reasoning about concurrency”. Sounds like a must read column. (via LtU)
  • Scott Hanselman describes “Sez You Architecture”. I wonder, do architecture ninjas get to wear a Shinobi shozoku?
  • From the Not Everyone Agrees With DevHawk Dept.: Libor Soucek disagrees with me and thinks that durable messaging should be avoided. I had a hard time following Libor’s logic but needless to say, I disagree with his disagreement. He writes that one of the reasons to use DM is for “Cooperating on transaction with external system”. While multiple systems may be cooperating on a business transaction, in no way do I believe they are going to cooperate on a database transaction. But since he started talking about the DTC, I suspect we’re talking past each other. Libor, drop me a line and we can discuss further.