John Heintz responded to my serendipitous reuse post. Nice to see I misunderstood his opinions about how easy RESTful systems are to integrate:

I didn’t mean to imply that building RESTful system would lead to magical integration without any hard work. I can see how that came across in my post, and I guess I got the reaction I asked for 😉

I get the feeling that John would be a good guy to have a beer with.

John spends most of his post writing about the SOA in the Real World book. I’ve flipped thru it and I’m familiar with the model (it is my old team after all) but I haven’t read it so I don’t really want to comment about the book specifically. But there were two things John mentioned that I did want to comment on.

First, at the end of his post John writes:

Can some of the constraints of REST be applied to SOA? Absolutely. I think an asynchronous, message-passing architecture with a uniform interface would be astoundingly interesting! I’m not the only one: see MEST, AMPQ, and Erlang.

This goes back to a REST question I asked two months ago: is it still REST if you don’t use HTTP? I’m guessing John would say yes.

I might be going out on a limb here, I’ll bet the core of John’s problem with SOA is how toolkits like WCF all but force you to build RPC style services that can easily be modeled as method calls. That’s certainly one of my problems with SOA. Tim Ewald said it best:

It’s depressing to think that SOAP started just about 10 years ago and that now that everything is said and done, we built RPC again. I know SOAP is really an XML messaging protocol, you can do oneway async stuff, etc, etc, but let’s face it. The tools make the technology and the tools (and the examples and the advice you get) point at RPC. And we know what the problems with RPC are. If you want to build something that is genuinely loosely-coupled, RPC is a pretty hard path to take.

If SOA == RPC and REST == loosely coupled messages, then I’ll start growing dreadlocks right now. Frankly, as Tim says, I think it’s a problem with the tools (I’m looking at you WCF) and not the underlying architecture, but how many people can distinguish the architecture from the tools? Not many, I’m afraid.

Second, John asks an interesting question:

Where are the SOA mashups?

That’s easy! They’re inside the firewall where you can’t see them! 😉

Seriously, I’m not sure about “SOA” mashups, but I’m working with what you might call a huge “enterprise” mashup system inside Microsoft. Our Enterprise Data Integration Services push around massive amounts of data to downstream systems. There are over fifty datasets in production, each with scores of tables, millions of rows and hundreds of subscribing systems. One example, our Products dataset, has over 100 tables and nearly 300 subscribing systems.

Is it “service oriented”? No, but then again it was originally developed ten years ago on SQL 6.5. But is it a mashup? Is it an “application that combines content from more than one source into an integrated experience“? Yep. Is it easy to work with? No, but guess why I’m involved? We’re looking at ways to “modernize” the system. Am I going to build RPC style services as part of this modernization? Hell, no.

So John, am I right or wrong about that beer?

• The last day of the service factory workshop was much like the second, primarily focusing on stuff p&p built to integrate GAT and DSLs. We also got a briefing in what’s coming for factories after VS08 (can’t blog about that). We ended with a look at the DSL Editor Power Toy, which provides additional views on a given model and allows you to completely replace the graphical editor with a Windows Forms UserControl. I wonder if you could use ElementHost in order to build a WPF based editor?
• Finished the last Harry Potter book last night. My wife finished it last week but kept quiet about it until I got to the end. No spoilers here, but I wasn’t exactly surprised by how it played out. I wonder what J.K. Rowling will write next?
• As promised, Silverlight 1.0 RC and Silverlight 1.1 Alpha Refresh were released last week. Also finishing out this beta wave were Silverlight 1.1 Alpha Tools for VS08 and a new preview of Expression Blend 2. Scott Hanselman has all the details on all the releases.
• In one of his articles on LINQ to SQL, Scott Guthrie mentioned the LINQ to SQL debug visualizer in passing. Now, he drills into that feature in more detail. Apparently, this isn’t a built-in feature of VS08 – it has to be installed separately. Make sure you do that, this seems like a must-have extension for LINQ to SQL development.
• Jeff Atwood is worried that he spends more time talking about programming than actually programming. That’s exactly why I left evangelism to join MSIT.
• I’m still way behind on blogs, but if I don’t post this soon, it’s going to be an afternoon coffee. I’ve also got this day job thing that I’ve been away from for several days. So more old news tomorrow.

In case you didn’t know, GotDotNet is shutting down. I recently got an email asking about my very old DevHawk Wiki project, which had been hosted on GDN. I didn’t realize anyone was still interested in this project, so I set up a new project for it on CodePlex. As part of the move, I decided to rebrand it “HawkWiki” and change its license to Ms-PL. (CodePlex doesn’t support the zlib license DevHawk Wiki was originally released under.)

I uploaded two versions to the HawkWiki source repository. The version that until recently was hosted on GDN is version 0.2. There’s also later version that I never publicly released before. This later version compiles the wiki markup text into an IHttpHander class, similar to how ASP.NET compiles web pages. I’m not sure if you’d ever really want a wiki built this way, but it does provide a good example for building your own compiled web page infrastructure. If you ever read my old MSDN magazine article, you’d know this is an approach I’ve been interested in for a long time.

I also used this project as an excuse to get to know CodePlex. Though CodePlex can integrate directly into VS via Team Explorer, I chose instead to use the CodePlex client. CPC provides an edit-merge-commit command-line experience like Subversion. I found it much easier than using Team Explorer, though adding new files was troublesome since they had to be added to the project and source control separately. If you like this approach, there’s also a version that works with vanilla TFS instances (CPC is hardwired to CodePlex).

So feel free to take the wiki code and mangle it to your heart’s content. If there’s interest, I’d be willing to grant some other folks checkin permission. However, it’s more a curiosity than a real project, so if you’re really interested in a .NET based wiki, there are better choices out there.

As expected, day two of the WSSF customization workshop was much meatier than day one. If you look at the technologies that power these p&p software factories – GAT and DSL – you’ll notice that they’re not very well integrated. They share a text templating engine and there are some docs, but that’s pretty much it. WSSF v3 is the first factory from p&p to heavily use both technologies, and they’ve had to do significant integration work to make them play nice.

In the new WSSF, there’s a special “model project” where you define data and service contracts independently of an implementation technology. This allows you to postpone your implementation decision as long as possible. For WSSF, this seems pretty pointless – if you’re starting a new service project from scratch, why would you build on ASMX instead of WCF? – but conceptually the idea is rock solid. But this requires a variety of cross-model and cross-project code generation and validation. The vanilla DSL toolkit doesn’t support that, so the p&p guys had to build it themselves.

I’d like to see p&p take their GAT/DSL integration work and package it separately from WSSF. As it currently stands, you’ve got to install GAT & the VS SDK (to get DSLs) plus the integration bits from WSSF. There’s also the GAX Extension Library (aka GEL) plus Clarius’ Software Factory Toolkit. How about a single install to get all that stuff guys?

BTW, there’s a new version of GAT coming soon that will support both VS05 and VS08 Beta 2. According to the VSX team, VS08 beta 2 SDK should also be available “early August”. So if you’re moving over to VS08 (I am), you can still get down and dirty with software factories.

Update: VS08 SDK for Beta 2 is now available. The new version of GAT is apparently done, but isn’t available for download yet.

Later Update: GAX/GAT July CTP is now available.

Thanks to Technorati, I found this post by John Heintz. He’s checking out John Evdemon’se-book on SOA and has a problem with this overview:

SOA is an architectural approach to creating systems built from autonomous services. With SOA, integration becomes forethought rather than afterthought. This book introduces a set of architectural capabilities, and explores them in subsequent chapters.

To which John H. responds:

I, for one, would rather build on an architecture that promotes integration as an afterthought, so I don’t have to think about it before hand!!!

Yeah, I’d rather not have to think about integration before hand either. On the other hand, I want integration that actually works. It sounds like John H. is suggesting here that REST somehow eliminates the need to consider integration up front. It doesn’t. Consider this: if you’re building a Web 2.0 site then you are expected to expose everything in your site via APP, RSS and/or RESTful POX services. In other words, the Web 2.0 community expects you to have the forethought to enable integration. If you don’t, Marc Canter will call you out in front of Bill Gates and Tim O’Reilly.

This integration by afterthought approach seems to be big among RESTifarians. John H. links to a REST discussion post by Nick Gall advocating the principle of generality, “unexpected reuse” and “design for serendipity”. Money quote:

The Internet and the Web are paradigms of Serendipity-Oriented Architectures. Why? Largely because of their simple generality. It is my belief that generality is one of the major enablers of serendipity. So here I immodestly offer Gall’s General Principle of Serendipity: “Just as generality of knowledge is the key to serendipitous discovery, generality of purpose is the key to serendipitous (re)use.”

Serendipity means “the accidental discovery of something pleasant, valuable, or useful“. “Serendipitous reuse” sounds an awful lot like accidental reuse. Most enterprises have been there, done that and have nothing to show for their efforts or $$$ except the team t-shirt. Yet Tim Berners-Lee believes “Unexpected reuse is the value of the web” and Roy Fielding tells us to “Engineer for serendipity”. What gives?

First off, enterprises aren’t interested in unexpected or serendipitous reuse. They want their reuse to be systematic and predictable. The likelihood of serendipitous reuse is directly related to the number of potential reusers. But the number of potential reusers inside the enterprise is dramatically smaller than out on the public Internet. That brings the chance for serendipitous reuse inside the enterprise to nearly zero.

Second, enterprise systems aren’t exactly known for their “simple generality”. If Nick’s right that “generality of purpose is the key to serendipitous (re)use”, then enterprises might as well give up on serendipitous reuse right now. As I said last year, it’s a question of context. Context is specifics, the opposite of generality. Different business units have different business practices, different geographies have different laws, different markets have different competitors, etc. If an enterprise operates in multiple contexts – and most do - enterprise solutions have to take them into account. Those different contexts prevent you from building usable – much less reusable – general solutions.

Finally, I think the amount of serendipitous reuse in REST is overstated. If you build an app on the Facebook Platform, can you use it on MySpace? Nope. If you build an app that uses the Flickr services, will it work with Picasa Web Albums? Nope. Of course, there are exceptions – pretty much everyone supports the MetaWeblog API – but those exceptions seem few and far between to me. Furthermore, the bits that are getting reused – such as identifier, format and protocol – are infrastructure capabilities more suitable to reuse anyway. Serendipitously reusing infrastructure capabilities is much easier than serendipitously reusing business capabilities, REST or not.

The problems that stand in the way of reuse aren’t technology ones. Furthermore, the reuse problems face by enterprises are very different than ones faced by Web 2.0 companies. REST is a great approach, but it isn’t a one-size-fits-all technology solution that magically relegates integration and reuse to “afterthought” status. Serendipity is nice, when it happens. However, by definition it’s not something you can depend on.