• My boss let me borrow a Tecra M4 that he scavenged from his boss. The display is fairly twitchy, I think it’s a motherboard issue. But it’s very intermittent and I’ll get help desk to take a look. In the meantime, it sure is nice to driving a Tablet PC again. And it’s Vista ready to boot.
• Speaking of Vista, Visual Studio 2005 Service Pack 1 Update for Windows Vista. It’s a mouthful but it’s now available. Soma answers questions about the new release on MS PressPass.
• The DSL tools team keeps on rolling with the power toys. First it was the Designer Integration PowerToy, now it’s the DSL Tree Grid Editor PowerToy. Jeff Santos has the details.
• I missed the TechFest keynote yesterday, but it’s available on demand. They also have descriptions and videos of some of the technologies on display. (well, only one video so far, but I assume since the page is labeled “TechFest 2007 Videos” that more are on the way.)
• There’s new support for integrating WCF and WF coming in VS “Orcas”. Moustafa Khalil Ahmed has the details on what’s new for WF & WCF in the latest CTP drop. For me personally, the WCF/WF integration is some of the most important stuff coming in Orcas, second only to LINQ.

• According to Reuters surgeons who play video games are more skilled. Remind me to ask the doctor if s/he owns an XBOX 360 the next time I am getting operated on.
• I have reached the National Championship game in dynasty mode of NCAA Football 2007. The opponent of my BYU Cougars…why that would be Harry’s alma mater, the USC Trojans. Funny how that worked out.
• Nicholas Allen writes in his blog about when you should use Indigo to write a channel, and more importantly when you should not. As most of you know Harry and I are doing quite a bit of work with WCF so we are interested in this type of advice.
• Our team has been thinking about how to manage a large number of services in an automated fashion. This would include deploying new services, monitoring the services, automatically handling scaling, service discovery, and automated provisioning to name a few possible capabilities. I almost think of it like the next version of UDDI, especially when it comes to provisioning. I think that as systems become more distributed that the ability to automatically manage these systems is going to be key to their success. I know that some thought has already gone on in this area by people far smarter than I, but as I consider how to operate an infrastructure with thousands of services in it it is apparent that the opportunity is there for us to design and implement a system management framework that automates the majority of the tasks. I need to spend some time to consider how the framework would work, and document the capabilities.

(Harry is on a secret mission in uncharted space this week, so instead of the daily Morning Coffee post, you get a series of autoposted essays. This post builds on Harry’s recent epiphany about WCF and long running services)

Way, way, way back in March of 2003, IBM and Microsoft published an “overview and roadmap” white paper entitled “Reliable Message Delivery in a Web Services World“. It contained the following paragraph under the section “Exchanging Messages Reliably”:

WS-ReliableMessaging is not bound to underlying transport protocols or sessions. This means that the lifetime of a WS-ReliableMessaging conversation can span long periods of time (days, weeks) even when one or both systems are rebooted. This allows conversations to be suspended mid-stream (for example, to allow system maintenance) and then resumed without needing to retransmit the entire conversation. [emphasis added]

Now I know how I got confused about WCF and long running services in the first place. Support for long running services was part of the original web services vision!

About three years after that white paper was published, Shy Cohen wrote a post entitled Reliable Messaging Demystified on his blog. Shy was at one time the feature owner of WS-RM in WCF (according to his post) and wrote the following:

Reliable sessions [in WCF] are implemented using the WS-ReliableMessaging protocol. This protocol is yet another misnamed WS-* protocol, as it actually only deals with the reliability of the transfer and says nothing about durability, delivery acknowledgments, TTL for a message, long running sessions where a particular message is lost forever, etc.

At some point in the three years between March 2003 and February 2006, WS-RM went from being the enabler of long running services to “yet another misnamed WS-* protocol”. And with it, WCF lost (never had?) the ability to support long running services (as I’ve written previously).

Now all and all, this isn’t a big deal. I agree with Shy that WS-RM is under specified as mechanism for durable messaging (Shy calls this “queued messaging”). Attempting to build durable messaging on top of WS-RM sounds like it would have been both difficult and unlike to broadly interoperate. So implementing WS-RM for TCP style reliability and leveraging MSMQ as a transport for people that need durable messaging sounds like a pretty good compromise, especially for a v1 product. Of course, it is not exactly unheard of for a project’s end result not to completely live up to the original vision. But I have a specific requirements in this case, so I wanted to know more.

By calling it “misnamed”, it sounds like WS-RM was never really intended to be used for durable messaging. However, the July 2003 Reliable Messaging Feedback Workshop indicates that it was. In particular, Rodney Limprecht’s “Reliable Messaging Scenarios” deck describes WS-RM as supporting scenarios requiring “either volatile or durable endpoint state”. His list of scenarios included both an “Intermittent Connectivity” scenario where “messages pending transfer are staged to disk and exchanged when connected” as well as a “Message Queue Integration” scenario that used WS-RM to interop between JMS and MSMQ. Seems safe to say that WS-RM was originally intended to support durable messaging. So what happened? How did it become “misnamed”?

Rodney’s deck describes WS-RM as having the “flexibility to meet scenario requirements”. But flexibility comes at a cost. For example, the flexibility of WCF’s configuration comes at the cost of significant complexity. In the case of WS-RM, it appears that by trying to make it flexible enough to support both volatile and durable reliability, the authors might have made it too flexible. WS-RM implementers have broad latitude in building the capabilities Shy mentions (durability, acknowledgements, TTL, etc) as well as describing said capabilities in policy. By providing that latitude, we lost the ability to broadly interop durable messaging, which I would suspect is why it ended up out of scope for WCF v1.

As I said before, lack of support for WS-RM based durable messaging isn’t that big a deal. As long as you understand WCF’s sweet spot - the current version’s sweet spot anyway – and don’t try and make it be something it’s not, you should be fine. Furthermore, Shy mentions the need for an “interoperable Queued Messaging specification” and wrote that it’s something he “expect[s] that we will get to it in the near future”. Here’s hoping that spec is less flexible than WS-ReliableMessaging.

Regular readers of DevHawk are likely aware of my obsession interest in SQL Service Broker (aka SSB). I’ve also been doing a lot of WCF work lately. While there are parts of WCF that I think rock, overall I’ve found WCF lacking due to it’s lack of support for long running services, which SSB excels at.

So it was with great interest that I read this recent article on Integrating WF and WCF. WF is expressly designed for long running systems, so I wanted to see how the article dealt with the WCF’s lack of support for such scenarios. Unfortunately, the article basically sidesteps the issue. While it has lots of great info about hosting WF inside a WCF service, the article uses duplex channels for communication between the service and its clients. As I have pointed out before, this approach is impractical because it requires that both the service and its consumer remain alive in memory until the WF end.

Remember this quote from Essential WF?

“It is wishful thinking to assume that the operating system process (or CLR application domain) in which the program begins execution will survive for the required duration.”

So basically this WCF/WF sample is wishful thinking. Fine for a demo, but given the severe lack of information out there on integrating these two technologies, I’m worried that many people will read this article as best practice guidance, which in my opinion would be a mistake.

But instead of firing up my blog (that is, like last time) to write a scathing post about how broken this sample is, I emailed Paul which led to a concall with Shy to discuss WCF’s lack of support for long running services. Imagine my surprise when Shy agreed with me completely, furthermore saying that support for long running services had been “out of scope” for v1 of WCF. I thought that the whole point of duplex channels was for long running services. But apparently I was wrong.

Shy said to think of the duplex channel in terms of sockets, rather than long running conversations. And just like that, WCF made a ton more sense to me. I had been directly comparing the SSB and WCF communication models, but that’s apples and oranges. It would be like comparing SSB to TCP.

If you think about it, vanilla HTTP works a lot more like UDP, even though it’s layered on top of TCP. Both UDP and HTTP support connectionless operations and neither UDP nor HTTP are reliable or provide message ordering. The comparison isn’t perfect: for example, UDP isn’t limited to a single response for an incoming request. But by and large, HTTP is a very UDP style protocol.

If HTTP is basically UDP, then WS-* is trying to be TCP. Frankly, I never understood the point of WS-ReliableMessaging. I always thought reliability == durability == SSB or MSMQ. But when you realize that HTTP lacks TCP-like reliability and ordering capabilities, suddenly this WS spec makes sense. In fact, Shy made this exact point almost a year ago. At the time, I didn’t get it because I didn’t understand the duplex channel as sockets analogy. Now, I see the value of adding these capabilities to HTTP.

What Shy said was clear and to the point but unfortunately completely missing in the official WCF documentation. For example, the docs on Duplex Services say this:

A duplex service contract is a message exchange pattern in which both endpoints can send messages to the other independently. A duplex service, therefore, can send messages back to the client endpoint, providing event-like behavior. Duplex communication occurs when a client connects to a service and provides the service with a channel on which the service can send messages back to the client.

The docs make no mention that the “event-like behavior” of duplex services only works within a session. And I’m not the only one who mistakenly believed that duplex services could be used for long running services (here’s an article in DDJ that makes the same mistake). Shy used the term “episodic” to describe services that span session boundaries. I’d like to see the docs updated to include that concept.

Taking the TCP/UDP analogy even further, I think it demonstrates how pointless the REST vs. SOAP debate is. As UDP is a thin layer on top of IP, REST is a thin layer on top of HTTP. But nobody argues much about UDP vs. TCP these days. I was in grade school when UDP and TCP were standardized, so maybe there were big TCP vs UDP flame wars at the time. But twenty five years later, it’s pretty clear that TCP vs UDP is not an either-or proposition. Some protocols are better built on UDP while others are better built on TCP. I’m guessing we’ll see a similar evolution with SOAP and REST.

Personally, I would expect that message exchanges between services will become more complex over time. Complex message exchanges would seem to favor stateful SOAP over stateless REST for the same reason complex network protocols favor connection-oriented TCP over connectionless UDP. But SOAP could never displace REST any more than TCP could ever displace UDP. Furthermore, as Larry O’Brien recently wrote “the onus is on the WS-* advocates to prove the need”. TCP standardization only lagged a year behind UDP standardization where WS-* has lagged at least six years behind REST. I wonder if UDP would be more prevalent today if it had gotten a six year head start on TCP.

Finally, this “SOAP as Sockets” flash of understanding has also helped me understand how SSB / WCF can evolve together in the future. Some folks have suggested an SSB transport for WCF and I’ve personally looked into such an approach. But given since SSB is at a higher level of abstraction than WCF, it makes much more sense to layer SSB on top of WCF instead of the other way around. Today, SSB uses two protocol layers: the top level Dialog Protocol, which is built on top of the lower-level Adjacent Broker Protocol (ABP), which in turn is built on TCP. I’d like to see a version of ABP that was built on top of WCF instead of directly on top of TCP. SSB’s Dialog Protocol would tie together the WCF duplex sessions into a long-running conversation the same way that it ties together TCP sessions today.

Eventually, I would love to see something that has the programming semantics of SSB and the interoperability of WCF. That would be like the the Reese’s Peanut Butter Cup of service messaging.

• I figured out something to build with PowerShell. Not ready to spill all the beans yet, but I’ve started by building a provider for SQL Server. SQL already has the SQL Management Objects (SMO) library, so I can really focus on how PowerShell works rather than getting too bogged down in the logic of the provider. I’m sure someone else is working on something similar, but my primary goal with building this provider is to understand PowerShell. The codebase itself is a distant second in priority.
• On the subject of missing LINQ’s, Alex James wants LINQ to Web. A good potential use for the Windows Live Search API.
• I posted three recent blog posts to DotNetKicks yesterday, but I’ve only collected one additional kick so far (for my Compiling Workflows post). Looking at what does gather kicks, I think I would need to write a more dev focused article if I wanted to make the DNK home page.
• Don Smith left a comment on my WSSF post where he talked about the developer and the architect perspective. He seemed surprised that I took the developer perspective. To quote David White: “Architect Must Implement”. The customization opportunity in guidance automation is huge, but the value is to the developer first. And while it’s a good start, it doesn’t go very far at automating the development experience. At heart, I guess I’m a developer even though my business card sez Architect (with a capital “A” these days no less).
• I moved my laptop over to Vista yesterday. My workstation is still on XP, but that’s because I haven’t had time to completely rebuild my dev environment. No Aero support with the
• I elbowed my way into the TS Gateway pilot at Microsoft and I’m loving it. TS Gateway is a new feature of Longhorn Server that “enables authorized remote users to connect to terminal servers and remote desktops on the corporate network from any Internet-connected device running Remote Desktop Connection 6.0.” So I can quickly and easily remote into my desktop from anywhere without establishing a VPC session. The only annoyance is my USB smart card reader, which is fine in a pinch but a pain to constantly have dangling off my laptop. However, my boss approved a PCMCIA smart card reader so soon that won’t be an issue.