Morning Coffee 66

Yesterday’s Morning Coffee was canceled on account of rain. In my living room. It’s fixed now.

  • Andre Vrignaud writes about MS Research’s new High Capacity Color Barcodes technology. As he points out, there’s some fascinating gaming potential for these barcodes because they have such high capacity (something like 2kb per square inch) and can be read without special equipment (a camera phone should work).
  • According to a Pew Research Center report, Daily Show/Colbert Report viewers are significantly better informed than Fox News viewers. On the other hand, they’re only slightly more informed than O’Reilly Factor viewers or Rush Limbaugh listeners so it seems like a wash.
  • Speaking of The Daily Show and The Colbert Report, you can now download them from Xbox LIVE Video Marketplace. But at $2 160 points an episode, it’s cheaper to set my DVR.
  • I recently re-discovered Remus Rusanu’s SSB blog. He went dark for a few months there, but he’s recently posted a new version of his Service Listing Manager utility, presented SSB at DevConnections and showed how to implement a managed stored proc to receive SQL DDL event notifications. Event notifications is one of those features I didn’t even realize was in SQL.
  • Dottie Shaw, one of the program managers on my project, has started blogging. That leaves two team mates and one project member still not blogging.
  • Yesterday, I stumbled into some other teams morale event. They were bogarting the cafeteria, so it wasn’t like I was crashing it or anything. Normally, I wouldn’t hang around some other teams party, but they had a projector, an Xbox 360 and two copies of Guitar Hero so I had to hang out and watch them play head-to-head for a while. That looks like a fun game.
  • Chris Anderson writes at length about the primary enemy of Long Tail economics: “the absurdly complicated and expensive process of rights clearance”. His case in point is the coming DVD release of WKRP in Cincinnati, which has replaced the dozens of songs used as background music with “Muzak-style songs that could be licensed in perpetuity for a small flat fee” that apparently “sucked ass”.

Morning Coffee 58

  • Nicholas Allen points out that Messaging is not a Transaction. Of course, what he really means is that Messaging with WCF is not a Transaction. Messaging with SSB is in fact a transaction (well, more accurately, it’s transactional but you get the idea). Nick’s right that you may not need anything more than simple retry semantics that you can implement yourself in your application protocol. But SSB has spoiled me. Why should I have to write the retry semantics? Why can’t my messaging stack provide that for me? Or put another way, if I need a “precicely defined failure model”, wouldn’t I be better of choosing a messaging stack that provides that out of the box?
  • Steve Hartman explains how to use span Remote Desktop Connection across multiple monitors for use with VPC. I wonder if this will work on my machine, since my multi-monitor setup is an L shape, not a rectangle. I’m offsite today, so I’ll try it tomorrow (via DotNetKicks)
  • According to news reports, Microsoft is negotiating to provide DRM free music too. I wonder how well this will work for Microsoft given that both Zune and PlaysForSure provide subscription services. I would assume the $15 all-you-can-listen style subscription services wouldn’t be DRM free. Given that program is one of the (few) selling points they have over Apple, will the availablity of DRM Free music undercut the interest in subscription? (via Dale)

Answering Dr. Nick’s Questions on SSB & WCF

Nick Allen asked on his blog about how people would like to see SSB and WCF work together. He’s already heard these from me, but I figured I’d put them out there for everyone to see and debate. Plus, I had several beers last night at the MVP dinner, so this is likely to be more coherent than I was yesterday! 😄

1. Are you interested in SSB because you’d like to have your service closer to the database? How close is close enough to the database?

I’ve first blogged about the endangered middle tier almost three years ago. My point at the time was that as you break your monolithic system up into services, the vast majority of those services won’t need to scale out. You performance gets better the closer you are to the data. If you don’t need to scale out, why not get the maximum boost by running in the database process itself?

Furthermore, in large IT shops, the database files are stored out on the SAN rather than on hard drives attached to the database server itself. That means the database server is effectively stateless. Why add a second stateless tier if you don’t need scale out? If you need more performance in a given service, you can detach the database file from it’s current SQL server box and attach it on another more beefy SQL server box without physically moving the database files at all. This enables what I call the “Star Trek Effect”, where you can shift computing power where it is needed most (more power to the payroll system!).

Of course, if you’re going to move the service, you do need to bring it down for a short time. That implies a need for durable messaging so that service consumers aren’t affected by the brief service interruption. Which brings us to…

2. Are you interested in SSB because you need durable, duplex messaging between two services? Do you need exactly-once-in-order message delivery?

Yes. SSB has a bunch of other nice features, but durable duplex messaging is what I need the most. Exactly-once-in-order is also fairly critical, though there may be scenarios where it’s not really necessary. Those are the exception, not the rule however.

Doesn’t WS-RM already do EOIO already?

3. Are you interested in using SSB from WCF because you want a better asynchronous messaging experience than MSMQ? What makes you prefer SSB to other queuing products?

My primary problem with MSMQ for the problems I’m tasked with solving is that MSMQ is one way while SSB supports duplex messaging. You could do duplex messaging with MSMQ if you didn’t mind managing multiple queues (one for each side of the conversation) but SSB does this for you for free. I’m sure there are scenarios where pure one-way messaging are useful, but they are few and far between in my day job.

Furthermore, SSB has the explicit idea of a service instance (they call it a conversation group) which MSMQ lacks. SSB’s implementation is conceptually similar to the new WCF/WF integration work in the latest Orcas CTP.

Finally, SSB uses logical naming. You have conversations between services, but services get mapped to physical addresses at the routing layer. This allows services to move around more easily (see the “Star Trek Effect” in #1 above). Both MSMQ and WCF use physical addresses, which makes them much more difficult to move.

4. Are you interested in having your data contracts defined in WCF, SQL, or both?

I like WCF’s data contract infrastructure. We did a early prototype long-running service with both WCF and SSB. the messaging stack code was obviously different, but we used the exact same data contract code. I even wrote some code to automatically deserialize the SSB message by mapping the SSB message type to a data contract.

I want my services to run inside the database, but that doesn’t mean I want to write them in T-SQL. Personally, I’m much more productive in C# and/or WF. So WCF data contracts are fine by me.

Morning Coffee 45

  • Yesterday’s morning coffee was canceled on account of going to main campus and hanging out with the Architect MVPs. I spent all morning + dinner with them yesterday. Some of these guys I hadn’t seen in nearly two years, so it was a ton of fun.
  • Nicolas Allen (aka Dr. Nick) is looking for SQL Service Broker users. I cornered him at the MVP dinner last night and gave him my thoughts on WCF + SSB. You can head over to his blog and do the same. (I’ll post my answers to his questions later today, hopefully)
  • The new Vista x64 driver for my workstation’s video card does support monior rotation, so I’m mostly XP free at this point. I’m dual-booting my workstation at this point, while I finish configuring stuff in the Vista partition. With both my workstation and work laptop tablet, I’m XP free at work. Next, I start getting home machines moved over.
  • Tom Hollander reports on a new drop of the Guidance Automation Toolkit. Mostly bug fixes like Vista support, but the full list is here. It’s an ugly upgrade process. You have to uninstall all existing guidance packages. I can’t wait until this technology is “integrate[d] … more deeply into Visual Studio and Team System”.
  • I mentioned the Podcast Authoring tool that I saw at TechFest last week. Herearesome pictures of it from engadget (via Loke Uei Tan)
  • I’ve been thinking of getting a Wii, and the fact I can hack code for it – managed code for managed snobs no less – is just another good reason to do it. (via DotNetKicks)

How I Learned to Stop Worrying and Love WCF

Regular readers of DevHawk are likely aware of my obsession interest in SQL Service Broker (aka SSB). I’ve also been doing a lot of WCF work lately. While there are parts of WCF that I think rock, overall I’ve found WCF lacking due to it’s lack of support for long running services, which SSB excels at.

So it was with great interest that I read this recent article on Integrating WF and WCF. WF is expressly designed for long running systems, so I wanted to see how the article dealt with the WCF’s lack of support for such scenarios. Unfortunately, the article basically sidesteps the issue. While it has lots of great info about hosting WF inside a WCF service, the article uses duplex channels for communication between the service and its clients. As I have pointed out before, this approach is impractical because it requires that both the service and its consumer remain alive in memory until the WF end.

Remember this quote from Essential WF?

“It is wishful thinking to assume that the operating system process (or CLR application domain) in which the program begins execution will survive for the required duration.”

So basically this WCF/WF sample is wishful thinking. Fine for a demo, but given the severe lack of information out there on integrating these two technologies, I’m worried that many people will read this article as best practice guidance, which in my opinion would be a mistake.

But instead of firing up my blog (that is, like last time) to write a scathing post about how broken this sample is, I emailed Paul which led to a concall with Shy to discuss WCF’s lack of support for long running services. Imagine my surprise when Shy agreed with me completely, furthermore saying that support for long running services had been “out of scope” for v1 of WCF. I thought that the whole point of duplex channels was for long running services. But apparently I was wrong.

Shy said to think of the duplex channel in terms of sockets, rather than long running conversations. And just like that, WCF made a ton more sense to me. I had been directly comparing the SSB and WCF communication models, but that’s apples and oranges. It would be like comparing SSB to TCP.

If you think about it, vanilla HTTP works a lot more like UDP, even though it’s layered on top of TCP. Both UDP and HTTP support connectionless operations and neither UDP nor HTTP are reliable or provide message ordering. The comparison isn’t perfect: for example, UDP isn’t limited to a single response for an incoming request. But by and large, HTTP is a very UDP style protocol.

If HTTP is basically UDP, then WS-* is trying to be TCP. Frankly, I never understood the point of WS-ReliableMessaging. I always thought reliability == durability == SSB or MSMQ. But when you realize that HTTP lacks TCP-like reliability and ordering capabilities, suddenly this WS spec makes sense. In fact, Shy made this exact point almost a year ago. At the time, I didn’t get it because I didn’t understand the duplex channel as sockets analogy. Now, I see the value of adding these capabilities to HTTP.

What Shy said was clear and to the point but unfortunately completely missing in the official WCF documentation. For example, the docs on Duplex Services say this:

A duplex service contract is a message exchange pattern in which both endpoints can send messages to the other independently. A duplex service, therefore, can send messages back to the client endpoint, providing event-like behavior. Duplex communication occurs when a client connects to a service and provides the service with a channel on which the service can send messages back to the client.

The docs make no mention that the “event-like behavior” of duplex services only works within a session. And I’m not the only one who mistakenly believed that duplex services could be used for long running services (here’s an article in DDJ that makes the same mistake). Shy used the term “episodic” to describe services that span session boundaries. I’d like to see the docs updated to include that concept.

Taking the TCP/UDP analogy even further, I think it demonstrates how pointless the REST vs. SOAP debate is. As UDP is a thin layer on top of IP, REST is a thin layer on top of HTTP. But nobody argues much about UDP vs. TCP these days. I was in grade school when UDP and TCP were standardized, so maybe there were big TCP vs UDP flame wars at the time. But twenty five years later, it’s pretty clear that TCP vs UDP is not an either-or proposition. Some protocols are better built on UDP while others are better built on TCP. I’m guessing we’ll see a similar evolution with SOAP and REST.

Personally, I would expect that message exchanges between services will become more complex over time. Complex message exchanges would seem to favor stateful SOAP over stateless REST for the same reason complex network protocols favor connection-oriented TCP over connectionless UDP. But SOAP could never displace REST any more than TCP could ever displace UDP. Furthermore, as Larry O’Brien recently wrote “the onus is on the WS-* advocates to prove the need”. TCP standardization only lagged a year behind UDP standardization where WS-* has lagged at least six years behind REST. I wonder if UDP would be more prevalent today if it had gotten a six year head start on TCP.

Finally, this “SOAP as Sockets” flash of understanding has also helped me understand how SSB / WCF can evolve together in the future. Some folks have suggested an SSB transport for WCF and I’ve personally looked into such an approach. But given since SSB is at a higher level of abstraction than WCF, it makes much more sense to layer SSB on top of WCF instead of the other way around. Today, SSB uses two protocol layers: the top level Dialog Protocol, which is built on top of the lower-level Adjacent Broker Protocol (ABP), which in turn is built on TCP. I’d like to see a version of ABP that was built on top of WCF instead of directly on top of TCP. SSB’s Dialog Protocol would tie together the WCF duplex sessions into a long-running conversation the same way that it ties together TCP sessions today.

Eventually, I would love to see something that has the programming semantics of SSB and the interoperability of WCF. That would be like the the Reese’s Peanut Butter Cup of service messaging.