Morning Coffee 121

• My daughter had her tonsils & adenoids out on yesterday. It was a routine procedure and it went by-the-numbers, but any parent will tell you it's hard to see your kid in a hospital bed.
• Given the previous bullet, I'm not at the SOA/BPM conference for the big announcement. Don't worry, there's lots of other folks covering the news.
• It was a crappy sports weekend in the Pierson house. Va Tech snatched defeat from the jaws of victory, Southern Cal never led at Oregon, the Capitals lost twice, and the Redskins got blown out by the Pats. At least the Caps won big yesterday in Toronto.
• Speaking of the Capitals, Peter Bondra retired Monday. I still think it's a travesty that he didn't spend his whole career in DC, but I've made my peace with it.
• Nick Malik has a great series on business operations models and how they apply to SOA. Regular readers should be unsurprised that I favor low standardization, though I can see the value of high integration. That makes the Coordinated Operating Model my fav, though I can see the benefit of the Diversified Model as well. I can't wait to read what Nick has to say on changing models.
• Speaking of Nick, I'm doing a roundtable with him on "Making SOA Work in the Enterprise" @ the Strategic Architect Forum. Should be fun. Sorry for the lack of linkage on this, but it's an invite-only event.
• Jezz Santos has a new series of white papers on building software factories. First up "Packaging with Visual Studio 2005"
• Aaron Skonnard has a new whitepaper on using the WCF LOB Adapter SDK with BTS 2006 R2. I've been building one of these things recently, so I'm looking forward to checking that out. (via Sam Gentile)
• Tim Ewald looks at Resource Oriented Architecture (when did ROA become a TLA?) and wonders "what if your problem domain is more focused on processes than data?" I wonder that all the time. (via Jesus Rodriguez)
• It's not just durable messaging - Libor Soucek also disagrees with my opinions on centralized control. I agree 100% with Libor that centralized management would make operation's lives "much, MUCH easier" as he puts it. However, that doesn't make it feasible at any significant scale. Furthermore, I wouldn't describe an approach that requires that "all services adopt [the] same common management interface" as "pragmatic". Frankly, just the opposite.

Morning Coffee 117

• Quick update to the DevHawk 2007 World Tour: I won't be making it to the SOA & BP Conference. Riley's having her tonsils out. As much as I'd like to hang with my geek peeps, family is the priority. But I can still make an evening event or geek dinner later in the week if anyone is game.
• Caps season-opening winning streak continues. Still 100% on the PK, though the power play is pretty anemic. As I said yesterday, it's WAAAAY to early in the season to start bragging, but starting strong is much better than starting weak.
• Speaking of hockey, looks like the NHL Network is launching in the US this month (it's been available in Canada since 2001). Also, NHL.tv is up and running. Those wishing to see Caps highlights can go directly to Capitals.NHL.tv. Unfortunately, if you want to see full games, you've got to subscribe to Center Ice or Center Ice Online to the tune of $150. But I don't want to get "up to 40 games each week", I just want the Caps games. Between the time zone difference and kids, it's not like I have time to watch that much hockey anyway. Why can't I subscribe to just the Caps games online for say $25 a season?
• Finished Halo 3 Sunday night. Fun game and a great end of the trilogy. Looking forward to what the newly-independent Bungie does next. Something tells me we haven't seen the last of Master Chief. However, I do think Bioshock has better and more original storytelling. Mass Effect looks like it'll be better still.
• Sam Gentile pointed out that his Neudesic colleague David Pallmann has posted a series of WCF tips. Several of them are right on the money like "Take Advantage of One Way Operations" and "Use a Discovery Mechanism to Locate Services". However, I can't agree with "Maintain a Service Catalog". David warns that if you don't, "The left hand won’t know what the right hand is doing." Of course, that's probably the case regardless of how you maintain your service catalog. And "Retry on minor failures"? That's fine, if you've got an idempotent operation. Unfortunately, most non-read operations aren't idempotent unless you take the time to design them that way. And most people don't.
• Speaking of Sam, he's blown up his CodeBetter blog and walked away from the ALT.NET crowd. I've not been a fan of this ALT.NET stuff since it surfaced - as Sam said, "ALT.NET is a divisive thing" - so I'm happy to see my good friend walk away from it.
• Speaking of ALT.NET, Scott Hanselman blogged about previewing the new ASP.NET MVC Framework at the ALT.NET conference. Like Sam, Scott thinks the term ALT.NET is "too polarizing". I like Scott's suggestion for Pragmatic.NET. Oh, and the MVC framework stuff looks cool too.
• Reading Dare's description of OAuth gave me a distinct sensation of deja-vu.

Durable Services with Fake Persistence

I've been investigating the new WCF/WF integration in .NET Framework 3.5. I want to understand how the new context features work. Unfortunately, there's not much info out there (that I could find at any rate). You're pretty much stuck with the samples and Jesus Rodriguez's overview of durable services. So I sat down to dig a little deeper.

Note, since there are no docs on this stuff as I write this, many of the links below are Reflector code links.

I started by lifting the DurableCalculator sample contract and service implementation and dumping it into a new WCF Service Library project. I did this for two reasons. First, VS08 has added a WCF Service Host much like VS05 added the ASP.NET Development Server. Very cool. But the existing sample is still written to be hosted in IIS, so I wanted to change that. Second, and much more important, I wanted to start with a vanilla config file. I knew it wouldn't work out of the box, but the point of this exercise was to learn how this works under the covers.

When you fire up the durable service with the vanilla config file, you get an error (as expected). Services marked with the DurableServiceAttribute require a binding that supports the context protocol. WsHttpBinding, the default binding when you create a new service, doesn't. However, it's easy to fix by switching to wsHttpContextBinding instead. Via Reflector, we see that wsHttpContextBinding inherits from wsHttpBinding and adds a ContextBindingElement to the binding element collection created by the base class. BasicHttpContextBinding and netTcpContextBinding work the same way.

Even after changing to wsHttpContextBinding, we're still getting an error on service start. But it's a new error, so we're making progress. Now, we're told that services marked with DurableServiceAttribute need a persistence provider to be specified. If we look in the original sample's web.config file, we find a persistenceProvider element in the service behavior. This element references the SqlPersistenceProviderFactory type. Obviously, the point here is to persist durable service instances to the database between calls, much as WF can do.

However, merely configuring the existing SQL persistence provider doesn't really tell you what's going on under the hood. Besides, often when you're experimenting, you don't really want to go thru the headache of setting up a SQL store for persisting instances to. Somewhere along the line, I implemented a fake persistence service for WF that stored the serialized instances in memory. So I decided to do the same for WCF durable services.

Building a WCF Persistence Provider requires building two classes: a factory and the provider itself. Factories inherit from PersistenceProviderFactory, which exposes only one non-CommunicationObject method: CreateProvider. It appears that the service host creates a single persistence provider factory and calls CreateProvider whenever it needs a persistence provider. Providers themselves inherit from PersistenceProvider, which exposes methods to Load, Save and Delete durable service instances.

My FakePersistenceProvider (and factory) are brain dead simple, though the ratio of "real" code to factory and CommunicationObject scaffolding is quite low (about 40 lines out of 258). The factory keeps a dictionary of serialized service instances, keyed by guid. When providers are created, this key guid is passed as a parameter to CreateProvider. The provider instances delegate Load, Save and Delete back to internal methods on the factory class. The methods themselves use the NetDataContractSerializer to serialize the service instance out to a byte array and deserializing it back again. I chose NetDataContractSerializer because that's what the SQL persistence provider uses under the hood.

PersistenceProvider supports async versions of Load, Save and Delete but I didn't implement them. Also, there's a LockingPersistenceProvider abstract class which adds (you guessed it) instance locking semantics. However, my fake provider doesn't span machines, much less require locking semantics so I skipped it.

So it looks like DurableServiceAttribute, context-supporting bindings and persistence providers are all inter-related. Certainly, you can't use the attribute with out the binding and persistence provider. As I continue to dig, I'm want to see how context inter-relates with WF as well as it's possible usage outside of DurableServiceAttribute based scenarios.

If you're interested in the code, I've stuck it up on my SkyDrive. In addition to the FakePersistenceProvider implementation and the simple Durable Calculator service, it includes a simple client to test the service and persistence provider. The WCF Service Host includes a test client, but it doesn't appear to support the context protocol, so I had to build a simple test app instead. Enjoy.

Morning Coffee 113

• I'm in Chicago today and tomorrow for a reunion of sorts. In my last job, I managed a group of external architects called the Microsoft Architecture Advisory Board (aka the MAAB). We discontinued the program a while back, but the core of the group found the program valuable enough they have continued to meet anyway. I found the MAAB meetings incredibly valuable and insightful, so I'm really excited to be invited to continue my involvement with the group.
• I picked up Bioshock Tuesday (Circuit City had it on sale) on my way to my bi-weekly campus excursion. My meetings were over around 2pm so I headed home early, expecting to surprise the kids. But Jules had decided to skip naps and go shopping with them. Her cell phone was dead, so I ended up at home with a couple of hours to myself and a brand new copy of Bioshock. Wow, is that a good game. Certainly deserving of the amazingly good reviews it's garnered.
• Speaking of reviews, this transparently biased review of Bioshock over at Sony Defense Farce Force is frakking hilarious. Somehow, I doubt their dubious review will stem the tidal wave of Bioshock's well-deserved hype. Can't wait to read their Halo 3 review.
• Pat Helland writes at length on master-master replication. I reformated it into PDF so I could read it - the large images were messing up the text flow on my system. As usual for Pat, there's gold in that thar post. His thoughts on DAGs of versions and vector clocks as identifiers are very exciting. However, I think he glosses over the importance of declarative merging. I would think programmatic merge would likely be non-deterministic across nodes. If so, wouldn't you end up with two documents with the same vector-clock identifier by different data?
• Joe McKendrick points to a few people who predict the term "service-oriented" will eventually be subsumed under the general heading of "architecture". Not to brag, but I made that exact same prediction almost three years ago.
• Erik Johnson thinks that SOA 2.0 centers on transformational patterns. The idea (I think) is that if systems "understand each other more deeply", then we can build a "smarter stack" and design apps via new constructs to promote agility and simplicity. Personally, I'm skeptical that we can define unambiguously system semantics except in the simplest scenarios, but Erik talks about using "graph transformation mathematics" to encode semantics. I don't know anything about graph transformation mathematics, but at least Erik has progressed beyond hand waving to describing the "what". Here's looking forward to the "how".
• New dad Clemens Vasters somehow finds time to implement an XML-RPC binding for WCF 3.5. I was encouraged that it didn't require any custom attributes or extensions at the programmer level. Of course, XML-RPC fits semantically into WCF's interface based service model, so it shouldn't be a huge surprise that it didn't require any custom extensions. But did it need WCF 3.5? Would this work if recompiled against the 3.0 assemblies?
• Phil Haack writes a long post on Duck Typing. VB9 originally supported duck typing - the feature was called Dynamic Interfaces - when it was first announced, but it was subsequently cut. I was really looking forward to that feature. Between it and XML Literals, VB9 was really stepping out of C#'s shadow. I guess it still is, even without dynamic interfaces.
• Since I've been doing some LINQ to XML work lately, I decided to go back and re-write my code in VB9 using XML literals. While XML literals are nice, I don't think they're a must have. First, LINQ to XML has a nice fluent interface, so the literals don't give you that much cleaner code (though you do avoid writing XElement and XAttribute over and over.) Second, I find VB9's template syntax (like ASP <%= expression %>) clunky to work with, especially in nested templates. Finally, I like the namespace support of XNames better. As far as I can tell, VB9 defines namespaces with xmlns attributes just like XML does. So I'm not dying for literal XML support in a future version of C#. How about you?

Morning Coffee 111

• I'm not sure if I should laugh or cry at Nick Malik's definition of politecture. I mean, it's funny so I'm laughing, but it's so true that it makes me want to cry.
• Don Box comments on retiring the tenets. It's good to see him say "please God tell me we can do better" than CLR interfaces or WSDL.
• Looks like the P2P APIs are finally getting the managed treatment in .NET FX 3.5. A long time ago, John deVadoss asked me what an enterprise system like CRM might look like if it used a peer-to-peer approach instead of client-server. If I had any free time, I'd prototype one out on this API. (via Mike Taulty)
• Scott Guthrie goes back to his LINQ to SQL series to tackle Stored Procs and UDFs. Being able to use UDFs inline with LINQ queries is very cool. However, it seems to me that LINQ discourages the use of stored procs. As a developer, I'd rather write LINQ queries than stored procs, if I can. The probably puts me at odds with DBAs who'd rather all DB access be via stored procs they control.
• Soma writes about new MSBuild enhancements in VS08: multi-targeting and parallel build.
• I just discovered Vista Battery Saver. Basically, it turns off Aero and Sidebar when you're on battery. I'm traveling to Chicago next week, so we'll see if it has much impact on my battery life. (via Plenty of Code and Larkware)

Morning Coffee 110

• Monday @ Gamefest, the XNA team announced XNA Game Studio 2.0. The two big new things are support for the entire VS product line (1.0 only works on VC# Express) and the addition of networking APIs. Let's Kill Dave has a good wrapup of the announcements from Gamefest Day One.
• Speaking of Xbox 360, I played thru the demos of Stranglehold and Bioshock. Two thumbs up on both. It's gonna be an expensive year for Xbox gamers.
• Mark Cuban noodles on taking your house public. "Why not create a market or exchange where homeowners can sell equity in their homes?" I've thought about this myself from time to time. However, Mark thinks making it happen would "probably take the country's biggest banks working together". I wonder if there's a more Web 2.0 social lending approach that would work better.
• Jeff Atwood calls virtualization as "the next great frontier for computer security". I agree 100%. But I don't think the action is going to be in "full-machine" virtualization like Virtual PC. Rather, it's going to be sandbox virtualization. Jeff mentions GreenBorder (now part of Google) but it's not the only solution. Some time ago, Microsoft acquired SoftGrid which uses sandbox virtualization for application deployment, but using SystemGuard for security sandboxing seems like a logical step.
• The WCF LOB Adapter SDK has released. Sonu Arora has the details. As part of the Integration team @ MSIT, I have a feeling we're going to become fairly familiar with this technology. (via Jesus Rodriguez).
• Speaking of Jesus, he thinks the six new SCA4SOA committees are "going to help". Why? Because inventing technology in committee has turned out so well in the past?
• John deVadoss cements BPM's fad du jour status by contrasting "big" BPM and "little" BPM. It's fairly obvious to me that big *anything* just doesn't work in the enterprise. But I worry that little *anything* doesn't work that well either. So how long until someone (probably Nick) starts arguing for "middle out" BPM?
• David Bressler wonders "What is it about registries that everyone thinks is a panacea for all things SOA?" Amen, Brother! Joe McKendrick claims it's required for governance, but then gets to what I think is the *real* reason for focus on registries: the "registry is a tangible offering" that vendors can sell. Just because it's productizable doesn't mean you need it.
• Hartmut Wilms responds to my retire the tenets post, but he seems to contradict himself. On the one hand, he suggests that "the four tenets just expressed, what “almost” everybody outside the MS world knew already". But then he goes on to dispute that the SO paradigm shift has even occurred! Hartmut, I'll grant you that WCF (among other similar stacks) are way too focused on "you write the classes, we'll handle the contracts and messages". On the other hand, if you don't provide a productive interface that most everyone can pick up and run with, the technology won't get adopted in the first place.

Retire the Tenets

John Heintz and I continue to be in mostly violent agreement. It's kinda like me saying "You da architect! Look at my massive scale EAI Mashup!" and having him respond "No, you da architect! The SOA tenets drive me bonkers!" Makes you wonder what would happen after a few beers. What's the architect version of Tastes Great, Less Filling? [1]

Speaking of the tenets, John gives them a good shredding:

Tenet 1: Boundaries are Explicit
(Sure, but isn't everything? Ok, so SQL based integration strategies don't fall into this category. How do I build a good boundary? What will version better? What has a lower barrier to mashup/integration?)

Tenet 2: Services are Autonomous
(Right. This is a great goal, but provides no guidance or boundaries to achieve it.)

Tenet 3: Services share schema and contract, not class
(So do all of my OO programs with interface and classes. What is different from OO design that makes SOA something else?)

Tenet 4: Service compatibility is based upon policy
(This is a good start: the types and scope of policy can shape an architecture. The policies are the constraints in a system. There not really defined though, just a statement that they should be there.)

Ah, I feel better getting that out.

As John points out, the four tenets aren't particularly useful as guidance. They're too high level (like Mt. Rainier high) to be really actionable. They're like knowing a pattern's name but not understanding how and when to use the actual pattern. However, I don't think the tenets were ever intended to be guidance. Instead, they were used to shift the conversation on how to build distributed applications just as Microsoft was introducing the new distributed application stack @ PDC03.

John's response to the first tenet makes it sound like having explicit boundaries is obvious. And today, maybe it is. But back in 2003, mainstream platforms typically used a distributed object approach to building distributed apps. Distributed objects were widely implemented and fairly well understood. You created an object like normal, but the underlying platform would create the actual object on a remote machine. You'd call functions on your local proxy and the platform would marshal the call across the network to the real object. The network hop would still be there, but the platform abstracted away the mechanics of making it. Examples of distributed object platforms include CORBA via IOR, Java RMI, COM via DCOM and .NET Remoting.

The (now well documented and understood) problem with this approach is that distributed objects can't be designed like other objects. For performance reasons, distributed objects have to have what Martin Fowler called a "coarse-grained interface", a design which sacrifices flexibility and extensibility in return for minimizing the number of cross-network calls. Because the network overhead can't be abstracted away, distributed objects are a very leaky abstraction.

So in 2003, Indigo folks came along and basically said "You know the distributed object paradigm? The one we've been shipping in our platform since 1996? Yeah, turns out we think that's the wrong approach." Go back and check out this interview with Don Box from early 2004. The interviewer asks Don if WCF will "declare the death of distributed objects". Don hems and haws at first, saying "that's probably too strong of a statement" but then later says that the "contract, protocol, messaging oriented style will win out" over distributed objects because of natural selection.

The tenets, IMHO, were really designed to help the Windows developer community wrap their heads around some of the implications of messaging and service orientation. These ideas weren't really new - the four tenets apply to EDI, which has been around for decades. But for a generation of Windows developers who had cut their teeth on DCOM, MTS and VB, it was a significant paradigm shift.

These days, with the tenets going on four years old, the conversation has shifted. Platform vendors are falling over themselves to ship service/messaging stacks like WCF and most developers are looking to these stacks for the next systems they build. Did the tenets do that? In part, I think. Mainstream adoption of RSS was probably the single biggest driver of this paradigm shift, but the tenets certainly helped. Either way, now that service orientation is mainstream, I would say that the tenets' job is done and it's time to retire them. Once you accept the service-oriented paradigm, what further guidance do the tenets provide? Not much, if any.

[1] Not that you would catch me drinking Miller Lite. Ever.

Another InitImportantThing Approach

I thought of another approach to the InitImportantThing problem that I blogged about yesterday. I think it's a bit harder to code, but it's certainly explicit and avoids the magic method that Jon dislikes so much.

The crux of the problem is that ServiceHostBase needs a valid ServiceDescription in order to operate. The WCF team chose to provide said description to ServiceHostBase via the abstract CreateDescription method. But as we saw, ServiceHostBase can't call CreateDescription from it's own constructor. So instead, derived classes are forced to call InitializeDescription in their own constructor. Since that call isn't enforced by the compiler, it's easy to forget to include it. Since the exception that gets thrown doesn't really tell you what went wrong, it's easy to spend hours trying to figure it out.

So here's a better approach: since the ServiceHostBase needs a valid ServiceDescription in order to operate, why not pass it in as a constructor parameter?

ServiceHostBase has a protected constructor with no parameters. But since it needs you to call InitializeDescription in your derived class constructor, it really needs the ServiceDescription, a collection of ContractDescriptions (also returned from CreateDescription) and a collection of base addresses (passed into InitalizeDescription). If these were parameters on ServiceHostBase's constructor, it could validate that information directly, without needing abstract or magic methods.

The one problem with this approach is that the creation of a ServiceDescription is non-trivial. ServiceHost's implementation of CreateDescription generates the ServiceDescription by reflecting over the service type. You still need that code, but now you would call it from the base constructor initializer instead. That means it has to be a static method, but otherwise it would work just fine. Here's yesterday's code, updated for this approach:

This seems like the best approach to me. You remove the un-obvious magic method call requirement when deriving your own service host while still enforcing the data consistency check in the base class during construction. Best of both worlds, right?

So I wonder why the WCF team didn't do it this way? 

Early Afternoon Coffee 105

• My two sessions on Rome went very well. Sort of like what I did @ TechEd last month, but with a bit more kimono opening since it was an internal audience. Best things about doing these types of talks is the questions and post-session conversation. I've missed that since moving over to MSIT.
• Late last week, I got my phone switched over to the new Office Communications Server 2007 beta. In my old office, I used the Office Communicator PBX phone integration features extensively. However, when we moved we got new IP phones that didn't integrate with Communicator. So when a chance to get on the beta came along, I jumped. I'll let you know my impressions after a few weeks, in the meantime you can read about Mark Deakin's experience.
• Matevz Gacnik figures out how to build a transactional web service that interacts with the new transactional file system in Vista and Server 08. Interesting, but personally I don't believe in using transactional web services. The whole point of service orientation is to reduce the coupling between services. Trying two services (technically, a service consumer and provider) together in an atomic transaction seems like going in the wrong direction. Still, good on Matevz for digging into the transactional file system.
• Udi Dahan gives us 6 simple steps to being a "top" IT consultant. I notice that getting well known, speaking and publishing are at the top of the list but actually being good at what you're well known for comes in at #5 on the list. I'm sure Udi thinks that's implicit in becoming a "top" consultant, but I'm not so sure.
• Pat Helland thinks Normalization is for Sissies. Slide #6 has the key take away: "For God's Sake, Don't Normalize Immutable Data".
• Larry O'Brien bashes the new binary efficient XML working group and working draft. I agree 100% w/ Larry. These aren't the droids we're looking for.
• John Evdemon points to a new e-book from my old team called SOA in the Real World. I flipped thru it (figuratively) and it appears to drill into the Foundations of Solution Architecture as well as provide real-world case studdies for each of the pillars recurring logical capabilities. Need to give it a deeper read.

Morning Coffee 102

Seems like a slow week.

• Jules and I went to see the latest Harry Potter movie this past weekend. It's easily the weakest of the six HP stories so far. The first two stories were about discovering this magical world, the next two about discovering Harry's past, and the last two about confronting said past. That leaves OotP as the odd-story-out, mostly bridging from the end of the fourth story to the start of the sixth.
• Speaking of movies, the new movie feature of Mobile Search v2 rocks, though I have two quick suggestions. First, it would be nice to have a time-sorted view of when a given movie is playing. So if it's playing at 4pm at one theater and 4:30pm at another, you'd see them in a list ordered that way. Second, how about an option to buy tickets directly from the phone?
• If you're interested in WPF and 3D, Eric Sink has a series for you.
• Old news, but Windows Home Server RTMed on Monday. I'm really looking forward to this product.
• I was looking for some information on how WCF pumps messages in the service host and I found this post from Maheshwar Jayaraman. Between that post and Reflector, I think I've got a good handle on how ChannelDispatcher works.
• Larry O'Brein calls out three MS Research Projects. Microsoft Research Accelerator is a high-level data-parallel library that targets GPUs. Graph Layout Execution Engine (aka GLEE) is a library for graph layout and viewing. VirtualEarth MapCruncher converts existing maps (PDF and bitmaps) to work with Virtual Earth.
• Ted Neward weighs in on the David Chappell's Korean War REST vs. WS-* analogy. Skim the history lesson, but make sure you read his points about security and reliability interop. WS-* has addressed these areas, so if you need those capabilities, why wouldn't you use WS-* to get them rather than re-invent the wheel? As for the history lesson, Ted does say he thinks software development is more analogous to making war than building a house. He expands on that idea and recommends Robert Greene's The 33 Strategies of War. I want to read the book and mull it over a bit, but I certainly see where Ted's coming from.

Morning Coffee 91

• My wife loves me. I'm a very lucky man.
• I'm starting to really dig Safari Books Online. Having a tablet really helps here, I can sit in bed and read and it's ALMOST like reading a real book. Is there an offline experience? Something like the NYTimes WPF Reader app would be killer.
• I'm not a Twitter guy, but I like the idea of using it to publish CI results. Not quite as cool as using the Ambient Orb, but close. (via DotNetKicks)
• Soma details the dogfood usage of TFS in Developer Division. Sorta interesting if you're into knowing that stuff. Brian Harry apparently has much more.
• I realize that linking to Pat Helland every time he writes something is fairly redundant. If you want his feed, you know where to find it. But he writes great stuff! The latest is Accountants Don't Use Erasers, which talks about append-only computing. His point that the database is a cache of the transaction log is mind blowing, yet makes total sense.
• Bruce Payette blogs a PS DSL for creating XML documents.
• Jesus Rodriguez details WCF's new Durable Service support in .NET 3.5. I get the need for the [DurableServiceBehavior] attribute, but do I really have to adorn each of the service methods with [DurableOperationBehavior] too? That seems redundant. Also, I wonder how this looks at the channel layer?
• Speaking of WCF's channel layer, I recently picked up a copy of Inside Windows Communication Foundation by Justin Smith. This is the first book I've found that has more coverage of the channel layer than the service layer, so I like it.
• Dare writes about Web3S, Windows Live's general purpose REST protocol. Apparently, WL started with Atom Publishing Protocol, but found that it didn't meet their needs around hierarchy and granular updates. David Ing says it's "not that similar" to my concept of REST, but I going to read the spec before I comment.
• Scott Hanselman writes about how he learned to program and some thoughts about teaching his son. Patrick has recently started expressing interest in programming (he want's to do what Daddy does). At four, I'm thinking I'll start him on Scratch (though ToonTalk looks interesting). As he gets older, I was thinking about Squeak, though I'm a smalltalk noob. I really like Scott's idea of creating a connection to the physical world via something like Mindstorms. Patrick loves Lego almost as much as his dad, so that would be cool.

Morning Coffee 75

• 3D Printing is going to be huge. According to the NYT, we'll be looking at around $1,000 for one within four years. For the impatient, check out Fab@Home and build one right now.
• It's been a while since I experimented with the P2P stack in Windows, but it looks like it's getting the managed treatment in VS "Orcas".
• The managed Ruby hits keep on coming. Last week was DLR and IronRuby. This week it's a new drop of Ruby.NET which includes VS integration.
• Looks like Sun is trying to get back into the Ajax/Flash/Silverlight fray with JavaFX Script. I wrote over a year ago that "In platform portability, Flash has succeeded where Java failed." I can't help but believe JavaFX is too little too late. Also, it's yet another Java technology name that sounds like it's been blatantly copied from MSFT. JDBC, JSP, JDO ... What's next? JINQ? (via TechMeme)
• Steve Maine has a great series of posts on the new Web Programming Model that's coming in .NET 3.5 and is currently being previewed as part of the BizTalk Services SDK. But it was his Balancing reach and rich post that I found most illuminating. The first version of WCF feels hopelessly bound to the WS-* view of the world, which makes it difficult to incorporate alternative messaging models into the same programming model. I've run into this trying to use SSB with WCF. In the next version, that WCF / WS-* marriage looks like it's getting a little more open. In my current role, I'm not so interested in the web programming model, but I am very interested in how they are integrating these alternative models.

Morning Coffee 58

• Nicholas Allen points out that Messaging is not a Transaction. Of course, what he really means is that Messaging with WCF is not a Transaction. Messaging with SSB is in fact a transaction (well, more accurately, it's transactional but you get the idea). Nick's right that you may not need anything more than simple retry semantics that you can implement yourself in your application protocol. But SSB has spoiled me. Why should I have to write the retry semantics? Why can't my messaging stack provide that for me? Or put another way, if I need a "precicely defined failure model", wouldn't I be better of choosing a messaging stack that provides that out of the box?
• Steve Hartman explains how to use span Remote Desktop Connection across multiple monitors for use with VPC. I wonder if this will work on my machine, since my multi-monitor setup is an L shape, not a rectangle. I'm offsite today, so I'll try it tomorrow (via DotNetKicks)
• According to news reports, Microsoft is negotiating to provide DRM free music too. I wonder how well this will work for Microsoft given that both Zune and PlaysForSure provide subscription services. I would assume the $15 all-you-can-listen style subscription services wouldn't be DRM free. Given that program is one of the (few) selling points they have over Apple, will the availablity of DRM Free music undercut the interest in subscription? (via Dale)

Answering Dr. Nick's Questions on SSB & WCF

Nick Allen asked on his blog about how people would like to see SSB and WCF work together. He's already heard these from me, but I figured I'd put them out there for everyone to see and debate. Plus, I had several beers last night at the MVP dinner, so this is likely to be more coherent than I was yesterday! :)

1. Are you interested in SSB because you'd like to have your service closer to the database? How close is close enough to the database?

I've first blogged about the endangered middle tier almost three years ago. My point at the time was that as you break your monolithic system up into services, the vast majority of those services won't need to scale out. You performance gets better the closer you are to the data. If you don't need to scale out, why not get the maximum boost by running in the database process itself?

Furthermore, in large IT shops, the database files are stored out on the SAN rather than on hard drives attached to the database server itself. That means the database server is effectively stateless. Why add a second stateless tier if you don't need scale out? If you need more performance in a given service, you can detach the database file from it's current SQL server box and attach it on another more beefy SQL server box without physically moving the database files at all. This enables what I call the "Star Trek Effect", where you can shift computing power where it is needed most (more power to the payroll system!).

Of course, if you're going to move the service, you do need to bring it down for a short time. That implies a need for durable messaging so that service consumers aren't affected by the brief service interruption. Which brings us to...

2. Are you interested in SSB because you need durable, duplex messaging between two services? Do you need exactly-once-in-order message delivery?

Yes. SSB has a bunch of other nice features, but durable duplex messaging is what I need the most. Exactly-once-in-order is also fairly critical, though there may be scenarios where it's not really necessary. Those are the exception, not the rule however.

Doesn't WS-RM already do EOIO already?

3. Are you interested in using SSB from WCF because you want a better asynchronous messaging experience than MSMQ? What makes you prefer SSB to other queuing products?

My primary problem with MSMQ for the problems I'm tasked with solving is that MSMQ is one way while SSB supports duplex messaging. You could do duplex messaging with MSMQ if you didn't mind managing multiple queues (one for each side of the conversation) but SSB does this for you for free. I'm sure there are scenarios where pure one-way messaging are useful, but they are few and far between in my day job.

Furthermore, SSB has the explicit idea of a service instance (they call it a conversation group) which MSMQ lacks. SSB's implementation is conceptually similar to the new WCF/WF integration work in the latest Orcas CTP.

Finally, SSB uses logical naming. You have conversations between services, but services get mapped to physical addresses at the routing layer. This allows services to move around more easily (see the "Star Trek Effect" in #1 above). Both MSMQ and WCF use physical addresses, which makes them much more difficult to move.

4. Are you interested in having your data contracts defined in WCF, SQL, or both?

I like WCF's data contract infrastructure. We did a early prototype long-running service with both WCF and SSB. the messaging stack code was obviously different, but we used the exact same data contract code. I even wrote some code to automatically deserialize the SSB message by mapping the SSB message type to a data contract.

I want my services to run inside the database, but that doesn't mean I want to write them in T-SQL. Personally, I'm much more productive in C# and/or WF. So WCF data contracts are fine by me.

Morning Coffee 45

• Yesterday's morning coffee was canceled on account of going to main campus and hanging out with the Architect MVPs. I spent all morning + dinner with them yesterday. Some of these guys I hadn't seen in nearly two years, so it was a ton of fun.
• Nicolas Allen (aka Dr. Nick) is looking for SQL Service Broker users. I cornered him at the MVP dinner last night and gave him my thoughts on WCF + SSB. You can head over to his blog and do the same. (I'll post my answers to his questions later today, hopefully)
• The new Vista x64 driver for my workstation's video card does support monior rotation, so I'm mostly XP free at this point. I'm dual-booting my workstation at this point, while I finish configuring stuff in the Vista partition. With both my workstation and work laptop tablet, I'm XP free at work. Next, I start getting home machines moved over.
• Tom Hollander reports on a new drop of the Guidance Automation Toolkit. Mostly bug fixes like Vista support, but the full list is here. It's an ugly upgrade process. You have to uninstall all existing guidance packages. I can't wait until this technology is "integrate[d] ... more deeply into Visual Studio and Team System".
• I mentioned the Podcast Authoring tool that I saw at TechFest last week. Here are some pictures of it from engadget (via Loke Uei Tan)
• I've been thinking of getting a Wii, and the fact I can hack code for it - managed code for managed snobs no less - is just another good reason to do it. (via DotNetKicks)

Morning Coffee 40

• My boss let me borrow a Tecra M4 that he scavenged from his boss. The display is fairly twitchy, I think it's a motherboard issue. But it's very intermittent and I'll get help desk to take a look. In the meantime, it sure is nice to driving a Tablet PC again. And it's Vista ready to boot.
• Speaking of Vista, Visual Studio 2005 Service Pack 1 Update for Windows Vista. It's a mouthful but it's now available. Soma answers questions about the new release on MS PressPass.
• The DSL tools team keeps on rolling with the power toys. First it was the Designer Integration PowerToy, now it's the DSL Tree Grid Editor PowerToy. Jeff Santos has the details.
• I missed the TechFest keynote yesterday, but it's available on demand. They also have descriptions and videos of some of the technologies on display. (well, only one video so far, but I assume since the page is labeled "TechFest 2007 Videos" that more are on the way.)
• There's new support for integrating WCF and WF coming in VS "Orcas". Moustafa Khalil Ahmed has the details on what's new for WF & WCF in the latest CTP drop. For me personally, the WCF/WF integration is some of the most important stuff coming in Orcas, second only to LINQ.

Morning Doughnuts 4

• According to Reuters surgeons who play video games are more skilled. Remind me to ask the doctor if s/he owns an XBOX 360 the next time I am getting operated on.
• I have reached the National Championship game in dynasty mode of NCAA Football 2007. The opponent of my BYU Cougars...why that would be Harry's alma mater, the USC Trojans. Funny how that worked out.
• Nicholas Allen writes in his blog about when you should use Indigo to write a channel, and more importantly when you should not. As most of you know Harry and I are doing quite a bit of work with WCF so we are interested in this type of advice.
• Our team has been thinking about how to manage a large number of services in an automated fashion. This would include deploying new services, monitoring the services, automatically handling scaling, service discovery, and automated provisioning to name a few possible capabilities. I almost think of it like the next version of UDDI, especially when it comes to provisioning. I think that as systems become more distributed that the ability to automatically manage these systems is going to be key to their success. I know that some thought has already gone on in this area by people far smarter than I, but as I consider how to operate an infrastructure with thousands of services in it it is apparent that the opportunity is there for us to design and implement a system management framework that automates the majority of the tasks. I need to spend some time to consider how the framework would work, and document the capabilities.

Reliably Beating a Dead Horse

(Harry is on a secret mission in uncharted space this week, so instead of the daily Morning Coffee post, you get a series of autoposted essays. This post builds on Harry's recent epiphany about WCF and long running services)

Way, way, way back in March of 2003, IBM and Microsoft published an "overview and roadmap" white paper entitled "Reliable Message Delivery in a Web Services World". It contained the following paragraph under the section "Exchanging Messages Reliably":

WS-ReliableMessaging is not bound to underlying transport protocols or sessions. This means that the lifetime of a WS-ReliableMessaging conversation can span long periods of time (days, weeks) even when one or both systems are rebooted. This allows conversations to be suspended mid-stream (for example, to allow system maintenance) and then resumed without needing to retransmit the entire conversation. [emphasis added]

Now I know how I got confused about WCF and long running services in the first place. Support for long running services was part of the original web services vision!

About three years after that white paper was published, Shy Cohen wrote a post entitled Reliable Messaging Demystified on his blog. Shy was at one time the feature owner of WS-RM in WCF (according to his post) and wrote the following:

Reliable sessions [in WCF] are implemented using the WS-ReliableMessaging protocol. This protocol is yet another misnamed WS-* protocol, as it actually only deals with the reliability of the transfer and says nothing about durability, delivery acknowledgments, TTL for a message, long running sessions where a particular message is lost forever, etc.

At some point in the three years between March 2003 and February 2006, WS-RM went from being the enabler of long running services to "yet another misnamed WS-* protocol". And with it, WCF lost (never had?) the ability to support long running services (as I've written previously).

Now all and all, this isn't a big deal. I agree with Shy that WS-RM is under specified as mechanism for durable messaging (Shy calls this "queued messaging"). Attempting to build durable messaging on top of WS-RM sounds like it would have been both difficult and unlike to broadly interoperate. So implementing WS-RM for TCP style reliability and leveraging MSMQ as a transport for people that need durable messaging sounds like a pretty good compromise, especially for a v1 product. Of course, it is not exactly unheard of for a project's end result not to completely live up to the original vision. But I have a specific requirements in this case, so I wanted to know more.

By calling it "misnamed", it sounds like WS-RM was never really intended to be used for durable messaging. However, the July 2003 Reliable Messaging Feedback Workshop indicates that it was. In particular, Rodney Limprecht's "Reliable Messaging Scenarios" deck describes WS-RM as supporting scenarios requiring "either volatile or durable endpoint state". His list of scenarios included both an "Intermittent Connectivity" scenario where "messages pending transfer are staged to disk and exchanged when connected" as well as a "Message Queue Integration" scenario that used WS-RM to interop between JMS and MSMQ. Seems safe to say that WS-RM was originally intended to support durable messaging. So what happened? How did it become "misnamed"?

Rodney's deck describes WS-RM as having the "flexibility to meet scenario requirements". But flexibility comes at a cost. For example, the flexibility of WCF's configuration comes at the cost of significant complexity. In the case of WS-RM, it appears that by trying to make it flexible enough to support both volatile and durable reliability, the authors might have made it too flexible. WS-RM implementers have broad latitude in building the capabilities Shy mentions (durability, acknowledgements, TTL, etc) as well as describing said capabilities in policy. By providing that latitude, we lost the ability to broadly interop durable messaging, which I would suspect is why it ended up out of scope for WCF v1.

As I said before, lack of support for WS-RM based durable messaging isn't that big a deal. As long as you understand WCF's sweet spot - the current version's sweet spot anyway - and don't try and make it be something it's not, you should be fine. Furthermore, Shy mentions the need for an "interoperable Queued Messaging specification" and wrote that it's something he "expect[s] that we will get to it in the near future". Here's hoping that spec is less flexible than WS-ReliableMessaging.

How I Learned to Stop Worrying and Love WCF

Regular readers of DevHawk are likely aware of my obsession interest in SQL Service Broker (aka SSB). I've also been doing a lot of WCF work lately. While there are parts of WCF that I think rock, overall I've found WCF lacking due to it's lack of support for long running services, which SSB excels at.

So it was with great interest that I read this recent article on Integrating WF and WCF. WF is expressly designed for long running systems, so I wanted to see how the article dealt with the WCF's lack of support for such scenarios. Unfortunately, the article basically sidesteps the issue. While it has lots of great info about hosting WF inside a WCF service, the article uses duplex channels for communication between the service and its clients. As I have pointed out before, this approach is impractical because it requires that both the service and its consumer remain alive in memory until the WF end.

Remember this quote from Essential WF?

"It is wishful thinking to assume that the operating system process (or CLR application domain) in which the program begins execution will survive for the required duration."

So basically this WCF/WF sample is wishful thinking. Fine for a demo, but given the severe lack of information out there on integrating these two technologies, I'm worried that many people will read this article as best practice guidance, which in my opinion would be a mistake.

But instead of firing up my blog (that is, like last time) to write a scathing post about how broken this sample is, I emailed Paul which led to a concall with Shy to discuss WCF's lack of support for long running services. Imagine my surprise when Shy agreed with me completely, furthermore saying that support for long running services had been "out of scope" for v1 of WCF. I thought that the whole point of duplex channels was for long running services. But apparently I was wrong.

Shy said to think of the duplex channel in terms of sockets, rather than long running conversations. And just like that, WCF made a ton more sense to me. I had been directly comparing the SSB and WCF communication models, but that's apples and oranges. It would be like comparing SSB to TCP.

If you think about it, vanilla HTTP works a lot more like UDP, even though it's layered on top of TCP. Both UDP and HTTP support connectionless operations and neither UDP nor HTTP are reliable or provide message ordering. The comparison isn't perfect: for example, UDP isn't limited to a single response for an incoming request. But by and large, HTTP is a very UDP style protocol.

If HTTP is basically UDP, then WS-* is trying to be TCP. Frankly, I never understood the point of WS-ReliableMessaging. I always thought reliability == durability == SSB or MSMQ. But when you realize that HTTP lacks TCP-like reliability and ordering capabilities, suddenly this WS spec makes sense. In fact, Shy made this exact point almost a year ago. At the time, I didn't get it because I didn't understand the duplex channel as sockets analogy. Now, I see the value of adding these capabilities to HTTP.

What Shy said was clear and to the point but unfortunately completely missing in the official WCF documentation. For example, the docs on Duplex Services say this:

A duplex service contract is a message exchange pattern in which both endpoints can send messages to the other independently. A duplex service, therefore, can send messages back to the client endpoint, providing event-like behavior. Duplex communication occurs when a client connects to a service and provides the service with a channel on which the service can send messages back to the client.

The docs make no mention that the "event-like behavior" of duplex services only works within a session. And I'm not the only one who mistakenly believed that duplex services could be used for long running services (here's an article in DDJ that makes the same mistake). Shy used the term "episodic" to describe services that span session boundaries. I'd like to see the docs updated to include that concept.

Taking the TCP/UDP analogy even further, I think it demonstrates how pointless the REST vs. SOAP debate is. As UDP is a thin layer on top of IP, REST is a thin layer on top of HTTP. But nobody argues much about UDP vs. TCP these days. I was in grade school when UDP and TCP were standardized, so maybe there were big TCP vs UDP flame wars at the time. But twenty five years later, it's pretty clear that TCP vs UDP is not an either-or proposition. Some protocols are better built on UDP while others are better built on TCP. I'm guessing we'll see a similar evolution with SOAP and REST.

Personally, I would expect that message exchanges between services will become more complex over time. Complex message exchanges would seem to favor stateful SOAP over stateless REST for the same reason complex network protocols favor connection-oriented TCP over connectionless UDP. But SOAP could never displace REST any more than TCP could ever displace UDP. Furthermore, as Larry O'Brien recently wrote "the onus is on the WS-* advocates to prove the need". TCP standardization only lagged a year behind UDP standardization where WS-* has lagged at least six years behind REST. I wonder if UDP would be more prevalent today if it had gotten a six year head start on TCP.

Finally, this "SOAP as Sockets" flash of understanding has also helped me understand how SSB / WCF can evolve together in the future. Some folks have suggested an SSB transport for WCF and I've personally looked into such an approach. But given since SSB is at a higher level of abstraction than WCF, it makes much more sense to layer SSB on top of WCF instead of the other way around. Today, SSB uses two protocol layers: the top level Dialog Protocol, which is built on top of the lower-level Adjacent Broker Protocol (ABP), which in turn is built on TCP. I'd like to see a version of ABP that was built on top of WCF instead of directly on top of TCP. SSB's Dialog Protocol would tie together the WCF duplex sessions into a long-running conversation the same way that it ties together TCP sessions today.

Eventually, I would love to see something that has the programming semantics of SSB and the interoperability of WCF. That would be like the the Reese's Peanut Butter Cup of service messaging.

Morning Coffee 22

• I figured out something to build with PowerShell. Not ready to spill all the beans yet, but I've started by building a provider for SQL Server. SQL already has the SQL Management Objects (SMO) library, so I can really focus on how PowerShell works rather than getting too bogged down in the logic of the provider. I'm sure someone else is working on something similar, but my primary goal with building this provider is to understand PowerShell. The codebase itself is a distant second in priority.
• On the subject of missing LINQ's, Alex James wants LINQ to Web. A good potential use for the Windows Live Search API.
• I posted three recent blog posts to DotNetKicks yesterday, but I've only collected one additional kick so far (for my Compiling Workflows post). Looking at what does gather kicks, I think I would need to write a more dev focused article if I wanted to make the DNK home page.
• Don Smith left a comment on my WSSF post where he talked about the developer and the architect perspective. He seemed surprised that I took the developer perspective. To quote David White: "Architect Must Implement". The customization opportunity in guidance automation is huge, but the value is to the developer first. And while it's a good start, it doesn't go very far at automating the development experience. At heart, I guess I'm a developer even though my business card sez Architect (with a capital "A" these days no less).
• I moved my laptop over to Vista yesterday. My workstation is still on XP, but that's because I haven't had time to completely rebuild my dev environment. No Aero support with the
• I elbowed my way into the TS Gateway pilot at Microsoft and I'm loving it. TS Gateway is a new feature of Longhorn Server that "enables authorized remote users to connect to terminal servers and remote desktops on the corporate network from any Internet-connected device running Remote Desktop Connection 6.0." So I can quickly and easily remote into my desktop from anywhere without establishing a VPC session. The only annoyance is my USB smart card reader, which is fine in a pinch but a pain to constantly have dangling off my laptop. However, my boss approved a PCMCIA smart card reader so soon that won't be an issue.

Hawkeye on Web Service Software Factory

As I wrote the other day, I've been investigating the Web Service Software Factory. The WSSF includes four guidance packages: Data Access, ASMX, WCF and WCF Security. The Data Access isn't service specific (I'm guessing it's also included in p&p's other factories) so I didn't focus on it much. I also didn't investigate the ASMX package at all.

For the WCF and WCF Security packages, I walked through the tutorial in the documentation. The tutorial contains the usual WCF suspects such as creating a data contract, creating a service contract and implementing a service contract. While you can do these things manually, the WSSF provides wizards for most if not all these operations. This seems like overkill for some of these operations. For example, filling out the grid of data contract members was more cumbersome (for me) then just typing the values in the code file.

Also on the subject of somewhat more complex than necessary, WSSF provides wizards for building type translators. This is pretty standard stuff: given an instance of a given type, the translator returns an instance of a different type. Again, I find it faster to write the code for this directly than to individually select the matching fields in the wizard UI. Somewhere on the complexity scale between CRUD stored procedures and service data contracts is the tipping point where it's faster and easier to just write the code than it is to manipulate the wizard UI which generates the code.

On the plus side, the WSSF includes snippets which are very convenient to use. For example, WSSF includes the WCFDataMember snippet (short name: wcfDM). It's a lot like the standard prop snippet, but with the automatic addition of the DataMember attribute.

I had much more success with the main WCF package than I did with the WCF Security package. I wasn't interested in the anonymous or direct authentication mechanisms, which left only two security recipes that I cared about: kerberos and x.509 certificates. Unfortunately, I couldn't make either of these tutorials work. For kerberos, the baseline configuration works (i.e. standard wsHttpBinding with no additional configuration), but after running the "Secure a Service Using Kerberos" recipe, I get an exception that "The token provider cannot get tokens for target". It wasn't in me to debug the sample to figure out what that meant. For x.509 certificates, I can't even complete the wizard - I click OK in the certificate selection dialog, but the wizard doesn't get updated and won't let me continue.

I really dig the new Guidance Navigator window, especially the history window, but I do have one problem with it. Is there a way to mark a recipe so it doesn't show up in the history view? The WSSF includes some recipes like "View Service in Browser" and "Run Client" that don't change the project state and quickly clutter up the history view. It would be great if they didn't show up there in the first place.

I'm guessing the WSSF is designed primarily new WCF developers, so I'm not exactly in the target demographic. Many of my issues above stem from my deeper than average experience with it (though nowhere near the depth of experience some members of the community have already). But it's a good start and I'm sure it will get better with successive releases.

Mastering WCF

Sam Gentile writes:

Harry finds Indigo daunting. Me, I find mastering 8 different stacks (COM+/ES, ASMX, WSE, Remoting, MSMQ, etc) to do the same thing and all the strange nuances a hell of a lot more daunting but that's just me, although the number of timeout settings and config settings is astronomical.

While mastering many different stacks is daunting, the reality is you don't have to master all of them to use one of them. Knowing Sam, he probably has mastered all the different stacks, but MVP's like Sam are an edge case. Most developer don't master any of the stacks, they get comfortable with the one or two stacks they use all the time.

From that perspective, WCF replaces the "legacy" stack a given developer understands with something much more complex, since WCF replaces legacy stacks other than the one said developer is familiar with. Options like message exchange patterns and network protocol were typically implicit to a given technology stack. For example, if you used ASMX, you could use any network protocol you want, as long as it's HTTP. Using WCF, you get to / have to choose which network protocol you want to use. Multiply that decision making process by the "astronomical" number of choices WCF provides, and you've got to spend a long time making decisions that the "legacy" stacks handled for you automatically.

To me, it looks like WCF's primary design goals were to support web service standards (aka WS-*) as well as to unify the disparate communications stacks. And I think WCF was fairly successful at these two goals. Previously, the capabilities you needed would drive your communication stack choice. Need web service interop? Use ASMX. Need low-level control over the message pipeline? Use .NET Remoting. Need to flow transactions? Use COM+. Need to flow transactions over web services with low-level control over the message pipeline? Until WCF came along, you were SOL.

However, if "ease of development" was a goal for WCF, it doesn't look like it was high on the list. And frankly, that's fine. As I've written before, I'd rather have a flexible if complicated low-level foundation to build higher-abstracted application infrastrucutre on top of.

Just don't try and sell me that WCF is making my life easier, because it's not. Not yet, anyway.

(Late) Morning Coffee 9

Took part of the morning off this morning to let the sun shine down on the icy roads. No major incidents getting to work, though the office parking lot is like an ice rink.

• As mentioned yesterday, I finally got my STS implementation working with WCS. Turns out that using WCS with the wsFederatedHttp binding requires you to specify which claims you want to send to the service. In comparison, using WCS with wsHttpBinding automatically requests the PPID claim. It would be nice if this was documented somewhere. I only figured it out by finding this demo from Michele.
• Last week, I said that we need a better tool than SvcConfigEditor. This tool is only marginally better than hand-editing the config files with intellisense. A "real" tool would keep you from building invalid config files. While I appreciate the need for this level of flexibility at the transport layer, we really need a better web service hosting story than IIS + ASP.NET + web.config. WCF makes me long for the days of the MTS/COM+ GUI interface. I never wasted hours troubleshooting config issues with MTS/COM+.
• Apparently, Xbox 360 outsold Wii and PS3 combined in December. That's probably more of a statement about PS3 and Wii shortages, but there's no arguing with numbers like 10.4 million Xbox 360 consoles, 5 millions Xbox Live users, and nearly 3 million copies of Gears of War. Congrats to the Xbox team!
• David may be hiding from his blog of late, but he did venture out long enough to point me to SOA Facts. My favorite: Dante has a special level in hell for consultants whose resumes do not say SOA.

Morning Coffee 8

The news got the amount of snow right, but the day wrong. Instead of hitting yesterday morning, the storm hit yesterday at rush hour. My boss declaired today "1st Annual Architect work from home day" even though we've already had several weather induced work from home days this winter.

• Growing up in Northern VA, when we got snow it was fairly consistent. If there was about four inches at my house, everyone had about four inches. Here, it seems like there's much more variance. My teammate Buzz who lives only 15 minutes from me (when it's not snowing) said he had 10" of snow while I have about half that.
• Speaking of Northern VA, the last few winters have been easy on us but hard on my parents who still live in McLean. This year seems to be the opposite. The forcast for McLean today is only 45, but it's supposed to get up to 65 by the weekend.
• As it turns out, my parents are in the Bahamas right now anyway, so while I make a snowman with my kids today, they're probably on the beach!
• I almost didn't make it home yesterday as I was trying to get my STS working with CardSpace. I have WCS workng in a direct client to service scenario, but not federated with an STS. I probably would have stayed there all night saying "just one more config tweak, and I'm sure it will work" if I had gotten snowed in.
• Speaking of WCS, check out Kevin's screencast on extending ASP.NET's built in SQL membership provider to support WCS. And Garrett published a WCS security token processor for .NET 1.1 and 2.0 a couple of months ago. So you can use WCS on your website, even if you don't have .NET 3.0 on your server. Pretty cool.
• My old teammate John doesn't like the JBOWS acronym. I agree with John that defining a "proper" SOA is waste of time best left to SOAholes. But web services != SOA. Making a distinction between having an architecture where the business and IT levels that rely on independent capabilities and services versus using web services as the protocol between tiers of a distributed application and hoping that you'll be able to integrate in the future makes sense to me.

Morning Coffee 4

Recurring 8am Friday meetings are not my style.

• I've been tracking WCF for a long time. Yet, it's still a dauntingly large topic. Yesterday I spoke to a friend who works in Windows Live who is just beginning to learn about WCF and it's literally overwhelming him. His reaction reminded me of my early reactions to COM. It feels like no matter how much you learn about WCF, the "pile" of stuff still to learn doesn't shrink. In contrast, while my initial exposure to the .NET Framework was overwhelming, eventually I got to the point where I felt like I had a good handle on what was in there. 
• With all the configuration settings in WCF, the number of valid combinations is astronomical. While WCF's configuration based approach is arguably more flexible than a code based approach, it's also more complex and harder to debug in my experience. Config debugging seems to be an endless cycle of tweaking the config file and running the app to see what the effect is. We need better tools than SvcConfigEditor.
• Responding to yesterday's Morning Coffee, Jon Flanders confirmed via email that the built-in SQL WF persistence service "doesn't recover from faults to the last good persistence point" and that "when a fault happens, the instance closes and...is removed from the persistence database." Is this the right behavior? I'm thinking it depends on the workflow. More on this later.
• I'm trying to get my teammates to start using the Beyond Bullet Points approach to the various presentations we build as a team. Interestingly enough, it's been easier to get my team to adopt an agile development methodology than to adopt the Beyond Bullet Point presentation methodology. I would have expected the opposite.
• The Caps trounced the Canadiens last night, ending a five game losing streak. I'm not that worried about the losing streak - the Caps have had several players out with the flu. But beating Montreal, who came into the game twelve games above .500, so badly is a good sign. And how about Nycholat? Two goals and four assists in seven games since he was called up from Hershey. Even more impressive, he's +2 even though the Caps were 2-5 and outscored 24-18 in those seven games while averaging nearly 21 minutes of ice time per game. Here's hoping Nycholat stays hot.

More WCS Docs Please

I love me some CardSpace, but I sure wish there was better documentation on how to use it, especially from WCF. The current documentation is very conceptual so you really only have the samples to go on, which sucks not to put too fine a point on it.

One example shows how to use WCS with WCF's wsHttpBinding. However, it turns out you can also use WCS with WCF's wsFederationHttpBinding. I've been experimenting with that binding talking to an STS, and it's never invoked the WCS UI before. Why does this example invoke WCS? I'm guessing because it doesn't specify an issuer in the config file, but I'd need to see documentation to be sure.

Also, using the federation binding appears to be the only way to request/demand additional claims beyond the private personal identifier (aka the PPID). If you want the client's email address, name, address, etc, you need to specify that via the claimTypeRequirements of the binding's message security element. But that configuration isn't valid for the wsHttp binding. Why?

Finally, the two bindings produce different results on the security token. Using wsHttp, you get three claims: RSA Identity, RDA Possess Property and PPID Possess Property. When using wsFederationHttp, you get Hash Possess Property and PPID Possess Property (plus the claims you request). What happened to the RSA claims? If you attempt to add RSA to the claimTypeRequirements, CardSpace throws an error as an invalid request. Again, why? Keith Brown recently wrote about how to use the RSA claim, so it sounds like a valuable piece of information to have. How come the federation binding doesn't send it?

The Two Types of Service Architects

Tomas Restrepo comments on my recent SSB and WCF posts:

Harry Pierson asks how well WCF supports long running tasks. He suggests that WCF does not support them very well, and says that's one reason he likes SQL Server Service Broker so much. I'd say SSSB is a good match only as long as the long running tasks you're going to be executing are purely database driven and can be executed completely within the database. Sure, this is an "expanded universe" with the CLR support in SQL Server 2005, but even so it makes me nervous at times .

You could also consider using a custom service with MSMQ or something like BizTalk Server for this if you had long running processes that were not completely tied to the DB (or a single DB for that matter).

Sam Gentile follows up:

In that same post, but I needed to call it out separate, Tomas rightfully says, "I'd say SSSB is a good match only as long as the long running tasks you're going to be executing are purely database driven and can be executed completely within the database," in response to Harry liking Service Broker so much. Talk about a narrow edge case. That's way I never really got excited or cared about Service Broker. Its a narrow solution to a special edge case when everything is database driven and can be executed totally inside the database. That's the old Microsoft Data-Driven Architecture for sure. Me, I'd rather have a rich Domain-Driven architecture most of the time. Then if you have Oracle databases in your architecture too, where does it leave you? Nowhere.

As you might expect, I have a few comments,  clarifications and corrections.

First, Tomas' statement that Service Broker only supports service logic "executed completely within the database" in flat out wrong. Service Broker can be used from any environment that can connect to SQL Server and execute DML statements. If you can call SELECT/INSERT/UPDATE/DELETE, then you can also call BEGIN DIALOG/SEND/RECEIVE/END CONVERSATION. This includes Windows apps and services, web apps and services, console apps and even Java apps. Of course, you can also access Service Broker from stored procedures if you wish, but you're not limited to them as Tomas suggested.

Tomas' misconception may come from a feature of Service Broker called Activation. Activation is a feature of Service Broker that dynamically scales message processing to match demand. For example, Service Broker can be configured to launch a new instance of a specified stored procedure if messaging processing isn't keeping up with incoming message traffic on a given queue. This is called internal activation and because it uses stored procedures it does execute within the database as Thomas said. Service Broker also supports external activation where it notifies an external application when activation is needed. You do have to build an application to host your service logic and handle these notifications, but that application doesn't execute within the database. So while you could argue that it's easier to execute your service logic within the database (no need to build a separate host app), it's not required.

Given that you don't have host your service logic in the database, then you're also not limited to "a single DB" as Tomas suggests. You don't, in fact, have to put your Service Broker queues in the same database with your business data. So if you have Oracle in your environment, like the scenario Sam mentioned, you would host your service logic in an external application that processed messages from a queue in a SQL 2005 database while accessing and modifying business data from tables in the Oracle database. Using multiple databases does require using distributed instead of local transactions, but if you're using MSMQ as Tomas recommended, you're already stuck with the DTC anyway.

Finally, I didn't get Tomas' "purely database driven" or Sam's "everything is database driven" comments at all. While there are exceptions, the vast majority of systems I've ever seen/built/designed have essentially been one or more stateless tiers sitting in front of a stateful database. If it's a traditional three tier web app, there's a stateless presentation tier, a stateless business logic tier and a stateless data access logic tier. For a web service, there's no presentation tier, but there's is the stateless SOAP processing tier typically provided by the web service stack. Does this mean the vast majority of web apps and services are  "purely database driven" too? If so, then I guess it's a good thing, right?

In the end, maybe there are two types of service architects - those that believe the majority of services will be atomic and those that believe the majority of services will be long running. For atomic services, Service Broker is overkill. But if it turns out that most services are long running, WCF's lack of support is going to be a pretty big roadblock.

I'm obviously in the long running camp. I'm not sure, but I get the feeling this is the less popular camp, at least for now. We'll have to wait to see, but I do know is that whenever someone brings me what they think is an atomic business scenario, it doesn't take much digging to reveal that the atomic scenario is actually a single step of a long running business scenario that also needs to be automated.

Here's a question for Tomas, Sam and the rest of you: Which group do you self select into? Are most services going to be atomic or long running in the (pardon the pun) long run?

Is WCF "Straightforward" for Long Running Tasks?

My father sent me a link to this article on SOA scalability. He thought it was pretty good until he got to this paragraph:

Long-running tasks become more complex. You cannot assume that your client can maintain a consistent connection to your web service throughout the life of a task that takes 15 minutes, much less one hour or two days. In this case, you need to implement a solution that follows a full-duplex pattern (where your client is also a service and gets notified when the task is completed) or a polling scheme (where your client checks back later to get the results). Both of these solutions require stateful services. This full-duplex pattern becomes straightforward to implement using the Windows Communications Foundation (Indigo) included with .NET 3.0.

When I first saw duplex channels in WCF, I figured you can use them for long running tasks also. Turns out that of the nine standard WCF bindings, only four support duplex contracts. Of those four, one is designed for peer-to-peer scenarios and one uses named pipes so it doesn't work across the network, so they're obviously not usable in the article's scenario. NetTcp can only provide duplex contracts within the scope of a consistent connection, which the author has already ruled out as a solution. That leaves wsDualHttp, which is implemented much as the author describes, where both client and the service are listening on the network for messages. There's even a standard binding element - Composite Duplex - which ties two one-way messaging channels into a duplex channel.

Alas, the wsDualHttp solution has a few flaws that render it - in my opinion at least - unusable for exactly these sorts of long running scenarios. On the client side, while you can specify the ClientBaseAddress, you can't specify the entire ListenUri. Instead, wsDualHttp generates a random guid and tacks it on the end of your ClientBaseAddress, effectively creating a random url every time you run the client app. So if you shut down and restart your client app, you're now listening on a different url than the one the service is going to send messages to and the connection is broken. Oops.

The issues don't end there. On the service side of a duplex contract, you get an object you can use to call back to the client via OperationContext.Current.GetCallbackChannel. This works fine, as long as you don't have to shut down your service. There's no way to persist the callback channel information to disk and later recreate it. So if you shut down and restart your service, there's no way to reconnect with the client, even if they haven't changed the url they're listening on. Oops.

So in other words, WCF can do long running services using the wsDualHttp binding, as long as you don't restart the client or service during the conversation. Because that would never ever happen, right?

This is part of the reason why I'm sold on Service Broker. From where I sit, it looks like WCF can't handle long running operations at all - at least, not with any of the built in transports and bindings. You may be able to build something custom that would work for long running services, I'm not a deep enough expert on WCF to know. From reading what Nicholas Allen has to say about CompositeDuplex, I'm fairly sure you could work around the client url issue if you built a custom binding element to set the ListenUriBaseAddress. But I have no idea how to deal with the service callback channel issue. It doesn't appear that the necessary plumbing is there at all to persist and rehydrate the callback channel. If you can't do that, I don't see how you can reliably support long running services.

Custom Authentication with WCF is Top Shelf

I've spent the last three days heads down in WCF security and color me massively impressed. I just checked in a prototype that provides customized authentication for a business service. The idea that you could bang up a custom authentication service fairly easily blows my mind.

The cornerstone to this support in WCF is the standard WSFederationHttpBinding. While the binding name implies support for WS-Federation which in turn implies the use of infrastructure like Active Directory Federation Services, the binding also scales down to support simple federation scenarios with a single Security Token Service (aka STS) as defined by WS-Trust. WS-Trust appears similar to Kerberos. If you want to access a service using the federation binding, you first obtain a security token from the associated STS. Tokens contain SAML assertions, which can be standard - such as Name and Windows SID - or entirely custom, which opens up very interesting and flexible security scenarios.

If you want to support multiple authentication systems (windows, certificates, CardSpace, Passport Windows Live ID, etc), STS is perfect because you can centralize the multiple authentication schemes at the STS, which then hands out a standard token the business service understands. Adding a new auth scheme can happen centrally at the STS rather than in each and every service. Support for multiple authentication schemes was the focus of our current prototype and it worked extremely well.

WCF includes a federation sample which is where you should start if you're interested in this stuff. That scenario includes a chain of two STS's. Accessing the secure bookstore service requires authenticating against the bookstore STS which in turn requires authenticating against a generic "HomeRealm" STS. Since there are two STS's, they factored the common STS code into a shared assembly. You can use that common code to build an STS of your own.

For our prototype, we made only minor changes to the common STS code from the sample. In fact, the only significant change we made was to support programmatic selection of the proof key encryption token. In the sample, both the issuer token and the proof key encryption token are hard coded (passed into the base class constructor). The issuer token is used to sign the custom security token so the target service knows it came from the STS. The encryption token is used to - you guessed it - encrypt the token so it can only be used by the target service. Hard-coding the encryption token means you can only use your STS with a single target service. We changed that so the encryption token can be chosen based on the incoming service token request.

Of course, it wasn't all puppy dogs and ice cream. While I like the config system of WCF, anyone who calls it "easy" is full of it. I've spend most of the last three days looking at config files. Funny thing about config files is that they're hard to debug. So most of my effort over the last few days has been in a cycle of run app / app throws exception / tweak config / repeat. Ugh.

Also, while the federation sample is comprehensive, I wonder why this functionality isn't in the based WCF platform. For example, the sample includes implementations of RequestSecurityToken and RequestSecurityTokenResponse, the input and output messages of the STS. But I realized that WCF has to have its own implementations of RST and RSTR as well, since it has to send the RST to the STS and process the RSTR it gets in response. A little spelunking revealed the presence of an official WCF implementation of RST and RSTR, both marked internal. I normally fall on the pragmatic side of the internal/public debate, but this one makes little sense to me.

Otherwise, the prototype went smooth as silk and my project teammates were very impressed at how quickly this came together. Several of the project teams we're working with have identified multiple authentication as the "killer" capability they're looking to us to provide, so it's good to know we're making progress in the right direction.

Hawkeye on Standard WCF Bindings

I'm in WF/WCF training this week, so any daytime blogging will be on breaks and at lunch. So far, the instructor is pretty good, though we've only covered "intro to WCF" so far. Given the amount of content he's laid out, I wonder how were going to get through it all.

The instructor said something interesting as he was going over the bindings that come "out of the box" with WCF. He commented that these bindings were the ones the WCF developers thought would be used most often. Of course, he doesn't speak for the WCF team, but it does make some kind of sense. You can extend WCF to support any potential binding, but it makes sense the WCF team would want to enable the common cases without having to write much code.

So take a look at the list of nine standard bindings. Given that WCF is about unifying the windows stack for distributed computing, most of the bindings are at least conceptually similar or in some cases leverage previous distributed paradigms and technologies. You have two HTTP based bindings (with and without WS-* extensions) which are analogous to ASMX and WSE. There's a TCP binding which is comparable to .NET remoting. And there are two MSMQ bindings (with and without SOAP support) for those needing to interop with existing MSMQ systems or who need durable messaging.

That leaves four "new" standard bindings. These are interesting as they don't herald back to previous technologies and paradigms of distributed computing (at least on the Windows platform) but still the WCF team thought enough of the scenarios they enable to include them in the box with WCF. For example, I the wsFederationHttpBinding is designed to take advantage of the significant investment they've made in federated identity. Several years ago, Don Box talked about shrinking the service metaphor rather than stretching the object metaphor across the network. NetNamedPipesBinding is an obvious implementation of that vision. And wsDualHttpBinding is a way to take advantage of the WCF's duplex channel shape while still using HTTP as your transport.

Finally, there's netPeerTCPBinding. From where I sit, this is a radical addition to an otherwise standard set of bindings. Now don't get me wrong, I'm glad it's there. But I'm guessing developers who look at it are more likely to think along the lines of "Wow, what can I do with this?" rather than "Yes, I expected that to be there." Certainly, that was my thought process.

Anyone got any cool uses for netPeerTCPBinding?

WCF Karma

Last fall, I was presenting to a group of architects about SOA. The previous speaker - Rich Turner - was running way late. As I walked in, he was doing a WCF demo and wanted to show how easy it was to change transport by changing the config file. He wanted to change it to run over named pipes, but he couldn't remember the name of the binding. He asked me, and I confessed that I didn't know either. So he gave up on demoing named pipes, finished his presentation and went on his way.

After he left, I confessed to the assembled architects that I knew *nothing* about WCF beyond the high-level concepts. I hadn't spent any time working with it at all. In fact, the only reason I had it installed was because it got installed automatically when you installed WPF which I was working with at the time. My reasoning, as I explained to them, was that WCF is a low-level abstraction. That is to say, WCF is nearer the bottom of the .NET Abstraction Pile than the top. I figured I'd let the people building the next generation of service-oriented infrastructure to worry about WCF.

Fast forward eight months, and my new job is about building service-oriented infrastructure. You know, the type that builds on WCF. Maybe it's karma, but I'm having to learn a lot about WCF right quick.

So as I get back into the blogging saddle, expect to see a bunch of stuff about WCF.

BTW, Major thanks to Sam Gentile, who's taken the time on email and the phone (on his vacation no less) to help talk some things thru. He suggested the WCF Hands On book, which is pretty good.