Morning Coffee 173

I'm on my way out the door for New Zealand and Australia, but I wanted to push out a few things.

• F# August September CTP is out! Don Syme has the announcement, Jomo Fisher has the link roundup and details are on the brand-spanking-new MSDN F# Dev Center. Major congrats to the F# team. I've been running a pre-release version of these bits, and they are a huge step forward if you're an F# developer.
• I've got an article on IronPython in the latest issue of CoDe magazine. Also check out Brad Wilson's IronRuby article, Ted Neward's F# article and Neil Ford's Polygot Programming article.
• Via Michael Foord I discovered that IronPython tester Dave Fugate is back on the blog. He starts with a couple of posts about measuring IronPython performance.
• Speaking of blogging teammates, I think the dynamic languages team has the highest percentage of bloggers in any group at MSFT. All four Program Managers (Dave (lead), John, Jimmy and me), four of five developers (Shri (lead), Dino, Curt and Oleg) and all three Testers (Jim, Dave and Srivatsn). The only non blogger right now is Tomas - who at least has a home page - and  the lead tester which is an open position right now. 11 bloggers out of 12 team members equals 91.67% team blogger coverage.
• I was really impressed with Newspeak when I saw it at Lang.NET so I'm very excited to see they have a new website. No public bits yet, but I like the part where they point out Newspeak "can be implemented independently of Squeak, Smalltalk or any particular VM or IDE". How about implementing a version on DLR guys?
• Maurice de Beijer shows off embedding IronPython inside a WF application. Kinda cool, but he's primarily showing off implementing a CLR interface in IPy. How about a WF activity that execute arbitrary IPy code. That would be cool. (via IronPython URLs)
• Ironclad has reached their 0.5 milestone, being able to import numpy from IronPython. BTW, guys - I'm not sure commenting out one line that appears to be unreferenced qualifies as a "monstrous caveat". Congrats guys! (via IronPython URLs)

Morning Coffee 96

• My friend David "LetsKillDave" Weller writes a long post on corporate blogging, responding to comments on the subject from Penny Arcade. Andre "Ozymandias" Vrignaud also responds. David is specifically talking about blogging within the gaming division, but they apply pretty broadly to Microsoft as a whole when it comes to blogging. "I don't want to get fired", "I don't want to do things that needlessly hurt my company" and "We can say things that PR or marketing people can't.  Or won't." all ring true to me.
• Speaking of gaming, there seems to be more that your average cool games coming our for Xbox 360 this summer. I just picked up Forza 2 which rocks with the Racing Wheel. The Darkness looks very cool and I laughed my ass off playing the Overlord demo. Both shipped this week and have gotten good reviews. On their way in August are Bioshock and Blue Dragon. Of course, there are a few other big games coming this holiday. A good, but expensive, year to be a gamer.
• I laughed my ass off reading Larry O'Brien's Top 10 Things To Do With Your Petaflop Supercomputer, esp. #9.
• WSDL 2.0, it's official. Nick Allen has the news. Personally, WSDL seems to be the spec most responsible for driving RPC-style request/response web services, so let's just say that I am not a fan.  
• Joe McKendrick thinks something is "holding back SOA"? I don't think it's any one thing, but certainly the RPC style that most web service toolkits pretty much force down your throat isn't helping.
• Nick Malik thinks Acropolis is promising as a SOA service consumer, but Udi Dalan thinks it doesn't support multi-threading well enough. I lean towards Nick on this one since I see multi-threading as a language problem, which a library like Acropolis can't solve on it's own.
• Jon Flanders has been busy building the BizTalk Server 2006 extensions for Windows Workflow Foundation (June CTP) SDK Sample. I'm not sure why the marketing folks gave this such a long and involved name, but the sample does look pretty cool. Paul Andrews has the project overview and demo video. However, given that the WF workflows are hosted in BTS, is it accurate to say "No Biztalk Experience Required"?
• Speaking of WF, Tomas Restrepo takes a detailed look at the new WF service hosting in .NET FX 3.5. Mostly, he likes what he sees. I have the same problem he does with the message correlation IDs. I'd like to have other options here, including support for what I call "message data correlation" (Tomas describes this as "natural correlating identifiers") and "address correlation" which is basically the REST model.

Morning Coffee 95

• New version of dasBlog is out, the final version on ASP.NET 1.1 (unless this release "kills a kitten" as per Scott Hanselman). I don't have the time (make the time?)to run daily builds, but I do try and upgrade to new major releases in a timely fashion. I'm also moving hosters, so expect a little downtime around here at some point in the near future.
• Matt Winkler is doing a series on alternate WF execution patterns. His first is the N of M pattern. While I can nitpick some things in WF - especially the limitations of transaction flow - WF's support for variability and extensibility of execution patterns is fraking brilliant. (via Sam Gentile)
• Joe McKendrick is all excited about a SOA built without web services! We've been "doing SOA" since the EDI days without web services, so I'm not sure this level of excitement - with an exclamation point and everything - is warranted. But it is good to see people realize web services != SOA. Instead of web services, CERN is using JMS to move messages around. I don't know much about JMS, but I do know it supports async and durable messaging, two things I think are critical for enterprise services.
• I saw on LtU that there's a new paper on Singularity out. For those who don't know, Singularity is a MS Research platform designed for reliability instead of performance. But there's more than just a new paper. According to the project home page, "Singularity Version 1.0 is complete. We've shipped the Singularity Research Development Kit (RDK) to a small number of universities for their research efforts." I wonder if I can get my hands on that RDK?
• Jeff Atwood is starting to show ads on Coding Horror, but he's donating "a significant percentage" of the ad revenue back into the programming community. He's starting with $5,000 and Microsoft is matching for a total of $10,000 to be donated to open source .NET projects. Go tell Jeff which projects you think he should donate to. Castle seems to be an early favorite.
• On Monday, Nick Malik posted what he called the Simple Lifecycle Agility Maturity Model (aka SLAMM) as a way of measuring your "agile factor". Surprisingly, the community response has been zilch. After Nick's comments on Agile last week, I figured someone would have something to say about it, even if only to slam it. (Slam SLAMM, ha ha.) Maybe nobody opened the spreadsheet and saw Mort has an agile factor rating of 71%? Personally, SLAMM seems like a rather coarse tool for measuring how agile you are, but coarse tools are better than no tools at all.

Early Afternoon Coffee 81

These morning meetings are really cramping the "morning" style of these posts. But better late than never, I guess.

• Politics 2.0 Watch: Democrats pwn Republicans online. (via Balloon Juice)
• Roger Wolter writes about integrating SSB with WF, something I've experimented with myself. I didn't find the integration quite as natural as Roger did - transactions are a real PITA, and Roger apparently he hasn't looked into that yet - but I agree 100% with the idea that "most SSB programs end up looking a lot like a workflow." Looking forward to seeing your code, Roger.
• Pat explains his Newton vs. Einstein view of distributed systems and then rants about Consistency, Availability and Partition-tolerance. In particular, he discusses the evolution of what consistency (and to less extent availability) means in the face of loose coupling. Do yourself a favor and give up on distributed transactions now. Also, Pat points to another paper from CIDR dealing with isolation in services. Haven't read it yet, but I've added it to "the pile".  
• David Chappell writes about Service Component Architecture vs. Service Oriented Architecture. Since I don't "do" evangelism anymore, I don't spend much time watching what our competitors are doing. According to the SCA website, SCA is supposedly a "a model for building applications and systems using a Service-Oriented Architecture." But according to David, SCA 1.0 focuses on "portability, not interoperability, and so they don’t fully define the interactions between components necessary to create composites that cross vendor boundaries." I realize that we don't industry agreement on all the details of what SOA means, but I think we all agree that it's cross platform and cross vendor. Or maybe we can't even agree on that much.

Morning Coffee 40

• My boss let me borrow a Tecra M4 that he scavenged from his boss. The display is fairly twitchy, I think it's a motherboard issue. But it's very intermittent and I'll get help desk to take a look. In the meantime, it sure is nice to driving a Tablet PC again. And it's Vista ready to boot.
• Speaking of Vista, Visual Studio 2005 Service Pack 1 Update for Windows Vista. It's a mouthful but it's now available. Soma answers questions about the new release on MS PressPass.
• The DSL tools team keeps on rolling with the power toys. First it was the Designer Integration PowerToy, now it's the DSL Tree Grid Editor PowerToy. Jeff Santos has the details.
• I missed the TechFest keynote yesterday, but it's available on demand. They also have descriptions and videos of some of the technologies on display. (well, only one video so far, but I assume since the page is labeled "TechFest 2007 Videos" that more are on the way.)
• There's new support for integrating WCF and WF coming in VS "Orcas". Moustafa Khalil Ahmed has the details on what's new for WF & WCF in the latest CTP drop. For me personally, the WCF/WF integration is some of the most important stuff coming in Orcas, second only to LINQ.

(Late) Morning Coffee 36

• It snowed again yesterday. Last year we had one snowstorm, the year before that none. We've now had I think five this year plus the massive windstorm that knocked out power for days.
• Technorati told me that "social news aggregator" Megite is linking to me. For some reason, this post of mine on Powershell is considered related to "Is PR Too Stupid for Conversational Marketing?" from Amanda Chapel. Seems like Megite has some bugs to work out.
• Paul Andrew announces BPEL support for WF but David Chappel writes "no one should interpret the announcement as an embrace of BPEL-based development by Microsoft". Personally, I think BPEL is just the latest attempt at "write once, run anywhere" and will meet with the same limited success of previous attempts. The last thing I think MSFT should do is embrace BPEL based development.
• BPEL actually has two flavors, Executable and Abstract. Abstract BPEL is potentially fairly useful. You could use to exchange of the publicly viewable parts of a process with a partner in order to make two processes work together. That's fairly exciting. I would welcome Abstract BPEL support for WF and/or BTS. But as far as I can tell, most of the BPEL focus has been around Executable BPEL, which as I wrote above is attempting to be a platform independent language for implementing business process. That's fairly unexciting since we've been down this road before many times (UNIX, CORBA, J2EE) and it has never worked out.
• Soma announces the launch of the Beginner Developer Learning Center. It includes Kid's Corner with the cutely named C# for Sharp Kids and VB for Very Bright Kids e-books. Very cool, I can't wait to share this with my kids in a few years. Only complaint: where's the XNA love?

Morning Coffee 34

• Old news, but Reflector 5.0 is out. W00t! Not sure when Scott Hansleman became chief Reflector cheerleader, but he's got the rundown on the new features.
• Politics 2.0 Watch: OpenCongress. Sort of like Wikipedia for government. If we can disseminate information on bills and resolutions via the Internet, couldn't we collect votes on them as well?
• I got my hardcopy of Powershell in Action while I was on vacation. Highly recommended.
• Sam Gentle is starting to dig into WF, and he posts about the difficulty getting data in and out of workflows. He's using the ExternalDataService infrastructure which I don't like very much. I recommend getting friendly with the WorkflowQueuingService which is the low-level communication infrastructure that ExternalDataService builds on top of. The WQS docs are severely lacking, but it's fairly straight forward to figure out.
• Speaking of WF, Tomas Restpro reviews Programming WF. Sounds fairly introductory. Personally, Essential WF is one of the best tech books I've read in a long time, so I'll be skipping this book.
• My teammate Dale is continuing his daily posts on his blog.
• Joe McKendrick wonders if EDA is the new SOA. Frankly, both terms are so poorly defined that it's hard to determine exactly what each term means, much less how they're related. If you're an IT industry analyst, you probably can make a ton of cash describing the differences between them. Maybe it's me, but I don't see that much value in SOA without EDA. In fact, I'd go so far as to say service orientation without events isn't much a new architecture paradigm at all. It's just the Same Old Architecture with better support for interop.

Morning Coffee 27

• Is there a good solution to colorize source code that looks good in RSS feeds? I've tried Insert Code and Paste from VS for WL Writer and both look fine in HTML but awful in RSS.
• My friend David Geller launched his latest venture Eyejot recently. Eyejot is a Flash-based video messaging system, so you can send and receive video clips without having to install anything but a webcam. According to the Eyejot blog, they're getting some good press. See an interview with David about Eyejot up on YouTube.
• Here's an interesting article on using WF with Amazon's Mechanical Turk service. Invoking MTurk isn't that interesting - it's just a web service and WF has a built-in InvokeWebService activity. But since MTurk has no way to asynchronously call out to the WF, you have no choice but to regularly poll MTurk to see if the task is complete. Yuck. (via Larkware)
• Yahoo! Pipes looks interesting. At least the screen shots of it on various websites and blogs look cool. Too bad the site is absolutely hammered this morning. (via Dare Obasanjo)
• Like GAT? Like DSL? Then use them together!
• If I can more than raise my Gamerscore by 1,500 points by April 12th (i.e. more than double it), I can get a free $5 game. But why wait to start the contest until next Monday? Doesn't that discourage people from playing until then?

Morning Coffee 20

• Jim Gray has been missing at sea since Sunday. My thoughts are with him. (via Werner Vogels)
• We launched Windows Vista and Office 2007 yesterday. There were parties on campus, but my office is a little off the beaten path, so we didn't get a party.
• WF Activity Validation is very cool, but make sure you put your activities into a separate project from your workflows. It took me a few minutes yesterday to figure this out, but the validators are invoked not only for the activities in workflows, but for the actual activity implementation itself. My validation logic is checking to ensure properties are specified and that given activities are or are not inside a transaction scope. Obviously, the activity implementation is invalid according to these rules. Also, you need to remove the Workflow.Targets import from your activity project file, as that is what invokes the activity validation.
• A quick follow-up to yesterday's compiling workflows post: WF appears to be fairly short on out of the box functionality, but more than makes up for it with an expansive extensibility model. It makes the learning curve longer, but it's well worth it the trip.
• I'm demoing the result of the proof of concept work we've done over the last few months today. It's been a while since I've presented to any kind of audience so we'll see how it goes.

Compiling Workflows

Scott Allen pointed out that if you need to declare top level properties in a XAML only workflow, you can subclass the root activity class, add the properties you want, then specify the custom subclass in the XAML workflow. That doesn't really solve my problem. I'm trying to limit to activity vocabulary that the workflow author has access to. If they can specify their own custom type as the root activity of their workflow, they can also add whatever execution logic they want, which is what I'm trying to avoid.

There is an example in the Advanced Authoring chapter of Essential WF where they describe building a root activity class that generate top level properties as part of the workflow compilation process. The root activity (in the example, it's called "SequenceWithVars") where you can specify the top level parameters in the XAML. Here's an example from the book:

The SequenceWithVars type includes a custom ActivityCodeGenerator that loops thru the VariableDecls collection and adds a top-level property (via CodeDOM) for each VariableDecl instance. You end up with a workflow class that looks like this:

In order to use the XAML workflow with the SequenceWithVars activity, you do need to compile the XAML first using the WorkflowCompiler class. WorkflowCompiler.Compile() returns a compiled type which can then be passed to CreateWorkflow. But adding the separate compile step is a small price to pay, in my opinion. This approach lets me limit the workflows to XAML only while still allowing for top level properties which are needed in many data binding scenarios.

Update: removed syntax highlighting because it looked bad in my news reader.

Later Update: Fixed syntax highlighting.

Morning Coffee 19

• I find Jim Kobielus' "SOA as 50 square miles of fungus" analogy funny and strangely compelling. The "keep in the dark and feed it shit" jokes practically write themselves. (via Joe McKendrick)
• Politics 2.0 Rising: The number of Americans who got "most of their information" about the 2006 midterm elections was double the number from the 2002 elections.
• Do you use external/flash drives? Do you have issues when you try to "Safely Remove Hardware" with said drive? I do, all the time. Apparently, unlocker is the answer. (via Paul Andrew)
• How come there's no information about LogToTraceListeners in the WF documentation? As far as I can tell, it's not in the Windows SDK docs at all and the only reference to it on MSDN is this year-old article and this year-old blog post. I only discovered because someone on the internal WF discussion alias asked about it. I've added In my SSB/WF work, I subclassed the built-in SQL persistence service in order to add tracing support. If you're developing a WF host, you need to turn this on. I find it mind-boggling this isn't included anywhere in the official WF docs.
• Nice to see Soma bragging about Software Factories. As he writes, our current solution - consisting of the Guidance Automation Toolkit and the DSL Tools - are really just a first step. I'm just starting to experiment with the Web Service Software Factory (WSSF). Aaron Skonnard introduces both the ASMX and WCF version in his two most recent Service Station articles.
• Michael.NET makes Programming Promises and Ted Neward swears the Oath of the Conscientious Programmer. Why stop there? How about the Architect's Affidavit to actually implement the shit we draw on the white board? The Technologist's Testimony of understanding and belief in all things geeky and gadget? Come on, isn't this just called "doing your job"? Do we really need to make promises and swear oaths to take it seriously and do it to the best of our abilities?

Binding Across States in WF

I didn't get much done today - recovering from the norovirus and all. But I did figure out an important point about State Machine Workflows.

For my SSB/WF prototype, I decided to implement my business logic as both as a sequential and state machine workflow. While everything worked fine in my sequential prototype, things started failing once I got to my second state. Turns out that I was trying to bind property values across states, which is a no-no. Since states may be entered more than once, they are executed in their own subordinate activity execution context. As per the State Activity docs, "the definition of the child activity in the activity tree (referred to as the template) is never executed and is always in the Intialized state".

I'm fairly sure that when you try to property bind across states, you're connecting to this template activity, rather than the activity that was actually executed. So instead of getting real data (in my case, the handle of a SSB conversation) you get whatever that value was set to in the designer.

Of course, once I figured out about the subordinate AEC, this behavior makes perfect sense. But it could be better documented. So far, the best information on them is in the Advanced Activity Execution chapter of Essential WF.

BTW, to deal with this, you need to promote the values that matter out of the individual activity instance up into the workflow instance itself. In the databind dialog box, there are two tabs: "Bind to an existing member" and "Bind to a new member". I'm not sure why the WF dialog existing member tab is there for state machine workflow since it doesn't do what you might expect it to. Instead, you should create a new member as I've done here. This creates a field or dependency property (whichever you choose) on the parent workflow itself, which is then available to all activities in all states within the workflow.

I wonder how this works with XAML only workflows? There doesn't appear to be any way to declare a field or dependency property in XAML. The templates that ship with the WF SDK aren't XAML-only, they're a combination of XAML and code. When you create a new member, it's always created in code. The idea of XAML only workflows is very appealing - it severely cuts down the surface area that can be meddled with in your host. But you still need to easily share data between activities!

Morning Coffee 4

Recurring 8am Friday meetings are not my style.

• I've been tracking WCF for a long time. Yet, it's still a dauntingly large topic. Yesterday I spoke to a friend who works in Windows Live who is just beginning to learn about WCF and it's literally overwhelming him. His reaction reminded me of my early reactions to COM. It feels like no matter how much you learn about WCF, the "pile" of stuff still to learn doesn't shrink. In contrast, while my initial exposure to the .NET Framework was overwhelming, eventually I got to the point where I felt like I had a good handle on what was in there. 
• With all the configuration settings in WCF, the number of valid combinations is astronomical. While WCF's configuration based approach is arguably more flexible than a code based approach, it's also more complex and harder to debug in my experience. Config debugging seems to be an endless cycle of tweaking the config file and running the app to see what the effect is. We need better tools than SvcConfigEditor.
• Responding to yesterday's Morning Coffee, Jon Flanders confirmed via email that the built-in SQL WF persistence service "doesn't recover from faults to the last good persistence point" and that "when a fault happens, the instance closes and...is removed from the persistence database." Is this the right behavior? I'm thinking it depends on the workflow. More on this later.
• I'm trying to get my teammates to start using the Beyond Bullet Points approach to the various presentations we build as a team. Interestingly enough, it's been easier to get my team to adopt an agile development methodology than to adopt the Beyond Bullet Point presentation methodology. I would have expected the opposite.
• The Caps trounced the Canadiens last night, ending a five game losing streak. I'm not that worried about the losing streak - the Caps have had several players out with the flu. But beating Montreal, who came into the game twelve games above .500, so badly is a good sign. And how about Nycholat? Two goals and four assists in seven games since he was called up from Hershey. Even more impressive, he's +2 even though the Caps were 2-5 and outscored 24-18 in those seven games while averaging nearly 21 minutes of ice time per game. Here's hoping Nycholat stays hot.

Morning Coffee 3

I'm living in a tinder box, hosing down the roof
It's raging all around me, and I still refuse to move
There's a lesson I'm desperate to learn
And I'm willing to burn
     "Willing To Burn" by Maia Sharp

• A warm welcome goes out to the 110th congress. Between the Democratic majority in both houses and Republicans looking to distance themselves from President Decider and his abysmal approval ratings, maybe we'll actually get something accomplished in the next two years.
• Not as nice as USC trouncing Michigan, but I like seeing Notre Dame on the receiving end of a 41-14 beatdown from LSU in the Sugar Bowl. That's the 9th consecutive bowl loss for the Irish.
• Actually started getting some work done yesterday. Today I'm doing some WCF STS work, but yesterday I focused on SSB and WF.
• I need to better understand WF's faulting and compensation model. I got sidetracked yesterday when I realized that when a WF instance faults, the built-in SQL persistence service deletes the persisted instance from the database. That doesn't seem right to me, but I was wrong last time I called out the WF SQL persistence service so I want to do more digging before I open my trap.
• I dig WF persistence. I wrote a few weeks ago about shipping a WF instance to a developer for debugging. Yesterday, I thought about having a persistence service that kept a history of the WF instance rather than overwriting it. I wonder if that would help with production debugging?
• Great quote yesterday by my boss, speaking ill of a project that will remain nameless:
  "Basically, they've spent the last month building an executive presentation to say we're screwed"

Transactions in Workflow Foundation-land

I've been spending some quality time with SSB and WF of late. On the balance, my opinion of both these technologies is very positive, though each has some warts of note. For Service Broker, they got the transactional messaging semantics right, but much of the lower level connection management - what SSB calls "routes" are clumsy to deal with. For Workflow Foundation, the execution model is amazingly flexible. Unfortunately, WF's support for transactions is significantly more rigid.

If you're build a SSB app, you're typical execution thread looks like this:

1. Start a transaction.
2. Receive message(s) from top of the queue.
3. Execute service business logic. Obviously, this varies from service to service but it typically involves reading and writing data in the database as well as sending messages to other services.
4. Commit the transaction

When I sat down to marry SSB and WF, I naively assumed I could simply use WF for step three above. Alas, that turns out to be impossible. This thread on MSDN Forums has most of the gory details, but the short version is that WF does not support flowing host managed transactions into the workflow instance. As per Joel West in the aforementioned thread:

"[T]he WF runtime in V1 only supports flowing in a transaction on WorkflowInstance.Unload. There are various ways that you could try and hack this (with a custom persistence service or WorkflowCommitWorkBatchService) but if you do this it won't work correctly 100% of the time and the times when it fails (error conditions or failures causing the tx to rollback) will be exactly when you are expecting transactional consistency.

Bottom line - the only way to make this work is to call WorkflowInstance.Unload inside your transaction scope.  This was the best that we could do in V1 to try and enable this pattern in some form.  Not always ideal but it can be made to work for most scenarios that require usage of an external transaction."

So the WF compatible execution thread looks like this:

1. Start a transaction
2. Receive message(s) from the top of the queue
3. Load/Create the associated workflow instance for the received messages
   • All messages received are guaranteed to be from the same SSB conversation group, which is roughly analogous to a WF instance, so this turns out to be fairly easy
4. Enqueue the received message in the workflow instance
5. Unload the workflow instance
6. Commit the host transaction
7. Reload the workflow instance
8. Run the workflow instance (note, I'm using the manual scheduling service)
   • Workflow instance creates a transaction if needed
9. Unload the workflow instance (typically done via UnloadOnIdle in the persistence service)
• Assuming the workflow instance needed a transaction, it gets committed after unload

Basically, you use two transactions. One host managed transaction to move the message from SSB to WF instance and one WF managed transaction to process the message.The need for two transaction instead of one is unfortunate, but required given the current design of WF. And frankly, given the importance and difficulty of transaction management, I'm not that surprised that WF has hard coded transaction semantics. Trying to build a generic transaction flow model that would work in the myriad of scenarios WF is targeting would have been extremely difficult. At least there is a work around, even if it means using two transactions and loading and unloading the workflow instance twice.

However, there is a silver lining to the two transaction approach: two unexpected benefits when dealing with poison messages. First, SSB doesn't have dead letter queue like MSMQ does. Moving a poison message to a dead letter queue would break SSB's exactly once and in order semantics.(MSMQ doesn't guarantee in order delivery) But moving all messages into the WF instance gets them out of the main SSB queue so poison messages don't continue to get processed over and over.

Second, because the workflow instance is peristed after the messages are enqueued, there's a representation of the workflow after the message is received but before the message is processed. If there's a poison message, attempting to processing the message will fail and rollback to this state. This persisted workflow instance could be sent to a developer who could step through it to determine the cause of the error. We could even have developer versions of runtime workflow services so we could read remote data and simulate data updates. I wouldn't want the developer updating production data in this way, but it would be great for troubleshooting issues.

Slight Workflow Annoyance

One of the cool things about WF is that you can specify the Guid it uses to identify a workflow instance. WorkflowRuntime.CreateWorkflow has an overload (actually two) where you can specify said workflow instance identifier. This is awesome for using WF with Service Broker, as Service Broker already has the idea of a conversation group which is roughly analogous to a workflow instance. Conversation groups even use a Guid identifier, so there's not even any mapping required to go from conversation group to workflow instance.

However, things get less cool when you call WorkflowRuntime.GetWorkflow. If you call GetWorkflow with a Guid that has no corresponding workflow instance, it throws an InvalidOperationException instead of just returning null. That seems like an odd choice. If you're going to support specifying the instance identifier when you create the workflow instance, doesn't it make sense that you should also gracefully support the scenario where an instance identifier is invalid?

I see two ways to deal with this:

• Iterate through the list of loaded and persisted workflow instances looking for the one in question.
• Call GetWorkflow and swallow the exception.

I ended up picking the "Swallow the Exception" approach as I can't imagine the iteration thru every loaded and persisted instance would be very performant. But swallowing exceptions always makes me feel icky. I'm a fan of the "exceptions only for exceptional situations" approach and as far as I'm concerned, an invalid instance identifier isn't that exceptional. Still, it's a minor annoyance, especially given how cool it is to be able to specify the workflow instance identifier in the first place.

Essential Windows Workflow Foundation

On Don's recommendation, I picked up Essential WF. In the forward, Don writes "[S]omething big is about to happen." I'm only part way thru chapter one, and this is already a must read. Go get it. Now.

In the preface, they define the term "Reactive Program", which I'm adding to my personal lexicon.

"Windows Workflow Foundation (WF) is a general-purpose programming framework for creating reactive programs that act in response to stimulus from external entities. The basic characteristic of reactive programs is that they pause during their execution, for unknown amounts of time, awaiting input."

That "unknown amounts of time" is the kicker. Here's a paragraph from early in chapter one that expands on that:

"Real-world processes take a long time - days, weeks, or even months. It is wishful thinking to assume that the operating system process (or CLR application domain) in which the program begins execution will survive for the required duration."

Gee, that sounds familiar doesn't it?

WF Clarifications and Corrections

Last week, after I posting my WF learnings, I got a call from Paul Andrew, Technical Product Manager for WF. Seems calling the built-in persistence service and the built-in web service support "toys" created some extra work for Paul. He blogged a response and I wanted to follow up on a few things here.

1. The "toy" SQL Persistence Service - My understanding about how the built-in persistence service works was incorrect. As per Paul's blog, "The WF runtime doesn't load all idle instances on startup, that would be crazy." Of course, we're talking about the SQL Persistence Service, not the WF runtime, but it's still crazy. It's so crazy that when I thought that's what the SQL Persistence Service did, I called it a toy! So I'm flat out wrong on this one. Sorry about that Paul (and the rest of the WF team).

2. The "toy" Web Service Integration - Apparently, I was also mistaken about the use of ASP.NET sessions. But I was right about WF's use of ASMX, the use of the tempuri.org namespace, and that web service support is limited to WS-I basic profile request/response style services. So while "toy" is a bit harsh, the web service integration is still pretty light weight. Where's the WCF integration? I understand the need to support ASMX, but no WCF means no support for duplex conversations, either as service provider or consumer, and no support for reliable sessions. That makes WF's web service integration a non-starter in my project. Of course, the good news is that you can build your own WF activities, so I can toss the built-in web  service activites and still get to keep the rest of WF.

3. Is WF itself a "toy"? Paul has a list of reasons why WF isn't a toy, including some silly ones (it wasn't in Toy Story). In case there's any confusion about my opinon of WF, let me be clear: I think WF rocks, full stop. My negative comments about WF were isolated to the two areas listed above and not intended to apply to WF as a whole. The other seven points were all about cool things that I didn't realize WF does.

I'm not just trying to kiss up to Paul here - WF is one of two foundation technologies that my project absolutely depends on. (Any guesses on the other?) With the class out of the way and a better understaning as to what's possible with WF, I will be diving much deeper on WF in the future. Watch this space for more WF related posts.

More Stuff I Didn't Know About WF

4. All communication from the host to the workflows goes thru the WorkflowQueuingService. Unlike other WF services, the queuing service is not replaceable. Communication from a workflow activity instance to the host goes thru some other service. There is no "default" activity -> host communication service, though WF ships with the ExternalDataExchangeService. You have to manually add the ExternalDataExchangeService to your workflow runtime instance. WF also includes the wca.exe utility which generates strongly typed HandleExternalEvent and CallExternalMethod activities for your workflow. And according to Jon, the External Data Exchange system was added specifically for SharePoint.

5. The built-in sequential and state machine workflows are customizable. So if you wanted to have a custom designer or validation experience for an otherwise standard sequential or state machine workflow, you can inherit from the standard workflow type and add the custom validation and / or designer support. For example, as I wrote earlier, you can load a workflow from XAML. XAML workflows can't support the Code Activity. So if you wanted to use the standard sequence designer but disallow the use of the Code Activity, you wouldn't have to rewrite the sequence activity from scratch.

6. You can execute workflows across a farm, similar to how you build a web farm. Workflows that get persisted to the persistence service can be loaded on any node in the farm. Of course, like web servers in a farm, you'd have to have the same bits installed on all the machines in the workflow farm. Sounds like an opportunity for something like Application Center for WF.

7. The built in SQL Persistence Service is a toy. When you start your workflow engine, the SQL Persistence Service will auto-load all the existing persisted workflows into memory, as an "optimization". So if you have thousands of order workflows persisted in your database and you decide to bring another workflow processing node online, that new workflow processing node will happily load ALL of the outstanding orders that it can find in the database. This service should have been included as a sample like the file based persistence service sample rather than described as "a fully functional persistence service" in the docs.

8. WF Web Service integration is also a toy. WF leverages basic ASMX for all it's web service integration, so your only choice for web services - consuming or exposing - is atomic request/response style services. You can have a workflow with multiple operations, but the default ASMX hosting infrastructure squirrels the WF Instance ID in the ASP.NET session, which will time out in twenty minutes (by default). Furthermore, the built in ASMX workflow host doesn't provide an option to specify the web service namespace, so it ends up with the default http://tempuri.org namespace. Again, like the SQL Persistence, this should have been included as a sample, not included in the standard activities.

9. Activities can generate code. In the previous item, I mentioned there's basic ASMX hosting integration for a workflows. Turns out the WebService Activities generate that hosting code as part of the workflow compilation process. So in other words, if you add a WebService activity to your workflow, the activity will inject the custom ASMX hosting code into your compiled workflow. In this WebService activity case, the injected code isn't very good, but the fact you can do this at all is very cool.

Cool Toy From the Teacher

As I wrote this morning, I'm in training this week. The instructor (who I wrote earlier is "pretty good") is Jon Flanders. I didn't recognize his name, but I did recognize his Atlas based WF Designer that he released a month and a half ago or so. It's a cool piece of work so it's doubly cool (for me anyway) that he's teaching this class.

Things I Didn't Realize About WF

1. You can customize how you load workflows. My favorite part of Customizing the Microsoft® .NET Framework Common Language Runtime is custom loading assemblies from a structured storage file. In WF, you can load workflows by type or from XAML. Of course, these the XAML - or even the types - can be loaded from anywhere, or even generated on the fly. If you need more control, you can swap out the default workflow loader service with one of your own creation.

2. You can swap out the default workflow scheduler. How about a scheduler based on the Concurrency and Coordination Runtime (CCR)?

Side note - How about a official home page for CCR? And while I'm asking, how about a download for CCR separate from MS Robotics Studio?

3. You can add any object to the workflow runtime as a service. You're not limited the services the WF runtime knows about. Of course, WF won't use your service, but you can build activities that use it. This is likely to be huge for getting data in and out of the workflow instance.