Morning Coffee 138

• In writers strike news, the WGA has made side deals with Worldwide Pants (aka Dave Letterman's company), United Artists (aka Tom Cruise's company) and The Weinstein Company (previously known as Miramax). The WGA strategy of divide and conquer seems to me making slow progress. Update: The Weinstein Company was founded by Miramax's founders Harvey and Bob Weinstein after they left Miramax. But Miramax is still around. Thanks to GrantC for the correction.
• They're still two games under .500, but the Caps completed a season sweep of the Eastern Conference leading Ottawa Senators last night. They're only 3 games out of the top spot in the (admittedly very weak) Southeast division
• Big tech news today isn't coming from MSFT-land. Sun is buying MySQL and Oracle is (finally) buying BEA. Both deals seem like pretty significant culture clashes, though Sun/MySQL seems like the better fit of the two.
• There's a new draft of Service Modeling Language 1.1 available. If you'll recall, this used to be called the System Definition Model, part of the Dynamic Systems Initiative. Hadn't heard anything from those folks in a while, good to see they're making progress.
• Stephan Tolksdorf dropped me a line to tell me he was able to "vastly simplify" FParsec, and as a result it now runs on the current version of F#. Awesome!
• Speaking of F#, Scott Hanselman has a new F# podcast, this time interviewing Dustin Campbell. Check out all of Dustin's F# posts.
• I didn't know about the "Copy as Path" feature in Vista. Why is it hidden?
• I was a big fan of the WDS deskbar shortcut feature - a feature that is missing in Vista. Enter Start++ by Brandon Paddock, which adds shortcuts to Vista's search box. It also supports "iPhone apps" and scripting. But JScript? Where's the PowerShell love, Brandon?
• EA released the source code to the original SimCity under the GPL. Bil Simser is digging into the code and it looks like he's going to port it to XNA. (via Ozymandias)
• Wes Haggard has published the source code to CodeHTMLer on CodePlex. He took two updates from me: the F# language definition as well as the ability to choose the font when not using PRE tags.

Morning Coffee 118 - ITARC SoCal Edition

I'm not back on blog sabbatical, but between finishing my presentation and attending ITARC SoCal earlier this week - not to mention being sick - I didn't have time to write anything. Normal Morning Coffee resumes tomorrow, here's a summary of my notes from on my two days at ITARC.

• Scott Ambler did the opening keynote on agile enterprise architecture strategy.
• He claims that success is more prevalent in the industry that people think, because the industry has a narrow definition of success. If you change (aka widen) the definition, the success rate goes way up! That's not exactly useful, but he referred to an as-yet-unpublished survey on project success rate that should be up on DDJ "soon". I'd like to see that raw data.
• While I agree with most of his points, Scott's presentation style is very abrasive. For example, he makes the point that there is no one-size-fits-all process, which I couldn't agree with more. But does he say it like that? No, he says "Repeatable processes? What an incredibly stupid idea!" even though the room is full of folks who probably think repeatable process is actually a good idea.
• Scott suggested that unit tests are the best way to specify requirements. I've heard this before from agile practitioners, but something nags at me about it. Certainly, having executable requirements is a huge plus. But how can you be sure they're the right requirements if the stakeholders can't read them?
• This keynote setup what turned out to be a major theme for the conference - traditional vs. non-traditional enterprise architecture. Or as I would characterize it: Industrial vs. Post Industrial architecture.
• Simon Guest presented on user experience in architecture, which is his specialty these days. He lays out a UX model that was very compelling. I'm not sure if there's a whitepaper version of this model (there should be) but you can see the model as he lays it out in powerpoint. I've seen Simon's UX decks, but never actually seen him present it, so that was a treat.
• I skipped Ted Neward's session in order to take in something new. So I went to see Daniel Brookshier of No Magic talk about DoDAF - the Dept. of Defense Architecture Framework. I had met Daniel the night before at dinner and while No Magic primarily sells UML modeling tools, we seemed to agree that UML is most useful (in my opinion "at all useful") when you imbue the vanilla models with custom semantics - aka you turn them into a DSL. So while I liked hanging out with Daniel, his DoDAF session did nothing except ensure I never work for the DoD. There's no amount of money that's worth dealing with the two dozen or so bureaucratic models that are all wholly isolated from anything that actually executes. Daniel kept saying how easy these models are to build. I'm sure they are, but that's not the problem. Since they're not an intrinsic part of a construction process, they won't stay up to date. This was a very industrial approach - Daniel even stated at one point that he was "anti-Ambler".
• David Chappell did the second keynote on grid-enabled SOA.
• When did David join Oracle? I guess I haven't been paying much attention to competitors since I moved to MSIT.
• There's an article version of this presentation available, but I haven't read it yet.
• For me, the best part of this presentation was him acknowledging that there's a need for non-stateless services, something he has blogged about recently. I'm not sure I agree with his framework for stateful interaction, but at least he's admitting that it's needed. Now if I could only convince the Connected Systems Division...
• The rest of his talk was basically a sales pitch for the Coherence product Oracle recently bought. Basically, it's a huge, multi-node, redundant, in-memory database. While I'm sure there are a few high-end problems out there - my immediate thought was travel and David mentioned SABRE is one of their customers - this is not a good general purpose solution, though David was positioning it as such.
• My talk on "Moving Beyond Industrial Software" was after the second keynote. It was good, if sparsely attended. I'm doing it again @ the p&p Summit so I'll post the slides and hopefully a recording after that.
• I skipped the last session of the day to decompress, so the next session I went to was the day two opening keynote by Fred Waskiewicz, OMG's Director of Standards. His talk, unsurprisingly, was on the value of standards - in particular, OMG's standards. This was about as anti-Ambler, anti-agile, pro-industrial a presentation as you could make. I'd heard this spiel before, so I mostly tuned out. I did challenge Fred on his point that the UML models are at a higher level of abstraction than code. They're not - they're a visualization and they're very useful, but they're at the exact same level of abstraction as code. That's why you can automatically generate the visualization in tools like Visual Studio's class designer. Fred didn't have much of a response to my question, though he did point out that some models like Business Process Models are, in fact, higher levels of abstraction.
• Next was what I thought was the best presentation of the entire show, IASA Founder Paul Preiss on what architects need to know. Note, I'm not brown-nosing Paul here - I'm the guy that first decided to commit Microsoft as an IASA sponsor, so he has to like me even if I thought his session was crap. Paul talked about architect as a career, comparing it to doctors. He worries that he's over-using that analogy, but software architect has much more in common career wise than it does with building architects IMO. I wonder where one might do their architecture residency? He also thinks of architects as "living governance", saying that project managers answer to the stakeholders while architects are beholden to the stockholders. I like that approach to governance.
• Finally, I attended Vince Casarez's session on Web 2.0 in the enterprise. Vince is an Oracle VP and this turned into a sales pitch like David Chappell's keynote did. I'm not sure what product it was, but it reminded me of QEDWiki from IBM that I saw at ETech last year, which isn't a complement. If you're going to build an enterprise mashup designer, is it just me or is "lots of code spew" a poor model. Why not go for something like Popfly or Pipes?
• I left early the second day in order to get home before my kids went to sleep (which I failed at due to lack of naptime). Overall, the conference was pretty good, though a bit sparsely attended in part I think because they held it in San Diego. The Orange Country IASA user group is very popular, so I don't understand why they didn't just hold it around there somewhere. Live and learn, I guess. They did have to postpone the DC event until next year sometime. Here's hoping I get invited to that as well as well as ITARC SoCal '08 (note, that *is* brown-nosing a bit)

Morning Coffee 57

• Scott Hanselman's post on Mindful Coding reminded me of the practice of rubberducking. The basic idea is that when you're stuck on a problem, you explain it out loud to an inanimate object - aka the rubber duck. (though when I originally heard about this practice, it was a teddy bear.) Maybe instgead of Coding Mindfully, we should be Coding Out Loud?
• Quick side note to the previous bullet: I have often worked thru a problem by explaining it to my wife who, like Scott's wife, nods in all the right places, but cares not about such things. But calling your wife a rubber duck is bad for your health, so I'd rather call it Coding Out Loud.
• I'm a couple weeks behind on this, but Microsoft along with BEA, BMC, Cisco, Dell, EMC, HP, IBM, Intel, and Sun submitted the Service Modeling Language to the W3C. For those not plauing along at home, SML is the new name for the System Definition Model and is a core deliverable of the Dynamic System Initiative. Good to see it's gotten such broad support for this.
• Jezz Santos and Edward Baker wrote a series of posts entitled "Factories 201". The entire series is good, but I particularly liked Jezz' post How Long Will It Take? His rough estimate is that it takes at least five products built with a software factory before you recoup your investment in building the factory itself. Sounds like a fair assumption.

Cut Out The Middle Man

Nick Malik is an architect in MSIT's Enterprise Architecture group. He's been blogging a while, though I only discovered his blog a couple of weeks ago. Yesterday he posted about OMG's SOA SIG's Draft RFI on EDA and it's relationship to SOA and BPM. That's a veritable alphabet soup of acronyms! To translate, the Object Management Group's Special Interest Group on Service Oriented Architecture has posted a draft Request for Information on Event Driven Architecture and it's relationship to Service Oriented Architecture and Business Process Management. Here's the summary from the actual document:

The EDA Sub-group of the OMG SOA SIG seeks information from members of the EDA, BPM and SOA community as well as anyone interested in promoting standards in this area. Requested information will be evaluated by the EDA Sub-group, resulting in the development of Requests for Proposal(s) (RFP) for standardization of Event definition, relationship between EDA, BPM and SOA that will ultimately allow development of standards for Complete Life Cycle of Events -Ontology of Events, Sense and Respond Services, Events Metrics and processing of complex events. Please note that it is our intent to develop modeling standards for the EDA/SOA and EDA-Business Process interaction and provide standards for the implementation of that interaction as well.

First off, I'm a bit wary about this part: "it is our intent to develop modeling standards". Of course, OMG is responsible for UML and long time readers should be well aware of my opinion of UML. I don't want to set the bozo bit on an entire organization, but I am skeptical that any new modeling "standards" from OMG will be any more effective than the Unwanted Modeling Language.

Secondly, EDA seems to be vaguely defined, if at all. Wikipedia has this to say about EDA:

An event-driven architecture (EDA) defines a methodology for designing and implementing applications and systems in which events transmit between loosely coupled software components and services. An event-driven system is typically comprised of event consumers and event producers. Event consumers subscribe to an intermediary event manager, and event producers publish to this manager. When the event manager receives an event from a producer, the manager forwards the event to the consumer. If the consumer is unavailable, the manager can store the event and try to forward it later. This method of event transmission is referred to in message-based systems as store and forward.
[emphasis in original]

Assuming events are encoded as messages, then you can rewrite "event consumers / event producers" as "message receivers / message senders" and you really blur the line with SOA? 

The big difference in EDA seems to be the use of an "intermediary event manager". The problem I have with this approach is that the "intermediary event manager" works fine if you have a small number of endpoints, but how will it scale to handle hundreds of systems? Thousands? Tens of thousands? I don't see how the centralized event manager approach can possibly scale to handle tens of millions of events delivered between tens of thousands of systems. The management of such a system would be a nightmare? If a business process went south, you would obviously look in the central event manager as the source of the problem, but I would think that would be like finding a needle in a haystack. You could federate the event managers, instead of attempting to scale out the center. But a federated event manager approach would seem to defeat much of the purpose of an EDA in the first place. If you're going to federate your event managers, why not cut out the middle man and make each event producer it's own event manager as well? What is the benefit of separating these capabilities?

I guess fleshing out EDA isn't a bad idea, but it seems more like a solution looking for a problem to me.

Is SML Another Unwanted Modeling Language?

Piyush Pant wonders if SML solves a problem that nobody has? He also points out SysML project, which recently got folded into the OMG. Well, that explains why we called it "Service" instead of "System" modeling language.

Now that I work in IT, I can definitely say that SML will eventually solve a problem that I have. Most people agree that operations today is way to dependent on manual processes to scale effectively. Now SML doesn't solve that issue directly - as Piyush pointed out SML is a meta-modeling specification. However, SML is the foundation for the next generation of operational modeling tools like what we see in Visual Studio Team System for Architects. As I wrote several years ago, VSTS:A solves a very common problem - developers lack of understanding about the deployment environment. Piyush, haven't you ever had a long weekend going back to the drawing board because the solution you had built was undeployable and you didn't discover that fact until the operations team attempted to deploy it? If you haven't, I envy you.

On the surface, I agree with Piyush when he says that "history of software is littered with unsuccessful attempts to impose monolithic modeling constructs". However, the fact that it keeps happening indicates the problem hasn't been solved. Wanting to solve a problem and being able to solve a problem are two different things. Furthermore, the history of software is also littered with very successful attempts to raise the level of abstraction by the introduction of new programming languages: C, C++, VB and Ruby are all examples of this. Given that Code is Model, what we have is a history of software littered with some successful and some unsuccessful modeling constructs. I would argue that the successful modeling constructs have taken a bottom up approach - build a language a small abstraction step above something that actually runs and compile down. These unsuccessful modeling constructs (*cough* UML *cough*) take a top down approach - build a language way above anything that actually runs and hope a miracle happens to keep it in sync with the stuff you actually build.

The question is whether SML will be top-down (i.e. a failure) or bottoms-up (i.e. a success). So far, it's to early to tell, but I have high hopes.

Service Modeling Language

On the one hand, this seems like a somewhat arbitrary name change - from System Defintion Model to Service Modeling Language. And the use of the term "Service" instead of "System" seems sketchy to me.

On the other hand, you gotta love this line from the press release: "The group plans to submit the draft specification to an industry standards organization later this year." Given the companies involved in "the group" - MSFT, IBM, BEA, Sun, Dell, BMC, Cicso, Intel, HP and EMC - you gotta think SML has a bright future.

Here's hoping that publishing the SML spec is the first step in a public-workshop-based revision process, a la the Web Services Protocol Workshops.

HawkEye on Entity Data Model Announcement

My pal Tim dropped me an email last week to let me know they (the ADO.NET team) were publishing their vNext vision around entities. Of course, they picked the week when I'm in San Diego! So I didn't get a chance to look at it until today. In a nutshell, they are raising the level of abstraction for databases. Regular DevHawk readers know I talk about abstraction a lot around here. In fact, one of my earliest posts on this blog - 1 house, 2 kids and 5 jobs ago - was on Disruptive Programming Language Technologies. So this is a topic near and dear to my heart.

This is an amazingly good thing. Think of the impact VB had on the development industry, but bigger. The abstraction level of databases hasn't been raised in decades. It's about freaking time we did.

My only problem with the article is that it's pretty obtuse. Referring to this as "Making the Conceptual Level Real" makes it sound much less exciting than it really is. Nobody refers to C# as a "conceptual" programming language. But if you use the terminology from the vision article, that's exactly what it is. Machine code is the physical level, IL is the logical layer and C# would then be the conceptual layer. But lets say you build a compiler that compiles C# directly to machine code. Would it suddenly become the logical layer? Who knows? Who cares? Let's just raise the level of abstraction and not get all caught up naming the level we're currently at.

VB was introduced 15 years ago in 1991. Most developers in the industry are aware and remember the impact VB had (if you don't, check out Billy Hollis' History of BASIC). The relational model was introduced 36 years ago, The first RDBMS was introduced in 28 years ago. I'd bet the majority of developers in the industry today don't remember a time before databases. Hell, I was introduced 36 years old myself. (I'm sure my dad remembers programming before databases, but he doesn't code much these days.)

As I said, this is going to be big and it's about freaking time. So hats off to the ADO.NET team. Can't wait to see this running. According to this, first CTP drop is August, so you don't even have to wait too long.

June DSL CTP

Congrats to the team for their latest version of the DSL Toolkit, integrated into the June CTP of the VS SDK. According to the published product plans of the VS SDK, they're suppoesd to ship their next release - including the final DSL toolkit - next month. Looking forward to it.

Talking Dynamic Lanugages with Neal Ford

I spent a couple of hours chatting with Neal Ford from ThoughtWorks yesterday. Ted Neward had virtually introduced us a few months ago and he was in town for MTS, so he arranged a meeting. I had asked Ted to introduce me to some dynamic language folks for some research and public debate purposes, and Neal was one of the people he hooked me up with. Unfortunately, this was right before I changed roles and got real busy. Of course, dynamic languages in general and Ruby in particular plays a large role in Edge Architecture, so I’m thankful Neal took the time to drop me a line and meet with me.

Above all else, talking to Neal made me realize that I just don’t know enough about dynamic languages, which limits my ability to discuss them. To date, I’ve flirted with them, but haven’t made a real commitment. For example, I’ve played around with Instant Rails, but hadn’t actually installed Ruby yet. It was time to re-image my dev partition anyway, so I’m going to try using Ruby exclusively for a while.

Here’s a brain dump of some of what we talked about. Not sure what it all means yet, so I’ll try and refrain from making commentary.

• Hungarian notation for interfaces (i.e. ISomething) is a big code smell. This has nothing really to do with Ruby or dynamic languages, but it’s an important point that I wanted to include here. Neal’s point is that the interface defines the semantics of the type and the concrete class is an “implementation detail”. In other words, contract-first isn’t just for web services. Apparently, ThoughtWorks doesn’t use ADO.NET directly primarily because the interfaces “aren’t pervasive enough” and are difficult to mock out. Also, they’re using Rhino Mocks which I wasn’t previously aware of.
• For all the debate about static vs. dynamic languages, it seems like the value Ruby brings is in meta-programming rather than dynamic typing. Certainly, that’s one of the big differentiators for Ruby vs. other dynamic languages like Python. While Rails has pushed the popularity of Ruby thru the roof recently, Neal seems much more enamored with Ruby than Rails.
• There is an even bigger gulf between dynamic and static typing proponents than I had thought. I brought up Singularity, which uses static typing exclusively to deliver a provably dependable system. Neal disagreed with that approach, pointing out that “tests are the best way of encoding the specification of the system” rather than compile time checking. Given my lack of expertise in this space, I’m withholding comment (for now) but I’m guessing the truth is somewhere in the middle.
• However, while the dynamic vs. static typing gulf is big, meta-programming is potentially the bridge. I don’t believe meta-programming is exclusive to dynamic languages. Certainly, some of the new features in the “Orcas” versions of C# and VB bring more expressiveness to the languages while still remaining type safe.
• All this meta-programming leads to domain specific languages. Ruby has strong support what Martin Fowler called “internal DSLs”, but Neal thought over time the focus would shift to external DSLs as they are more expressive and not constrained by the semantics of an existing language. Obviously, we’re pretty heavily focused on DSLs. However, Neal did think our focus on graphical DSLs is misplaced. He called them a “hangover” from CASE/UML tools. He rightfully pointed out that “business analysis speak English”.

All in all, it was time well spent. Neal, I hope we can pick up the conversation again sometime.

New DSL Toolkit Drop

I have been so focused on Web 2.0 stuff that I've been reading a bunch of new blogs and disregarding the old ones I used to read. So I didn't realize until today that the DSL Tools team released a new drop last week. According to Gareth, the highlights include:

• Integration into the Visual Studio SDK. According to the site, they are shooting for an April release for v2 of the VS2005 SDK, so does that mean the DSL Tools will be done in April?
• Single file format and complete visual designer for all aspects of a DSL. I'm guessing this mean we no longer have to edit the designer definition by hand. That's a good thing. But I liked the seperation of domain model and designer, so I'll be interested to see how I like what they've built.
• Domain-specifiic model serialization. This is huge - previously, the domain model dictated the XML serialization format. Now, if you can customize this, you can provide a clean model syntax and even possibly read in other syntaxes as well
• Port Shapes and a revised modeling API

Update: Apparently, I can't read. Only the VS SDK integration is done in this build. improvements to the file format and model serialization will be in the next drop.

Evolving Language Fidelity

I haven't seen much in the way of response to my Hi-Fi Models post, but I did come across this great article by Ted Neward on the history of the tumultuous marriage of objects and relational databases, primarily in the context of LINQ. In the context of Code is Model, the following passage from the summary was the most interesting:

While Project LINQ doesn't purport to be the "final answer" to all of the world's object-relational mismatch problems, it does represent a significant shift in direction to solving the problem; instead of taking an approach that centers around code generation, or automated mapping based around metadata and type inference, both of which are exercises in slaving the relational model to an object-oriented one, Project LINQ instead chooses to elevate relations and queries as a first-class concept within language semantics and library-based extensions.
[Comparing LINQ and Its Contemporaries - Ted Neward]

When Ted says relations and queries are elevated to "first class concepts" within the language, it makes me think of Stuart's comment about language fidelity. I'm not sure I would say C# 3.0 is at a higher level of abstraction than 2.0, but I would say that the inclusion of these new abstractions does improve the language's fidelity. This fidelity improvement does come at the cost of complexity (TANSTAAFL) but compared to the current alternatives, I'm willing to pay that price.

The problem with increasing the language fidelity like this is dealing with the outdated code it leaves behind. You see this today with the addition of generics in the 2.0 CLR. How many hand-coded or generated strongly typed collections are floating around out there from the 1.x days? Lots. (As if 1.x was so long ago!) How much database access code is floating around out there today? An astronomical amount. Every app that touches a database or processes XML will be outdated with the arrival of C# 3.0 and VB 9.0. But the price of converting this outdated code to use the new abstractions probably won't be worth the time or risk. That means you're left with maintaining the outdated code while also writing any new functionality with the new language features.

I wonder how DSLs will be impacted by this evolving language fidelity issue? On the one hand, the nature of DSLs is that they have much narrower usage (i.e. one domain) than something like generics or LINQ. On the other hand, I expect DSLs to evolve faster than general mainstream languages like C# can. So I'm thinking the impact will be about the same.

Hi-Fi Models

I'm slowly but surely working through my holiday backlog of email and weblogs. Slowly being the operative word here.

Anyway, Stuart has a great post on the process by which we build models. And he's not talking theoretically here, he's working on a model for the designer definition file in the DSL toolkit. (Which is good news in and of itself as hand-writing the XML dsldd file is a pain in the butt. Though until then there's the great Dm2Dd tool from Modelisoft). The iterative process he describes certainly looks a lot like development, in the same way that C# development looks like C development. Similar steps taken on different concepts. Additionally, he's working bottom up - the output of the model will eventually be a working program (a designer in this case) which is the point I made in Abstraction Gap Leapfrog. There are existing abstractions that work now (i.e. the code generated from the existing dsldd file) and he's trying to building something one level up from there.

I also like Stuart's use of "fidelity" instead of my use of "complete". Stuart uses it as an indication of how correct a given model is. That's what I was implying when I said "complete" but "fidelity" captures the idea much better. I could imagine both lo-fidelity and hi-fidelity models for a given domain, though I would imagine you would always want to use the highest fidelity model available. The difference would be the amount of custom code you have to write - the higher the model fidelity, the less code you write by hand. And I would imagine the model's fidelity would evolve over time, which introduces interesting questions regarding language evolution as well as the evolution of projects built with those languages.

I hope Stuart keeps blogging about this project.

Abstraction Gap Leapfrog

One of the cool things about having a blogging conversation with someone on the other side of the world is that while you sleep they are thinking of a good response to your post. The only downside? Having to deal with rampant misspelling like "artefacts". :)

Anyway, Gareth responds to my post:

Until we get models that are perfectly aligned with our business domains, we'll have people who want to create models but who get them slightly wrong from a precision point of view - usually in the places where the imperfect models interact with other aspects of the system across or down the abstraction stack.

With code, you'd likely not want to have people check in sources that don't even compile and then hand them off to other folks who do make them compile, but I think that's exactly the type of process we'll see emerging in modelling for a while. I feel this way because I don't foresee us getting modelling languages of pure business intent 100% right for some time yet - we're simply not close enough to formal enough descriptions of systems as intensely human as a business yet. However, I hope we won't want to try and keep modelling as locked away with the techies as traditional development has been. (Hope I'm not talking myself out of a job here…)

[Pseudomodels and intent]

I keep saying incomplete and Gareth keeps saying imprecise, but I think we can both agree on the term "imperfect". There's a massive difference between having an precise language that is imperfect versus a language that is inherently imprecise like UML.

However, I think the primary disconnect here has to do with Gareth and my views on how higher abstracted languages will evolve. Gareth's comments about modeling "pure business intent", having "models that are perfectly aligned with our business domains" and not "keep[ing] modelling as locked away with the techies" imply to me that Gareth wants to work down from the high level business abstractions into implementable technical abstractions. Frankly, I don't think that's very likely. Leapfrogging a few levels of abstraction hasn't worked well in the past (CASE and UML/MDA) and I don't think it will work well now.

I find it much more likely that we will build higher level abstractions directly on top of existing abstractions. Again, this is similar to the way C++ built on C which in turn built on ASM. Sure, that could keep modeling "locked away with the techies" for a while, but we're already beginning to see the light at the end of that tunnel. Windows Workflow Foundation is a significant leap in abstraction while also being something than non-techies can use. Reports about about Sharepoint "12" embedding the WF engine and FrontPage "12" providing a Workflow Designer for building SharePoint workflows. While I imagine (and I haven't used any of the new Office "12" suite so this is pure conjecture) these WF tools are targeting the "power user", they certainly aren't only for developers.

Believe me, I would love to be wrong about this. I would much rather work down from or business user intent than up from the technical foundation. I just don't think it's feasible. The process Gareth describes breaks the "Model Transformation must be Deterministic" tenet of Code is Model, though the word "must" may be to strong to allow for language evolution.

Imprecise vs. Incomplete

Gareth responds to the first tenet of Code is Model:

[A]lthough as an industry we desperately need to drag models kicking and screaming from the far left of pretty-picturedom a good long way to the right in the direction of precision, I don't want to throw the baby out with the bathwater.

I'm going to take it as a given that folks believe that precise models are valuable development artefacts. Why do I think imprecise models are also valuable? Here are three things that tools for imprecise models help you to do:

1. Communicate with people about design
2. Think out loud in a way that's more shareable that your whiteboard
3. Start with an imprecise model and progress gradually toward precise models

Hopefully the first of these is obvious - there is value in model as communication device - it's just not enormous.  I've talked before about the value of the second - I draw pictures on my whiteboard and when I'm on a conference call to Redmond, they're effectively useless.

The third is something I've only recently become a convert to.  I'm happy to have models which are not precise so long as I can still reason about them programmatically.  This allows me to have development processes that are about a quantitive process of iterative refinement.

Here's an example - in some infrastructure modelling tool, I have a node type which specifies a logical machine group.  One of its properties is the number of actual machines required to suit the proposed scale of the application to be deployed.  I'd like to be able to put "Depends on outcome of Fred's scalability investigation" into that numeric field, or perhaps "4->8".  I can still generate a pretty good report from this model, but I can't really provision a set of physical servers from it.

But here's the kicker - it's vital that I can write tools that programmatically assess this model and tell me what work needs to be done in order to make it precise.  I want to know exactly what must be done on this model before, for example, it is suitable for feeding into some kind of provisioning tool.  You might say it needs to be precisely imprecise; I prefer to think of it as quantifiable imprecision.

[Imprecise Models and Killing Hippies]

The only thing I disagree with Gareth about is terminology. Like the term architecture, model has become a catch-all for things that aren't code. Regular readers of this blog know I like to be more precise than that. As such, I think items #1 and #2 from Gareth's list aren't actually models at all. I think of them as pseudomodels, similar to the concept of pseudocode. Actually, I like the name pseudomodel - it also applies well to Grady's scaffolding. Like psuedocode, pseudomodels have tons of value in communication and reasoning about a problem but they can't be used as development artifacts.

As for the third, I think what Gareth is describing is an incomplete model, rather than an imprecise one. If the model is imprecise, there's no way to programmatically reason about it. But if we look to code as an example, obviously, there are many cases where code is incomplete. Every compiler error you've ever seen is an example of incomplete code. And because the language is precisely specified, the compiler can tell you what needs to be done in order to make it precise, exactly as Gareth requested. I don't think of writing code as "progressing gradually towards precision" and I doubt anyone else does either. And while I do see development as an "iterative process", I don't think of it as "iterative refinement". Modeling shouldn't be any different.

One area where I do see refinement being critical is in the development of the modeling language itself. Traditionally, the language stays stable while the program written with it changes. But with the introduction of DSLs, it becomes possible for both to vary independently. I would assume that a DSL would evolve over time to have better "coverage" of a given domain. For example, if I was building a CAB DSL, I would implement support for WorkItem right off the bat, but supporting WorkItemExtension would be much lower on the priority list. This represents language refinement, but I would argue it's a refinement of coverage not a refinement of precision.

Scaffolding Isn't a Model

Grady Booch @ the OOPSLA 05 Structured Design and Modern Software Practices Panel at (according to the transcript)

The most important part is the executable code. I typically throw my models away, but I always save my source code. I design because I need abstractions to help me reason out my projects.

Grady Booch ranting on his blog:

It's sad how one can be misquoted and then for that misquote to be picked up by someone else with both then making a spin of the events to support their position. How silly is that.

Juha-Pekka Tovanen quoted me from my OOPSLA panel as saying that "when the project gets closer to the delivery you normally throw away UML models."

<snip>

Let me be excruciatingly clear: Over the years I have been consistent in saying that in a) the most important artifact of any software development organization is executable code and yet b) modeling is essential in constructing such executables. This is because c) models help us reason about, specify, construct, and document software-intensive systems at levels of abstraction that transcend source code (and the UML is the accepted open standard for doing so). That being said, it is a pragmatic reality that d) some models are essential (and should be retained) while others are simply scaffolding (and should be discarded). I have never said and do not say now that one should throw away all models, as Juha-Pekka then Harry then Steve imply.

First off, Grady claiming to be misquoted his pretty disingenuous. Juha-Pekka's account of what Grady said on the panel is pretty spot on. Furthermore, Grady claming that I implied he said "throw away all models" is also disingenuous. I specifically wrote that I thought Grady was being taken out of context:

I've gotta believe that this comment was somehow taken out of context and that the Grand Poobah of the Common Semantic Model doesn't actually believe that tossing the model at the end of the project is a good thing.

But he-said/she-said nitpicking aside, where's the guidance on which models are essential and which are "simply scaffolding". Last time I checked the UML spec, none of the models are labeled as disposable. How about Rational Rose? Any dialog boxes that pop up reading: "Don't worry keeping this model up to date, it's just scaffolding"? I don't think so.

Obviously, working at a higher level of abstraction helps you reason about a project. But reasoning at high levels of abstraction doesn't mean you're modeling. When a building architect sits down with a prospective customer and a sketchpad, they may be working out great ideas but no one is going to call the result a blueprint. Grady's scaffolding "models" break nearly every tenant of Code is Model. Scaffolding isn't precise or deterministic. And if it ends up in the recycle bin, I guess it's not intrinsic to the development process.

Actually, it's good that scaffolding isn't a model. It means Grady is specifically not suggesting to throw away models. He just needs to get his terminology right.

As for Grady's request for "just one DSM out there in production", Don lists a few: Workflow languages (XLANG and WF), Business Rules Engine Languages and Build lanugages (MSBuild, Ant and NAnt). Juha-Pekka pointed to this list of DSM case studies. I'd also add UI Designers such as the Windows Forms and ASP.NET designers in Visual Studio. In the Window Forms case, the code is stored in a seperate file (yay partial classes) and are specifically marked: "do not modify the contents of this method with the code editor." In the ASP.NET case, the code for an ASPX file isn't even generated until runtime. And how about HTML itself? I'm thinking HTML qualifies as "in production"?

TechEd 05 Day One - This One's for Chris Sells

Back in November, we published a series of articles about Software Factories. As excited as I am by the concept, Chris Sells really brought me back to earth with this post:

I've been reading each of the Software Factories articles with great interest. Part 1 and part 2 did a particularly good job describing the elements of the problem space, I thought. However, when I get to part 3, I was ready to see a solution. Instead what I got was a long abstract piece defining the bits of what makes up a software factory. This is the kind of thing I'd be ready to read after I was shown a concrete example or two of working, running software factories. Do other people like reading these long, abstract articles? I find them tedious unless they're filling in and generalizing the details of something that I've already got a handle on.
[Chris Sells - Concrete Examples of Software Factories?!? - Nov 7th, 2004]

When we were planning the ARC track for TechEd this year, I sent this post to Keith and Jack and told them I wanted to show a concrete example of a working, running software factory. I wanted to impress Chris. Today was ARC302 - Building and Using a Software Factory with Jack and Wojtek. I don't think Chris was here today, but if he had been, I think he would have been impressed. It's one thing to see all the various parts working on their own, but it's very different to see everything working together in concert.

The factory scenario they demoed was for building smart client apps. p&p already has a bunch of existing assets like EntLib and UIP that is useful for building smart clients as well as a ton of guidance for building smart client apps. But long books and source code projects are not the easiest form of guidance for developers to consume. The factory ties these assets and guidance together to provide a powerful in-tool experience for building such applications. For example, they started by instancing a smart client solution template. This had two projects - the UIP related code and the main application code. The UIP project had all the boilerplate code that every UIP project needs to use, but it didn't have any code for individual UIP page flows. So they invoked a GAT recipe to create a UIP page flow. This is one way where factories differ from traditional wizards - wizards are typically only invoked when creating a new solution. Factory recopies are invoked after the initial wizard runs, meaning you can unfold the template incrementally as you go along. So in this case, you invoke the recipe to create a UIP page flow multiple times, once per flow you want to create. Running the recipe created a bunch of files, but of primary note was a DSL model file and two code generation templates. The model file was for a cool little DSL for laying out UIP page flows. The code gen templates generated all the code for the forms and the flow control as well as the config needed to implement the page flow as designed in the DSL. Then they wired up the generated forms to a web service, including usign a service agent to cache web service call results on the client.

All in all, it was a very full featured app to build in a very very short amount of time. What was interesting is that there was very little hand waving when it came to adding code. You know how demos go where they add literally pages of code? In this demo, they'd swap out the file with the empty method for one where the method had like ten lines of code. And on top of being full featured, it followed the best practices design put forth by out patterns & practices group. So it didn't fit the mold of a "quick & dirty" demo - how often does a demo app that you build on stage conform to best practices?

I gotta get through the end of this week, but then I want to get a video of the demo up on the web so you can see what I'm talking about, even if you didn't go to TechEd or attend the session. I'd love to get the code too - it's all running on VS05 Beta 2 - but you know product group guys...the next version of the demo is going to be even cooler...I think I can convince Jack to ship the current code and then ship the even cooler demo code when they get that finished.

Finally, The New DSL Toolkit CTP

The May CTP of the DSL Toolkit is finally here, as per Jochen's blog. The big new feature is compatibility with VS05 Beta 2, but according to Jochen this CTP also includes:

• New features for the model explorer and property browser in the generated designers.   
• New shape type with collapsible compartments like in the Visual Studio Class Designer.
• New text templating (code generation) engine with richer features like an include directive.

They haven't updated the DSL Workbench pages yet, but you get the CTP directly from the Download Center. You can also leave feedback in the MSDN Product Feedback Center.

Agile and DSLs Workship

I'm not sounding off in the pub, so I guess I'll blog the workshop that Alan and Steven Kelly from MetaCase are doing at XP2005. It's called "Agile Development with Domain Specific Languages" and here's the abstract:

This workshop will investigate the application of Domain Specific Languages within Agile development. A Domain Specific Language (DSL) is designed to express the requirements and solutions of a particular business or architectural domain. SQL, GUI designers, workflow languages and regular expressions are familiar examples. In recent years, Domain-Specific Modeling has yielded spectacular productivity improvements in domains such as telephony and embedded systems. By creating graphical or textual languages specific to the needs of an individual project or product line within one company, DSM offers maximum agility. With current tools, creating a language and related tool support is fast enough to make DSM a realistic possibility for projects of all sizes.

DSL Tools Web Chat

I'm still digging out my inbox after my two week vacation. I'm down to 134 items from nearly 400 this morning. I was at 16 before I left. If it gets this bad in two weeks, what's going to happen when I take my four week paternity leave?

While I'm getting up to date on email, I am way way way behind on blog reading. But I did see that Gareth is promoting a web chat about the DSL toolkit tomorrow at 9am Pacific time. Here's the abstract:

Using the Domain-Specific Languages Tools for Visual Studio 2005
Domain-specific language tools lay the foundation for software factories by providing a framework and a set of tools for delivering domain-specific visual designers that plug into Visual Studio Team System. These designers could be tools for industry verticals, such as the health care or telecommunications industries or, they could be tools for development across numerous disciplines, such as object-oriented modeling and architecture. The Microsoft Tools for Domain-Specific Languages is part of the Visual Studio 2005 SDK.

Now if we could just get a copy of the DSL toolkit that runs on Beta 2!

Puget Sound IASA Meeting Tomorrow

FYI, I'm presenting at the monthly meeting of the International Association of Software Architects Puget Sound chapter Wed night (4/27). I'll be talking about DSLs and Software Factories, including a hands-on look at the tool. If you're in the Redmond area, the meeting is on the Microsoft campus in Building 43, Room 1560 - the Jefferson Room.

DMD 2 DD - Thanks Modelisoft!

Anyone who's played with the DSL toolkit knows that it's a 50/50 experience. Building the domain model means using a pretty straight forward designer (itself a DSL). Building a desinger means groveling around in a really ugly XML file. I hear that the designer design experience is going to get better in later builds, but for now it's all hand coding. If you work thru the DSL Toolkit tutorial, you spend a lot of time finding and replacing names of domain model elements in the designer definition file. But now, Modelisoft has a utility called DMD2DD (i.e. Domain Model Definition to Designer Defintion) that automates all of that nasty find and replace / hand coding. Once you build your DMD, you run their tool and it automatically adds reasonable defaults to the DD for new domain model elements and removes designer elements from the DD for domain model elements that have been removed. The DMD2DD tutorial is much more straightforward than the DSL walkthru, since you don't do all that hand editing.

Haven't tried it yet, but if works anywhere near as well as the tutorial makes it look I'm going to keep a sharp eye on Modelisoft. From their homepage, it looks like their primary business is generating and reverse engineering .NET code from/for Rational Rose. I wonder what they're doing w/ the DSL Toolkit?

System Definition Model SDK

Thanks to Pedro Silva, I now know about the DSL Tools forum on MSDN. There are also forums for VSTESA, MSF and VSTS Workshop. Plus, you can subscribe to RSS feeds of all these forums.

Speaking of the VSTS Workshops, the 3rd workshop (after DSL Tools and MSF Agile) just launched for the System Definition Model SDK. System Definition Model - or SDM - is the underlying model that powers the VSTESA modeling tools formerly known as WhiteHorse. VSTESA models 2 of SDM 4 levels - the Application Designer, the Logical Datacenter Designer as well as the Deployment Designer which essentially models the transformation between the two. These models map directly two the Application and Application Host layers of the SDM. Currently, we don't have modeling tools for the Network/OS and Hardware layers of SDM.

Barry Gervin recently complained that the Logical Datacenter Designer sucked because you can't model a web and database server deployed to the same physical machine. But in the context of SDM's layers, you model the deployment of multiple application hosts to a single OS instance one layer lower than we currently have modeling tools for. Alex Torone, PM for VSTESA (and presenter for ARC310 - the drilldown on the Logical Datacenter Designer) penned an extensive comment in response to Barry's complaints where he explains why SDM is factored the way it is and confirms that they are working on modelers for the other DSM layers "in future releases".

Anyway, back to the SDM SDK. What's cool about this is that while we ship some models "in the box" - primarily for our own stuff (SQL, IIS, WS03, etc) - there's no reason you couldn't model any system with SDM (Oracle, BEA, Linux, Z/OS, etc). The SDM SDK download is tools, samples and docs for building your own models and extending the modeling tools in VSTESA. (Note, the actual SDM SDK is a part of the VS05 SDK - i.e. what used to be called the VSIP SDK).

I wonder how long it is before the community models some of Microsoft's major competitors in SDM?

Various Modeling Info

I was in Barcelona all last week, so I'm behind on email and blogs. I missed a couple of great posts from various folks involved in modeling at Microsoft. Gareth blogged about the text template engine in the DSL toolkit. Ali Pasha blogged about the relationship between the Distributed System Designers and the System Definition Model. And R.Ramesh pointed to the second and third parts of the Channel 9 interview with John Stall from the Class Designer team.

BTW, if you want to keep up on all the goings on the VSTS world at large, you have to subscribe to Rob Caron's Blog.

Class Designer Demo

There's a great demo on Channel 9 of the new Class Designer for Visual Studio 2005. Note, as per the CD team blog, Class Designer is available in the standard, professional and team systems editions of VS2005. (i.e. everything but express). More info on the differences between the products in the VS05 product line are available on the VS05 web site.

DSL Toolkit Team Roundup

Talk about transparency: Pretty much everyone involved in the DSL toolkit is blogging. Keith, Jack, Steve, Stuart, Gareth, Alan and Jochen. Plus, I bug these guys on email alot and they always have answers. A couple of recent items from these folks:

• Steve was interviewed by DNJ Online about Software Factories. I didn't get to read this until yesterday afternoon, but I had to present the Microsoft Technology Roadmap at the Executive Briefing Center yesterday morning. We started talking about factories and one of the things that came up was the massive value of partial classes. Steve hit o