Last night, after the Turing Lecture, we hosted a FlashBoF on “DSL’s in Visual Studio”. Stuart answered a bunch of questions and gave a much more detailed demonstration of the new DSL Toolkit than we could show during the keynote. Here’s what I learned from the session:

• Models are stored in XML files. The language designer outputs an object model and will eventually also output an XSD. For example, here’s a screenshot of the language designer from the DSL Toolkit we’re releasing. Inside the designer, I’ve got a sample UIP DSL (I hacked this up on my own, this is not exactly the same one we demoed yesterday). As you can see, there’s a PageCollection concept which contains Page concepts that have Name and Kind values. Page concepts also has a collection of Transfer concepts, which in turn have Label values. Generating an object model makes it easier to write tools that manipulate models. Typically, I’m anti-XML-Serialization but in this case – where we have a relative simple XSD – it works fine. I could also manipulate the model by accessing the underlying XML if I want to.
• Code generation uses templates and looks a lot like CodeSmith or old-school ASP. You interleave the static elements of the generated code with blocks of code that access the model (via the object model described above) and generate the dynamic model-specific elements of the code. So I’m guessing that people using the codegen tools like CodeSmith will feel right at home with this toolkit.
• In the current builds (which is to say later than the build that we’re releasing first – the first build doesn’t include any of the code generation support) we’re generating a single code file from a model. Eventually we’ll be able to manipulate multiple files from a single model. This is similar to how the Class Diagram works – add a new class onto the diagram and a new file gets added to the project, delete the class from the diagram and the file gets removed from the solution.
• Not all models are used to generate code. For example, in VSTS the Logical Data Center and Virtual Deployment models don’t generate code. They are useful  because I can use them to validate the Distributed System Model which does generate code.
• Someone asked about the implications of code coverage, profiling and test-driven development on a DSL-based process. Frankly, I don’t know but it certainly got me thinking. The general consensus was that we’re still in the bootstrap phase of making DSL-based development a reality and these are issues we’ll have to deal with as we move forward.

This morning was Rick Rashid’s keynote with “the big announcement“: a framework and tools for developing Domain Specific Languages in VSTS. Essentially, this is a toolkit for building your own modeling tools on top of the infrastructure that powers the VSTS Distributed System Designers (aka WhiteHorse). Personally, I think this is pretty exciting stuff and I can’t wait to play with it more in the coming months.

As part of the announcement, they demoed the toolkit with a DSL for programming the flow of pages in a web application. Typically, you would use a framework like User Interface Process for this. However, the UIP comes with 130 pages of documentation and a variety of classes and configuration settings. By building a DSL on top of the framework, we can make the developers significantly more productive. I’ve posted a screen shot of the designer – it’s essentially a box-and-stick modeling environment – boxes are pages and sticks are transitions. Dragging a new page onto the design surface from the toolbar generates a new web page in the solution. Dragging a transition onto the design surface changes the config file. That’s much more productive than mucking around in source and config files to accomplish the same task.

Of course, the standard question when we announce stuff like this is “when can I get my hands on it?” We’re going to release a tech preview version of the DSL Toolkit “real soon now”. You’ll be able to download it from the newly minted VSTS Workshop page. I hear that this project, now that it’s public, is going to be very transparent to the community, so keep an eye out for more info (of course, I’ll blog it when the toolkit is released). Honestly, the version we’re shipping “real soon” is much rougher than what we previewed in the keynote this morning, but we’ll be updating it periodically as it marches towards release.

I’ve had several good conversations in the booth with people wanting more info about VSTS and the DSL toolkit as well as Software Factories in general. As is typical, I learn as much as I teach in these sorts of discussions. For example, I met Gan Deng from Vanderbilt who showed me GME – the Generic Modeling Environment – which is DSL modeling environment similar conceptually to what we announced today for VSTS. Not sure when I’m going to get to play with it, but it’s nice to see academia support for DSLs.

The only downer is that some of the OOPSLA crowd was put-off by a product demo during the keynote. Stuart described it as a “commercial break”. Several people gave me “brutal feedback” that brazen marketing displays like product demos don’t work in OOPSLA keynotes. But others thought it was cool. Good to keep in mind for next year.

Update: Alan Kay, in his Turing Award Lecture, made a joke about about our product announcement this morning and everyone laughed, including me.

Update 2: After making a joke about our morning’s product demo (and several other assorted jabs), Alan Kay ended his Turing Award Lecture with…wait for it…a product demo. Two product demos, actually. Squeak is educational programming environment, based on Smalltalk. It is freely downloadable and has an associated community and wiki. Alan did a simple little demo where he drew a car and a steering wheel and was able to steer the car as it moved by rotating the steering wheel. Cool. Alan’s other demo, Croquet is a “massively multi-user virtual learning environment” which is built on top of Squeak. While I liked Squeak, I don’t know many people who spend significant time in a virtual 3D space who aren’t gaming. Frankly, Croquet doesn’t seem particularly practical or useful – check out this screenshot of document collaboration in Croquet. In this example, the 3D space is more of a distraction than an enabler. The only place Croquet seems useful to me is for dynamic 3D simulations, such as this screenshot demonstrating the Coriolis effect.

Mac asked me to respond to Booch’s doubts about DSLs in his blog. This is at least the second time Booch expressed his skepticism about DSLs, having also done so back in May. The upshot of Booch’s argument is that the semantics across all stakeholder viewpoints must be common.

“There is no doubt that different domains and different stakeholders are best served by visualizations that best speak their language…but there is tremendous value in having a common underlying semantic model for all such stakeholders. Additionally, the UML standard permits different visualizations, so if one follows the path of pure DSLs, you essentially end up having to recreate the UML itself again, which seems a bit silly given the tens of thousand of person-hours already invested in creating the UML as an open, public standard.” (Grady Booch, Expansion of the UML, May 21st 2004)

“[W]hile I agree that development is a team sport and that multiple stakeholders must collaborate in weaving together their diverse, interdependent views, one still needs to have a common semantic basis for all those languages. If you accept that not unreasonable position, you will end up covering the identical semantic ground as has the UML – albeit in an open manner, quite unlike Microsoft’s historical record.” (Grady Booch, Domain-specific Languages, Oct 25th, 2004)

Stuart responded to this argument back in May:

“[Booch] seems to imply that a domain specific-language is just about having a different graphical notation – that the semantics that underpin different DSLs is in fact the same – only the notation changes…[W]hat if the semantic model excludes the concepts that the stakeholders actually want to express?…Surprise, surprise, there are differences from one domain to another, from one organization to another. Far from there being a common semantics, there are significant and actual differences between the interpretations of models being taken.” (Stuart Kent, Domain Specific Modelling. Is UML really the best tool for the job?, May 26th, 2004)

In other words, Booch’s argument against DSL’s falls apart if we don’t have common semantics across all phases of the system’s lifecycle. And frankly, we don’t. Furthermore, Booch offers no explanation as to why you need a common semantic model across all the stakeholders, instead choosing to call it a “not unreasonable position”. Sorry, but I’m not ready to accept the need for a common semantic model as an axiom. I will accept that you need traceability between viewpoints, but I think that is more readily achievable by precise transformations between viewpoints than by a common semantic model of all stakeholder viewpoints.

Common semantic model starts to sound too much like “monolith” as far as I’m concerned. UML 2 addresses a subset of the stakeholder’s viewpoints and many people consider it unusable. After last year’s OOPSLA, Fowler commented “Even on the MDA panel at OOPSLA, the pro-MDA speakers based their assumptions on the fact that they would be using a simplified subset of UML”. If we can’t get a single usable semantic model across UML 2, how are we going to have a single usable semantic model across all the stakeholder viewpoints? Booch paints a pretty rosy picture of UML , but I think the reality of UML is much less attractive, certainly according to the customers I’ve spoken to.

I’m sitting in the back of the room listening to Jack present the Factories overview for the DSL Tutorial. I’ve seen most of the content for this session before, but not brought together like this. Currently, he’s talking about mapping business capabilities to system requirements. I’ve started to see quite a bit of discussion of mapping business capabilities recently. Gurpreet talked about it in his EA session n Malaysia. Jack Calhoun also talked about it at SAF. Memo to self – more transparency around business capability mapping.

BTW, check out Michael Lehman’s blog. Michael has been building (it’s not really done) the factories demo for the tutorial. He’s got some great points about DSL’s, abstraction and product lines. As I type this, he’s demoing expanding an e-commerce factory template inside of VS.NET. Cool stuff. I expect him to be blogging a lot more about building factories after this week.

Keith is getting ready for OOPSLA:

In the day-long tutorial that runs on Monday 25^th (T40 : Using Domain Specific Languages, Patterns, Frameworks and Tools to Assemble Applications) we plan to use a large proportion of the time to walk through a detailed example of using Software Factories ideas to build four different variations of the same application. This involves us using some existing technology, some technology that will ship in Visual Studio 2005 Team System, and some technology that isn’t yet planned, but which could readily be built by Microsoft or one our partners. As we plan to show the whole process – from soup (business case, business capabilities and business processes) to nuts (running code), this involves a lot of work with help from several teams here at Microsoft. It is coming together well though, thankfully!

You thinking what I’m thinking? Software Factories hands-on lab…