• My boss let me borrow a Tecra M4 that he scavenged from his boss. The display is fairly twitchy, I think it’s a motherboard issue. But it’s very intermittent and I’ll get help desk to take a look. In the meantime, it sure is nice to driving a Tablet PC again. And it’s Vista ready to boot.
• Speaking of Vista, Visual Studio 2005 Service Pack 1 Update for Windows Vista. It’s a mouthful but it’s now available. Soma answers questions about the new release on MS PressPass.
• The DSL tools team keeps on rolling with the power toys. First it was the Designer Integration PowerToy, now it’s the DSL Tree Grid Editor PowerToy. Jeff Santos has the details.
• I missed the TechFest keynote yesterday, but it’s available on demand. They also have descriptions and videos of some of the technologies on display. (well, only one video so far, but I assume since the page is labeled “TechFest 2007 Videos” that more are on the way.)
• There’s new support for integrating WCF and WF coming in VS “Orcas”. Moustafa Khalil Ahmed has the details on what’s new for WF & WCF in the latest CTP drop. For me personally, the WCF/WF integration is some of the most important stuff coming in Orcas, second only to LINQ.

(Harry is on a secret mission in uncharted space this week, so instead of the daily Morning Coffee post, you get a series of autoposted essays. This post combines both some leftover learnings about Ruby from Harry’s Web 2.0 days with his recent obsession with PowerShell.)

My first introduction to the idea of internal DSLs was an article on Ruby Rake by Martin Fowler. Rake is Ruby’s make/build utility. Like most build tools like Ant and MSBuild, Rake is a dependency management system. Unlike Ant and MSBuild, Rake doesn’t use an XML based language. It uses Ruby itself, which has huge benefits when you start doing custom tasks. In Ant or MSBuild, building a custom task requires you to use a external environment (batch file, script file or custom compiled task object). In Rake, since it’s just a Ruby file, you can start writing imperative Ruby code in place.

Here’s the simple Rake sample from Fowler’s article:

task :codeGen do  
  # do the code generation
end

task :compile => :codeGen do  
  # do the compilation
end

task :dataLoad => :codeGen do  
  # load the test data
end

task :test => [:compile, :dataLoad] do  
  # run the tests
end

The task keyword takes three parameters: the task name, an array containing the task dependencies and a script block containing the code to execute to complete the task. Ruby’s flexible syntax allows you to specify task without any dependencies (:codegen), with a single dependency (:compile => :codegen), and with multiple dependencies (:test => [:compile,:dataLoad])

So what would this look like if you used Powershell instead of Ruby? How about this:

task codeGen {  
  # do the code generation
}
task compile codeGen {
  # do the compilation
}

task dataLoad codeGen {  
  # load the test data
}

task test compile,dataLoad {
  # run the tests
}

Not much different. PS uses brackets for script blocks while Ruby uses do / end, but that’s just syntax. Since it lacks Ruby’s concept of symbols (strings that start with a colon), PS has to use strings instead. Otherwise, it’s almost identical. They even both use the # symbol to represent a line comment.

There is one significant difference. For tasks with dependencies, Rake uses a hash table to package the task name and its dependencies. The => syntax in Ruby creates a hash table. Since the hash table has only a single value, you can leave of the surrounding parenthesis. The key of this single item hash table is the task name while the value is an array of task names this task depends on. Again, Ruby’s syntax is flexible, so if you have only a single dependency, you don’t need to surround it in square brackets.

In Powershell, the hash table syntax isn’t quite so flexible, you have to surround it with @( ). So using Rake’s syntax directly would result in something that looked like “task @(test = compile,dataLoad) {…}” which is fairly ugly. You don’t need to specify the square brackets on the array, but you having to add the @( is a non-starter, especially since you wouldn’t have them on a task with no dependencies.

So instead, I thought a better approach would be to use PS’s variable parameter support. Since all tasks have a name, the task function is defined simply as “function task ([string] $name)”. This basically says there’s a function called task with at least one parameter called $name. (All variables in PS start with a dollar sign.) Any parameters that are passed into the function that aren’t specified in the function signature are passed into the function in the $args variable.

This approach does mean having to write logic in the function to validate the $args parameters. Originally, I specified all the parameters, so that it looked like this: “function global:task([string] $name, [string[]] $depends, [scriptblock] $taskDef)”. That didn’t work for tasks with no dependencies, since it tried to pass the script block in as the $depends parameter.

Here’s a sample task function that implements the task function shown above. It validates the $args input and builds a custom object that represents the task. (Note, the various PS* objects are in the System.Management.Automation namespace. I omitted the namespaces to make the code readable.)

function task([string] $name) {
  if (($args.length -gt 2) -or ([string]::isnullorempty($name))) {
    throw "task syntax: task name [<dependencies>] [<scriptblock>]"
  }
  if ($args[0] -is [scriptblock]) {
    $taskDef = $args[0]
  }
  elseif ($args[1] -is [scriptblock]) {
    $depends = [object[]]$args[0]
    $taskDef = $args[1]
  }
  else {
    $depends = [object[]]$args[0]
    #if a script block isn't passed in, use an empty one
    $taskDef = {}
  }

  $task = new-object PSObject
  $nameProp = new-object PSNoteProperty Name,$name
  $task.psobject.members.add($nameProp)
  $dependsProp = new-object PSNoteProperty Dependencies,$depends
  $task.psobject.members.add($dependsProp)
  $taskMethod = new-object PSScriptMethod ExecuteTask,$taskDef
  $task.psobject.members.add($taskMethod)
  $task
}

Of course, you would need much more than this if you were going to build a real build system like Rake in PowerShell. For example, you’d need code to collect the tasks, order them in the correct dependency order, execute them, etc. Furthermore, Rake supports other types of operations, like file tasks and utilities that you’d need to build.

However, the point of this post isn’t to rebuild Rake in PS, but to show how PS rivals Ruby as a language for building internal DSLs. On that front, I think PowerShell performs beautifully.

I’m looking forward to using PowerShell’s metaprogramming capabilities often in the future.

I realize yesterday I said I was on vacation starting today. In reality, I’m not going to the office today, but I have time to post this before my vacation starts in earnest.

• I hit Zero Email Bounce in advance of my vacation. It’s been quite a while since the last time I got here and I hope to hit it much more often in the future.
• The DSL tools team shipped a Designer Integration PowerToy that allows you to integrate models from multiple DSL designers into a single authoring tool. Gareth has more here.
• Assorted PowerShell links: PowerShell Analyzer and PowerShellIDE. Both look interesting.
• Personally, I like Notepad2 but apparently the only way to add a new syntax highlight scheme requires modifying the source code. Ugh. Anyone out there already added PS support to Notepad2? How about a suggestion for a simple text editor that supports extensible syntax highlighting?
• Steve, Nick and Tomas all commented on my long running services WCF post. Tomas mentions Advanced Message Queuing Protocol (AMQP) which looks to be developing an open spec queuing system like MSMQ or MQ series. Interesting, but given the lack of involvement of the major MQ and DB vendors, I’m hard pressed to imagine this gaining any kind of critical mass.

Yesterday, I attended the Lang .NET 2006 Symposium – basically a public version of the CLR Compiler Lab I went to back in March. Unfortunately, with my new job, I couldn’t attend all three days, but I did attend day one. Here we’re my thoughts on the various sessions.

Anders Hejlsberg – LINQ and C# 3.0

• This was basically a rehash of his talk from the March Compiler lab. Makes sense as it was a new audience, but the “Query the Running Processes” demo is getting pretty old. Check out my notes from March for more details.

John Gough – Dealing with Ruby on the CLR

John is a professor from the Programming Languages and Systems group at Queensland University of Technology. They’re the ones building Ruby.NET. He’s also the author of Compiling for the .NET Common Language Runtime, a great if somewhat dated (i.e. .NET 1.0) book.

Much of John’s talk covered the ground that Jim Hugunin covered back in March around the difficulties of mapping dynamic languages to the static CLR. For example, most Ruby.NET objects are instances of Ruby.Object, with their link to a class – a Ruby.Class – managed by the Ruby.NET runtime rather than leveraging the CLR’s built-in class structure.

He didn’t spend much time talking about the really hard problems like continuations, which I was really hoping he would.

There are a series of “allied” tools coming out of this project which look really interesting in their own right:

• PE File Reader/Writer – a managed component for reading writing DLL and EXE files.
• Gardens Point Parser Generator (GPPG) – a Yacc/Bison style parser generator, written in and generating C#
• Gardens Point LEX (GPLEX) – companion to GPPG for generating C# scanners, a la LEX or Flex. Not released yet, but John indicated it would be available in the next couple of weeks.

Christopher Diggins – Cat Programming Language: A Functional Optimization Framework for the CLI

• I’m fairly sure Christopher doesn’t present often. Otherwise he would have know that there’s no way to present 107 slides in 30 minutes.
• Christopher had a hard time expressing why someone would use Cat, even when asked point blank by an audience member. Most of his 107 slides were describing various features of the language. I don’t know about the rest of the audience, but I got lost pretty quickly.
• It’s too bad Christopher was so obtuse as a speaker, as Cat seemed pretty interesting. If you skip the first 78 slides (!) of his deck, you get to a slide named “Transformation Engine” which seems to be the primary reason for Cat’s existence. The idea seems to be to build a large number (Chris said potentially thousands) of little optimization transformations which are used to “prune” the tree during the binary generation stage of a compiler.
• The only problem with this (other than the difficulty of following the presentation) is that I don’t think compiler optimization is a particularly useful area of study. I subscribe to “Proebsting’s Law” on this one: “Advances in compiler optimizations double computing power every 18 years.” This implies that programmer productivity is far more important than compiler optimization. Ruby is the latest great example of this phenomenon.

Mark Cooper – Page XML : An XML based domain specific language for developing web applications

• Page XML is a DSL for building web apps. Unfortunately, it isn’t released yet and it was hard to get a sense of what a solution built with Page XML would look like from the individual features described on slides. But I was certainly intrigued.
• As a DSL, Page XML needs to encode domain-specific abstraction. One example they provided that I thought was cool was their URL handling. Good URL design is an important usability feature. URLs in PageXML are split into constant and variable parts, so in a URL like mysite.com/news/somechannel/4, the “somechannel” and the “4″ would be variable parts that would map into parameters that are passed to a page handler. Very cool.
• There were a large number of what felt like small and simple yet eminently usable features. Too many for me to write down.
• The only think I didn’t like is the use of XML. No only are domain specific concepts like URLs encoded in XML, but also relatively mundane things like loops and if statements. This gets ugly really quickly. I imagine, the creators of Page XML did this so they wouldn’t have to build their own parser, but it really hurts the usability of the language.
• The last point really points to the need for a simple meta-language – a language for building languages. Lex/Yacc and their derivatives just don’t cut it. Ruby is good for building internal DSLs, but I’d like something faster and amenable to static typing as well as something more light weight for building external DSLs.

This post is long enough and I have “real” work to do (the downside of leaving evangelism! 😄 ). I’ll post about the afternoon sessions later.

Congrats to the team for their latest version of the DSL Toolkit, integrated into the June CTP of the VS SDK. According to the published product plans of the VS SDK, they’re suppoesd to ship their next release – including the final DSL toolkit – next month. Looking forward to it.