You know, this gets pretty long when I go a week between morning coffee posts.

Dynamic Language Stuff

• There’s a new version of the DLR Hosting spec. One of the biggest changes is the removal of DLR’s pre-canned knowledge of IPy and IRuby. Now, you use app.config to specify DLR languages and map them to extensions. Sesh has the details and a sample. If you only care about hosting IPy, you can use our language specific hosting API which doesn’t require app.config.
• Jimmy pushed out a new version of the Silverlight Dynamic Languages SDK. He’s on the lookout for a better name, one of his commenters suggested “SilverWhisper Kit”.
• Srivatsn shows how to pass Python keyword arguments to a C# method using ParamDictionaryAttribute, similar to how C# supports the params keyword via the ParamArrayAttribute. Unfortunately, there’s no syntactic sugar for calling param dictionary methods from C#. I wonder, do most people like or dislike the idea of language level support for keyword arguments in C#? I’m guessing most would be against, though I think it’s a cool idea.
• Maybe it’s me, but I’ve noticed an uptick in twitter posts containing “ironpython” of late. For example, via Summize I discovered Joseph Jude’s blog finally where he has posted a time tracking tool he built in IPy as well as his initial IPy learnings.
• Michael Foord, as usual, has been generating quite a bit of good IPy content. He did a tutorial on IPy at PyCon UK,  his book is almost done, he has some interesting thoughts on dynamic language performance and even admits that he likes C#, though not as much as he likes Python.
• Vernon Cole announced that the adodbapi project now supports IronPython 2.0 as well as CPython. This library is a DB-API 2.0 compliant interface for any database that support ADO (note, that’s native ADO not ADO.NET). I love to see Python projects with explicit support for IPy.
• My boss – who apparently will be called “Rem” from now on in the office – saves you “gazillions” by explaining non-positional negotiating strategy and draws the comparison to agile software development. Not exactly related to dynamic languages, but I’d like to see him further explore the relationship super-optimal, non-positional, significantly-hyphenated negotiation and software design. (OK, I made up “significantly-hyphenated”)

Other Stuff

• Don Syme blogs about an update to the F# CTP, a mere week after the original release. One week? That’s more often than even IPy releases. I can’t wait to see what they ship in next week’s release! 😄 Seriously, I hope they can keep the release sprints short, but every week would be a bit crazy!
• Speaking of F#, Matt Podwysocki updates FsTest for the F# CTP and posts about Extension Everything in F#. Unlike C#, which only supports extension methods, F# also supports extensions properties, static methods and events, though like Matt I can’t think of a good use for extension events.
• Still speaking about F#, Andrew Kennedy has a three part series on the new units of measure feature of F#. If you were going to use F# to build the physics engine of a game, I would suspect UoM would be extremely useful. (via Don Syme)
• Oh look, Chris Smith built an F# version of artillery game that uses Units of Measure for the physics code. I’ll bet UoM was extremely useful. 😄
• Talking about Live Mesh at TechEd Australia – where much to my surprise frankly they were demoing Live Mesh Apps – I pointed out to Scott Hanselman that Mesh is running an embedded CoreCLR (aka the same CLR from Silverlight 2). Scott went poking around and posted what he discovered. Looking forward to finding out what he digs up on using CoreCLR outside the browser.
• Speaking of Scott, I need to set up a family video conference solution like Scott’s before my next trip.
• Congrats to Glenn Block and the MEF team for their initial CodePlex source drop! I’ve been hearing about this possibility since Glenn joined the team, so I’m really excited to see it happen. I need to take a look at it in detail (in my copious spare time) because I want to find out how to make it work with IPy.
• Bart de Smet has a whole series (starting here) on Dynamic Expression Trees. However, given that he specifically writes “This blog series is not about DLR itself” makes it seem pretty conceptual to me. Why not talk about DLR expression trees instead Bart?
• I’m sure you noticed ASP.NET MVC preview 5 dropped last week. I really liked Brad Wilson’s discussion of the new view engine design.
• Tomas Restrepo has started publishing his source code on GitHub. Personally, I haven’t published any source code lately but I am using Git for all of my non IPy core work (which is stored in TFS). Like Tomas, I’m still getting the hang of Git but I’m really digging it’s speed, it’s branching and the fact that there’s zero infrastructure requirements. SVN provides the lightweight svnserve, but Git is even lighter weight than that.
• I liked Steve Yegge’s post on typing. I am a touch typer, but I doubt I type 70 words a minutes. I do know where the number keys are without looking though, so I guess that’s pretty good. I remember seeing Chris Anderson demo Avalon WPF long before it was public and being impressed at how fast he could type.

I’m on my way out the door for New Zealand and Australia, but I wanted to push out a few things.

• F# August September CTP is out! Don Syme has the announcement, Jomo Fisher has the link roundup and details are on the brand-spanking-new MSDN F# Dev Center. Major congrats to the F# team. I’ve been running a pre-release version of these bits, and they are a huge step forward if you’re an F# developer.
• I’ve got an article on IronPython in the latest issue of CoDe magazine. Also check out Brad Wilson’s IronRuby article, Ted Neward’s F# article and Neil Ford’s Polygot Programming article.
• Via Michael Foord I discovered that IronPython tester Dave Fugate is back on the blog. He starts with a couple of posts about measuring IronPython performance.
• Speaking of blogging teammates, I think the dynamic languages team has the highest percentage of bloggers in any group at MSFT. All four Program Managers (Dave (lead), John, Jimmy and me), four of five developers (Shri (lead), Dino, Curt and Oleg) and all three Testers (Jim, Dave and Srivatsn). The only non blogger right now is Tomas – who at least has a home page – and  the lead tester which is an open position right now. 11 bloggers out of 12 team members equals 91.67% team blogger coverage.
• I was really impressed with Newspeak when I saw it at Lang.NET so I’m very excited to see they have a new website. No public bits yet, but I like the part where they point out Newspeak “can be implemented independently of Squeak, Smalltalk or any particular VM or IDE”. How about implementing a version on DLR guys?
• Maurice de Beijer shows off embedding IronPython inside a WF application. Kinda cool, but he’s primarily showing off implementing a CLR interface in IPy. How about a WF activity that execute arbitrary IPy code. That would be cool. (via IronPython URLs)
• Ironclad has reached their 0.5 milestone, being able to import numpy from IronPython. BTW, guys – I’m not sure commenting out one line that appears to be unreferenced qualifies as a “monstrous caveat”. Congrats guys! (via IronPython URLs)

• I took the kids to see Fly Me To The Moon recently. We had to trek to Monroe (about 30 minutes away) because it’s a special 3D movie, and it was only playing there and in downtown Seattle. The movie’s story is insipid – three flies stow away on Apollo 11 – but all the space shots were actually kinda cool. It sure felt like they wanted to be scientifically and historically accurate about the the actual mission (well, other than the part about the flies). Patrick really liked it (he wants to build a rocket in the back yard) and Riley sat thru the whole thing with a minimum of fussing.
• I’m a big fan of Joe Biden, so I’m really happy Obama picked him to be his running mate.
• I know it’s old news but what the frak was John Edwards thinking? I like his policies, but the arrogance it takes to run for president when you know you’ve got that skeleton in your closet is mind-boggling.
• On the other hand, watching the Sean Hannity and guest’s hypocrisy on Edwards’ affair, only to watch them scramble like cockroaches when Colmes points out McCain had admitted to having an affair was frakking hilarious.

OK, onto geek stuff:

• My new boss Dave Remy has moved to a new blog. If you’re curious what he was up to for the 10 months he was away from Microsoft, he’s happy to share.
• IPy and IRuby developer Curt Hagenlocher (aka Iron Curt) is blogging. Cue the Ozzy…I AM IRON CURT. Or don’t. Anyway, he dives in the deep end of the pool – no “hello world” lollyblogging for Iron Curt – digging into the stack implications of rethrowing exceptions and debugging emitted IL.
• Srivatsn writes about static compilation of IPy scripts. Note, we’re not talking about static typing – it’s still the same good-old dynamically typed IronPython, just packaged up as an assembly, rather than as a bunch of .py files. Note, if you’re interested in compiling IronPython, you should check out the PYC sample we published as part of Beta 4.
• Speaking of IPy Beta 4, Shri Borde posts about the COM dispatch support which is enabled by default as of Beta 4. If you’re driving COM automation clients (like Office) from IPy, this is a huge improvement over the old mechanism.
• Jeff Hardy has released a new version of NWSGI, a managed version of Python’s Web Service Server Gateway Interface. My understanding is that this would allow any Python web stack written against WSGI to run in IIS with IronPython (subject to IronPython’s compatibility with CPython). Jeff’s been documenting his efforts getting Django running with NWSGI on his blog. Awesome work Jeff! (Thanks for the correction Seo!)
• I never really bought into the “Attention Economy”, but Chris Anderson’s economic analysis of his DIY Drones site traffic was fascinating.
• Lutz announces “it is time to move on” from Reflector and there was a collective horrified scream in the .NET community. He’s handing it over to Red Gate, who promised they “will continue to offer the tool for free to the community”.
• I missed this when he posted it in June, but I really liked Nikhil Kothari use of the DLR in Silverlight to cut down on the XAML verbosity in his ViewModel action binding.
• Brian McNamara previews the new Add Reference and file ordering support in the upcoming F# CTP. I’m really looking forward to the project-to-project reference support. I can’t tell you how many times I’ve gotten burned because my main project recompiled but my test project didn’t. You just get used to hitting Rebuild All instead of Build. As for file ordering, it’s a bit of a bummer that F# requires it, but the new experience is hella better than editing the project file by hand. I’m really looking forward to the new CTP.

Barry Kelly thinks that “programmers would understand monads better if they were described as a design pattern”. I agree 100% and would love to see a monad design pattern written out using p&p’s pattern form. The one thing I would note on this is that certain language constructs can make working with certain design patterns easier. For example, C# obviously has great language level support for the Iterator design pattern. Once you’ve got language level support, it doesn’t really feel like a design pattern anymore, it feels like a language feature. I mean, given that you can write OO code in a language like C, does that mean technically OO is a “design pattern”. I don’t think so.

A commenter named atp warned me not to “fall into the newbie trap of thinking that monads are about sequencing operations. They aren’t. A large number of monads (for example, Reader) are commutative and do not enforce any sort of statement ordering.” Fair enough. For example, you switch the order of some LINQ operators and still end up with the same result. If you switch Where and Select, you should end up with the same output (assuming the where clause isn’t invalidated by the select projection). But from a C#/F# perspective, I don’t really care about monads for enforcing order anyway – the language has that natively. I care much more about the context flow aspect of monads, which it sounds like atp thinks we should be focused on anyway. Works for me.

Finally, Yuri K. pointed out that we aren’t really stuck with the nested lambda expression syntax in C#. In Luke Hoban’s Monadic Parser Combinators using C# 3.0 post, he implements a Where, Select and SelectMany extension method for his Parser delegate type, which allows him to plug into C#’s query comprehension syntax. He’s 100% correct and I considered including this fact in my post. However, the mapping between query comprehension and the Bind and Result functions is a little murky, so I skipped it.

For C# query comprehensions, basically SelectMany does double duty, not only binding the parser and the parser generating function (which Luke called ‘selector’), but also taking the two parse values and calling to a projector function and returning the projection return value in a Result. By implementing SelectMany, you can rewrite the TwoValues parser like this:

static Parser<string> QueryTwoItems()  
{  
    return from v1 in Item()
           from v2 in Item()
           select string.Format("{0}{1}", v1, v2);  
}

which looks pretty much identical to the F# monadic syntax version. Luke also implements Where, which I have in my F# parser library as Satisfy. Where takes a parser and only returns the parser result if the provided boolean predicate returns true. Select is a projection, similar to SelectMany but only used with a single parser. I have a couple of specific projectors in my F# library (Ignore which tosses the parse result and Listify which turns a single result into a single item list) but I haven’t had any need for a generic projector like Select. I’m assuming Luke only implemented Select to make the query comprehension work when you don’t have multiple from statements.

In the last post, I built out a basic parser monad in C#. While the approach worked OK, the syntax is still a little foreign to your typical .NET programmer, what with it’s nested anonymous functions and all. Now, I’m going to translate that code to F# and take a look at the special monadic syntax F# supports that makes using monads as easy any sequential code.

First, let’s translate our Parser delegate, Bind, Result and Item functions over to F#. Just for kicks, let’s also port over the final version of TwoItems too.

type Parser<'input, 'result> = 'input-> ('result * 'input) option

// the Bind function, defined as a custom operator
let (>>=) p f : Parser<'i,'r> =  
    fun input ->
        match p input with
        | Some(value, input) -> (f value) input
        | None -> None

let Result v : Parser<'i,'r> = fun input -> Some(v, input)

let Item : Parser<string, char> =  
    fun input ->
        if string.IsNullOrEmpty(input)  
            then None
            else Some(input.[0], input.Substring(1))

let BestTwoItems =  
    Item >>= (fun v1 ->  
    Item >>= (fun v2 ->  
    Result (sprintf "%c%c" v1 v2)))

First, we start with the declaration of the Parser type. Unlike C#, F# has built in support for tuples, so I didn’t bother to define a Result type (just the Result function). A Parser is declared to be a function that takes in some generic input type and returns an optional tuple pairing the result with the remaining input to be parsed. As I’ve blogged before, F#’s option type is kinda like C#’s Nullable type, so a parser that returns None is considered to have failed to parse the input.

Next up is are the monad functions Bind and Result. The only significant change from the C# version is that I used the custom operator >>= for the Bind function. So instead of calling Item().Bind(some\_function), we can call Item \>\>= some\_function. F# functions aren’t attached to a type like C# extension methods are, so this is the only way to get the more readable infix notation. I’m using >>= as the bind operator because that’s the operator Haskell uses for their monad function. Other than the custom operator name, Bind and Result work identically to their C# counterparts. Note, I explicitly specified the return type of Bind, Result and Item, but I didn’t have to. F# can infer the types of all the parameters from usage just fine. I added the type specifications for the reader, in case you’re not familiar with F#’s syntax.

Likewise, Item is identical to the C# version including using strings as the parse input, except for than the F# syntax. Typically, in a real parsing app you would use an intrinsic list of chars instead of strings, since F#s list is a much more efficient data structure than strings for operations that strip characters off the head of the list (like parsers are wont to do). However, I wanted to make this code as similar to the previous code, so I stuck with strings.

Finally, we have BestTwoItems. Again, syntax aside, it’s exactly like it’s C# cousin though I did use the slightly more compact sprintf function instead of string.Format. Again, while BestTwoItems it works well, it uses the same nested anonymous function syntax from the C# version. Maybe I shouldn’t have called it “BestTwoItems”!

However, in F# it’s possible to define a custom syntax for your monad that let’s you write the function this way:

let VeryBestTwoItems =
    parse {
        let! v1 = Item
        let! v2 = Item
        return sprintf "%c%c" v1 v2 }

With this monadic syntax, we’ve now completely eliminated not only the Parser delegate and the input string, but also the nested anonymous functions needed by the Bind function, making the code appear completely sequential.

The secret to making this work is the parse monad object. It the code above, the word parse almost feels like a language keyword, but it’s not. It’s actually an instance of an parse monad object with a specific signature. F# knows how to take the syntax above and combine it with the parse monad object to produce the right code. Here’s the parse monad:

type ParseMonad() =
    member w.Delay(f) = fun input -> f () input  
    member w.Return(v) = Result v  
    member w.Bind(p, f) = p >>= f

let parse = ParseMonad()

As you can see, there’s an obvious direct correlation of Result and Bind functions we defined last time and the Return and Bind methods in the ParseMonad. The only thing we haven’t seen before is the Delay method. Monads are one of of F#’s many delayed expressions. F# wraps the entire monad in a call to Delay to ensure the monad isn’t executed prematurely.

As per the F# grammar spec, there are several other functions you can define on your monad if you so choose. My “real” parser monad also implements Zero and Combine. Zero returns a parser that unconditionally fails. By defining Zero on my monad object, I can write ifs without elses, the parser monad will implicitly inject Zero clause as your else statement. Combine combines results (I know, a shocker!). I use it as a prioritized choice. In other words, when you Combine two parsers, you only call the second parser if calling the first parsers fails. Prioritized Choice is used very often in PEGs, which is why I chose to define it this way.

F# monadic syntax also support For, Let, While, Using, TryFinally and TryWith. Frankly, I haven’t spent much time thinking about scenarios where you’d use these other syntax elements. The only one that’s obvious to me is Using for deterministic finalization, which you could see using anywhere you access IDispoasble objects. Here’s hoping the F# folks document in detail how to use this powerful syntax.

So that’s it for basic monads in F#. I’ve gotten some great comments (and one less than great comments) as I’ve written this series. In my last post on monads (in this series at least) I’ll repost some of those comments as well as provide some concluding thoughts.