IronPython and IronRuby CTPs for .NET 4.0 Beta 2

In case you’ve been hiding under a rock (or maybe just aren’t tracking developments in the .NET community outside of IronPython), Microsoft released Visual Studio 2010 beta 2 this week. Of course for me personally, the most important feature in Visual Studio 2010 is C# 4.0 new dynamic type (also available in Visual Basic, but since VB already supported some level of late binding it’s not exactly “new” to VB).

For those of you who want to experiment with this cool new feature, may I present IronPython 2.6 CTP for .NET 4.0 Beta 2. If you can’t think of any cool things to try with this new feature, the VB team blog has some scenarios to get your started.

Also available: IronRuby CTP for .NET 4.0 Beta 2 if you’re more into gemstones than snakes.

These are preview releases, which means they’ve gone thru basic testing. If you find any bugs, PLEASE report them via the usual channel. I wrote in my Post 2.6 Roadmap post, “we are committed to shipping the RTM of our .NET 4.0 version the day that Visual Studio 2010 is publicly available” but that means shaking out the bugs between now and then. We need your help so we’re ready to go by Visual Studio 2010 launch - March 22, 2010 as per Soma’s blog.

BTW, Alcides Fonseca suggested we call this release “IronPython 2.6 N4” since it’s designed to run on .NET Framework 4.0. I like that. What do you think?

I Hate Global.asax

One of the things I’ve always loved about ASP.NET is how easily extensible it is. Back in 2000, I had a customer that wanted to “skin” their website using XML and XSLT – an approach Martin Fowler later called Transform View. We were working with classic ASP at the time, so the solution we ended up with was kind of ugly. But I was able to implement this approach in ASP.NET in a few hundred lines of code, which I wrote up in an MSDN article published back in 2003. In the conclusion of that article, I wrote the following:

Using ASP.NET is kind of like having your mind read. If you ever look at a site and think "I need something different," you'll most likely find that the ASP.NET architects have considered that need and provided a mechanism for you to hook in your custom functionality. In this case, I've bypassed the built-in Web Forms and Web Services support to build an entire engine that services Web requests in a unique way.

Nearly ten years later, I finally ran into a situation where ASP.NET failed to read my mind and doesn’t provide a mechanism to hook in custom functionality: Global.asax.

I always thought of global.asax as an obsolete construct primarily intended to ease migration from classic ASP. After all, ASP.NET has first class support for customizing request handling at various points throughout the execution pipeline via IHttpModule. Handling those events in global.asax always felt vaguely hacky to me.

However, what I didn’t realize is that there are some events that can only be handled via global.asax (or its code behind). In particular, Application_Start/End and Session_Start/End can only be handled in global.asax. Worse, these aren’t true events. For reasons I’m sure made sense at the time but that I don’t understand, the HttpApplicationFactory discovers these methods via reflection rather than by an interface or other more typical mechanism. You can check it out for yourself with Reflector or the Reference Source – look for the method with the wonderful name ReflectOnMethodInfoIfItLooksLikeEventHandler. No, I’m not making that up.

The reason I suddenly care about global.asax is because Application_Start is where ASP.NET MVC apps configure their route table. But if you want to access the Application_Start method in a dynamic language like IronPython, you’re pretty much out of luck. The only way to receive the Application_Start pseudo-event is via a custom HttpApplication class. But you can’t implement your custom HttpApplication in a dynamically typed language like IronPython since it finds the Application_Start method via Reflection. Ugh.

If someone can explain to me why ASP.NET uses reflection to fire the Application_Start event, I’d love to understand why it works this way. Even better - I’d love to see this fixed in some future version of ASP.NET. You come the only way to configure a custom HttpApplication class is to specify it via global.asax? Wouldn’t it make sense to specify it in web.config instead?

In order to support Application_Start for dynamic languages you basically have two choices:

1. Build a custom HttpApplication class in C# and reference it in global.asax. This is kind of the approach used by Jimmy’s ironrubymvc project. He’s got a RubyMvcApplication which he inherits his GlobalApplication from. Given that GlobalApplication is empty, I think he could remove his global.asax.cs file and just reference RubyMvcApplication from global.asax directly.
2. Build custom Application_Start/End-like events out of IHttpModule Init and Dispose. You can have multiple IHttpModule instances in a given web app, so you’d need to make sure you ran fired Start and End only once. This is the approach taken by the ASP.NET Dynamic Language Support. [1]

So here’s the question Iron Language Fans: Which of these approaches is better? I lean towards Option #1, since it traps exactly the correct event though it does require a global.asax file to be hanging around (kind of like how the ASP.NET MVC template has a blank default.aspx file “to ensure that ASP.NET MVC is activated by IIS when a user makes a "/" request”). But I’m curious what the Iron Language Community at large thinks. Feel free to leave me a comment or drop me an email with your thoughts.

[1] FYI, I’m working on getting the code for ASP.NET Dynamic Language Support released. In the meantime, you can verify what I’m saying via Reflector.

Functions that Create Functions in Powershell

Since I started using Powershell, I’m very picky about what I let on my path. I feel it’s much cleaner to create aliases or functions rather than letting all kinds of crud creep into my path.

Recently, I installed the latest IronRuby release and discovered there’s a whole bunch of little batch file wrappers around common Ruby commands like gem and rake. While being able to simply type “igem” or “irake” is much easier than typing “ir "C:\Program Files\ironruby-0.6.0\bin\igem"”, I didn’t want to pollute my path - even with a product from my team. Instead, I wanted to create a Powershell function for each of those IronRuby-fied commands. Furthermore, I wanted to avoid manually creating a function for each Ruby command – these batchfiles are literally identical except for their name, so I figured it would be possible automate the function creation in Powershell. Here’s what I came up with:

I start by getting the ir alias, which I’m setting in my traditional fashion. The Ruby command files are in the same directory as ir.exe, which is what ir is aliased to. If the ir alias isn’t set, I quit out of the script without setting anything.

The make-rubyfunction function is the primary workhorse of this script. You pass in a command name as a string, and it uses set-item on the function provider to create a new function. Note, I had to explicitly create this function in the global scope since I’m running the set-item cmdlet inside a script.

Getting the value for the function took a bit of head banging to figure out. I’m used to Python, which automatically closes over variables, so my first attempt was to set the function value to something like { ir $cmdpath $args }. But Powershell doesn’t close automatically, so that fails since $cmd isn’t defined inside the function. I asked around on the internal Powershell alias, and someone pointed me to the new GetNewClosure function in Powershell v2. In other words, Powershell only supports manual closures, which is kind of wonky, but works OK for this scenario. I create a new script block that references in-scope variable $cmdpath and GetNewClosure automatically creates a new script block where that value is captured and embedded. More info on GetNewClosure in the docs.

Now, I’m using Win7 exclusively at this point, so depending on a v2 feature didn’t bother me. However, if you’re using Powershell v1, you could still accomplish something similar using text substitution. Here’s my original (i.e. pre-GetNewClosure) version of make-rubyfunction

I’m using Powershell’s standard text substitution mechanism to create the function value as a string. Note that I’m escaping the dollar sign in $args, so that does not get substituted the way $cmdpath does. GetNewClosure feels cleaner, so that’s how I ended up doing it, but both ways seem to work fine.

Finally, I pass an array of IronRuby commands down the pipe to make-rubyfunction. I love the pipe command, though it feels strange to use parentheses instead of square brackets for list comprehensions like Python and F#!

Anyway, the script – as usual – is up on my SkyDrive. At some point, I want to do something similar for common IronPython scripts like pyc and ipydbg. Until then, hopefully someone out there will find it useful (like maybe the IronRuby team?).

IronRuby 0.3

Last week was Mix09, Microsoft’s annual conference for web development and design. There were some big announcements – Silverlight 3 Beta, ASP.NET MVC RTM, TDS support for SQL Data Services, new drops of Azure and LiveFX SDKs and I’m sure a bunch of other things that I’ve forgotten.

Of course, by far the most important thing that shipped at Mix09 was IronRuby 0.3.

Jimmy has the details on the new release and John Lam did a talk at Mix on dynamic languages in Silverlight. I haven’t seen an announcement, but it also looks like there’s a new version of AgDLR - aka the Silverlight Dynamic Languages SDK – as well.

Early Christmas from Iron Languages and DLR

Tomorrow may be Thanksgiving, but the Microsoft DevDiv dynamic language teams are trying to make it feel like Christmas with three separate pre-holiday releases.

1. IronPython 2.0 RC2 
   We were really hoping to only have one release candidate, but we ended up with a couple of significant bugs that we couldn’t push off to 2.0.1. With December holidays coming soon, RC2 has a pretty small window before we declare RTM so now is the time to download the release and try your code out.
2. IronRuby 1.0 Alpha 2 
   There’s been zero blog traffic on this, just a notice on the IronRuby mailing list. As per said notice, “Notable features” include “the inclusion of iirb.bat, igem.bat, irails.bat, irake.bat”.
3. New DLR CodePlex Project T
   he DLR source has been available as part of IronPython for over a year but now they have their own home on CodePlex. Check out the Release Notes for an overview, reads some Docs and Specs or just download their initial v0.9 beta. Their v0.9 beta is synced with IPy 2.0 RC2 (and their v0.9 final will sync with IPy 2.0 RTM) but it also includes synced versions of IronRuby and ToyScript in both source and binaries. Plus, Sesh has promised “weekly code drops”. Finally, unlike IronPython and IronRuby, DLR is using the discussion section of their CodePlex site – I’m eager to see how well the new-ish discussion/mailing list integration works.

So there you go, new versions of IronPython and IronRuby plus a whole new DLR CodePlex project to boot. Enjoy.

DLR Namespace Change Fire Drill

[Ed. Note – This long post is about changes we made in the DLR to avoid type collisions with System.Core. The short version of this post is “You can safely ignore the CS1685 warning you’ll get if you embed IPy 2.0 Beta 5 or later in your C# 3.0 application”. The long version below has the gory details of what we did, why we did it, a little about how extension methods work and why you can ignore warning CS1685.]

[Author note – I don’t really have an editor.]

Between Beta 3 and Beta 4 of IronPython 2.0, the DLR team made a very significant change to Microsoft.Scripting.Core.dll. As JB noticed, the DLR expression trees have merged with the LINQ expression trees. As part of this effort, they moved the newly merged expression tree types in Microsoft.Scripting.Core.dll into the System.Linq.Expressions namespace – the same namespace where those types live in System.Core.dll in .NET Framework 3.5.

This change caused quite a few issues with our users. Basically, because of the expression tree merge and the namespace change, the beta 4 version of Microsoft.Scripting.Core.dll had type collisions with System.Core.dll all over the place, making it hard or impossible to use them together. If you're using C# you could get around these type collisions by using an assembly reference alias. However, assembly reference aliases aren’t supported for Web Sites projects or in Visual Basic.

To fix this, we're changing the top level namespace in Microsoft.Scripting.Core.dll from System to Microsoft. We're not going back to the namespaces as they were in Beta 3 - for example, DLR expression trees were originally in the Microsoft.Scripting.Ast namespace but now they'll be in Microsoft.Linq.Expressions. We don't think this change will be much of an issue because most people don't use types from Microsoft.Scripting.Core.dll directly. Unless you're building your own DLR language, this namespace change shouldn't affect you at all except to solve the type collision problem.

However, we did hit a small snag.

The LINQ expression tree code, having been written for .NET 3.5, is a heavy user of extension methods. This means IronPython is now also a heavy user of extension methods. However, unlike LINQ, IronPython has to run against .NET 2.0 SP1. That means we can't reference System.Core.dll in IPy or DLR. If you try to compile C# code with extension methods but without a reference System.Core.dll, you get a compiler error complaining that it can't find the required ExtensionAttribute type, which is defined in System.Core.dll.

This might appear to be an unsolvable problem, but it turns out the C# compiler doesn't actually care where the ExtensionAttribute type comes from. You can actually define your own copy of ExtensionAttribute (in the right namespace) and C# will happily compile extension methods without complaint. Furthermore, ExtensionAttribute is only used as a marker - there's no real code in it - so implementing your own copy is trivial. In the DLR source, you'll find they have defined their own copy of ExtensionAttribute so they can use extension methods and remain .NET 2.0 SP1 compatible. Since we were using them in Microsoft.Scripting.Core.dll, we started using extension methods in Microsoft.Scripting.dll and IronPython.dll as well.

If you'll recall back to the start of this post, we're changing namespaces in order to eliminate type collisions. The snag we hit was that we couldn’t change the ExtensionAttribute namespace without breaking all the extension methods. But we couldn’t leave it the same without having a type collision with the ExtensionAttribute defined in System.Core.dll. If you had a project including both copies of ExtensionAttribute, C# would generate a multiple type definition warning and VB would generate a multiple type definition error.

We've looked at a several possible solutions to this. One idea was to ship two completely different sets of binaries - one for .NET 2.0 and one for .NET 3.5. But the upgrade story for that stinks - you want to upgrade your app to .NET 3.5 you have to swap out all your IPy and DLR dlls. Yuck. We considered having separate copies of just Microsoft.Scripting.Core.dll - one defining ExtensionAttribute and the other linked to System.Core.dll and using TypeForwardedTo - but since the assemblies are strongly typed they'd have to same exact version number in order to be swappable. Double yuck.

In the end, we decided to put an internal copy of ExtensionAttribute in each assembly that needs it. As previously indicated, that's IronPython.dll and Microsoft.Scripting.dll as well as making the copy already in Microsoft.Scripting.Core.dll internal. For IronRuby fans reading this, we also added a copy of ExtensionAttribute to IronRuby.dll and IronRuby.Libraries.Scanner.dll as well.

It seems counterintuitive, doesn't it? To solve a multiple type definition problem, we defined even more copies of the type in question.

The key thing is that all these copies of ExtensionAttribute (except the one in System.Core.dll) are internal rather than public types. If you build a VB app that references System.Core.dll and Microsoft.Scripting.Core.dll (beta 4), you end up with multiple public copies of ExtensionAttribute and are rewarded with a VB compiler error. However, as long as there's only one public copy of ExtensionAttribute - regardless of the number of internal copies of that type - VB is happy. So if you're building a VB app against Microsoft.Scripting.Core.dll (beta 5) and System.Core, you should be golden.

In C# 3.0, on the other hand, continues to throw a warning. If ExtensionAttribute was a user-defined type, we'd be fine. However, since Extension attribute is a "predefined system type", you get C# warning 1685 even though the copies of ExtensionAttribute are all internal. Furthermore, since there are multiple internal copies of ExtensionAttribute in the IPy dlls, you'll get this warning even if you're not referencing System.Core. It seems here that C# 3.0 considers ExtensionAttribute a predefined system type while VB doesn't.

I realize that always having a warning in C# 3.0 - even if you're not referencing System.Core.dll - doesn't feel particularly clean. Given our desire to support both .NET v2 and v3.5 with the same binaries, it was the only choice. Remember that ExtensionAttribute has literally no code and is only used to signal the compiler for extension methods, so we decided it was fairly ignorable as warnings go.

If you're willing to compile from source yourself, it's fairly easy to build a set of binaries for a specific version .NET that doesn't have the warning. If you're building for v3.5, you need to remove Extension.cs from the three projects that have a copy of it (Microsoft.Scripting.Core, Microsoft.Scripting, IronPython) and add a reference to System.Core.dll. If you're building for v2.0, remove the Extension.cs from Microsoft.Scripting and IronPython then change the visibility in the Microsoft.Scripting.Core version from internal to public. Note, we treat warnings as errors in IPy, but we did add CS1685 to the list of WarningsNotAsErrors so the code still compiles. Of course, if you’re defining a framework specific version, you won’t get the warning anyway.

As usual, we appreciate all feedback from our community so hammer on this build as much as you can - esp. if you've been having type conflict errors with Beta 4. As I said in an earlier post, this is our last planned beta, so now’s the time put it thru the paces to make sure there’s nothing blocking you before we get to 2.0.

Finally, major props to Curt...aka IRON CURT...for driving these dev changes and putting up with the constant barrage of "where are we now?" status requests from yours truly. I'm sure he now regrets sitting across the hall within easy earshot.

Morning Coffee 173

I'm on my way out the door for New Zealand and Australia, but I wanted to push out a few things.

• F# August September CTP is out! Don Syme has the announcement, Jomo Fisher has the link roundup and details are on the brand-spanking-new MSDN F# Dev Center. Major congrats to the F# team. I've been running a pre-release version of these bits, and they are a huge step forward if you're an F# developer.
• I've got an article on IronPython in the latest issue of CoDe magazine. Also check out Brad Wilson's IronRuby article, Ted Neward's F# article and Neil Ford's Polygot Programming article.
• Via Michael Foord I discovered that IronPython tester Dave Fugate is back on the blog. He starts with a couple of posts about measuring IronPython performance.
• Speaking of blogging teammates, I think the dynamic languages team has the highest percentage of bloggers in any group at MSFT. All four Program Managers (Dave (lead), John, Jimmy and me), four of five developers (Shri (lead), Dino, Curt and Oleg) and all three Testers (Jim, Dave and Srivatsn). The only non blogger right now is Tomas - who at least has a home page - and  the lead tester which is an open position right now. 11 bloggers out of 12 team members equals 91.67% team blogger coverage.
• I was really impressed with Newspeak when I saw it at Lang.NET so I'm very excited to see they have a new website. No public bits yet, but I like the part where they point out Newspeak "can be implemented independently of Squeak, Smalltalk or any particular VM or IDE". How about implementing a version on DLR guys?
• Maurice de Beijer shows off embedding IronPython inside a WF application. Kinda cool, but he's primarily showing off implementing a CLR interface in IPy. How about a WF activity that execute arbitrary IPy code. That would be cool. (via IronPython URLs)
• Ironclad has reached their 0.5 milestone, being able to import numpy from IronPython. BTW, guys - I'm not sure commenting out one line that appears to be unreferenced qualifies as a "monstrous caveat". Congrats guys! (via IronPython URLs)

Morning Coffee 172

• I took the kids to see Fly Me To The Moon recently. We had to trek to Monroe (about 30 minutes away) because it's a special 3D movie, and it was only playing there and in downtown Seattle. The movie's story is insipid - three flies stow away on Apollo 11 - but all the space shots were actually kinda cool. It sure felt like they wanted to be scientifically and historically accurate about the the actual mission (well, other than the part about the flies). Patrick really liked it (he wants to build a rocket in the back yard) and Riley sat thru the whole thing with a minimum of fussing.
• I'm a big fan of Joe Biden, so I'm really happy Obama picked him to be his running mate.
• I know it's old news but what the frak was John Edwards thinking? I like his policies, but the arrogance it takes to run for president when you know you've got that skeleton in your closet is mind-boggling.
• On the other hand, watching the Sean Hannity and guest's hypocrisy on Edwards' affair, only to watch them scramble like cockroaches when Colmes points out McCain had admitted to having an affair was frakking hilarious.

OK, onto geek stuff:

• My new boss Dave Remy has moved to a new blog. If you're curious what he was up to for the 10 months he was away from Microsoft, he's happy to share.
• IPy and IRuby developer Curt Hagenlocher (aka Iron Curt) is blogging. Cue the Ozzy...I AM IRON CURT. Or don't. Anyway, he dives in the deep end of the pool - no "hello world" lollyblogging for Iron Curt - digging into the stack implications of rethrowing exceptions and debugging emitted IL.
• Srivatsn writes about static compilation of IPy scripts. Note, we're not talking about static typing - it's still the same good-old dynamically typed IronPython, just packaged up as an assembly, rather than as a bunch of .py files. Note, if you're interested in compiling IronPython, you should check out the PYC sample we published as part of Beta 4.
• Speaking of IPy Beta 4, Shri Borde posts about the COM dispatch support which is enabled by default as of Beta 4. If you're driving COM automation clients (like Office) from IPy, this is a huge improvement over the old mechanism.
• Jeff Hardy has released a new version of NWSGI, a managed version of Python's Web Service Server Gateway Interface. My understanding is that this would allow any Python web stack written against WSGI to run in IIS with IronPython (subject to IronPython's compatibility with CPython). Jeff's been documenting his efforts getting Django running with NWSGI on his blog. Awesome work Jeff! (Thanks for the correction Seo!)
• I never really bought into the "Attention Economy", but Chris Anderson's economic analysis of his DIY Drones site traffic was fascinating.
• Lutz announces "it is time to move on" from Reflector and there was a collective horrified scream in the .NET community. He's handing it over to Red Gate, who promised they "will continue to offer the tool for free to the community".
• I missed this when he posted it in June, but I really liked Nikhil Kothari use of the DLR in Silverlight to cut down on the XAML verbosity in his ViewModel action binding.
• Brian McNamara previews the new Add Reference and file ordering support in the upcoming F# CTP. I'm really looking forward to the project-to-project reference support. I can't tell you how many times I've gotten burned because my main project recompiled but my test project didn't. You just get used to hitting Rebuild All instead of Build. As for file ordering, it's a bit of a bummer that F# requires it, but the new experience is hella better than editing the project file by hand. I'm really looking forward to the new CTP.

Morning Coffee 171

• Big news for IronRuby out of OSCON. John and Jim have the details. Congrats to the IronRuby folks on reaching these milestones and paving the way for others (i.e. IPy) to follow some of the same paths.
• One of those OSCON announcements, is a project my teammate Jimmy Schementi has been working on: Silverline, which "let's you write Rails code that can run on the client".
• Shri Borde - the dev manager for IPy, IRuby and F# - tackles a tricky subject of static compilation of dynamic Python code. This came up on the mailing list recently as one of the outstanding requests for IPy to do is support custom attributes, which requires static compilation. Shri lays out some of the big issues with this approach. However, the community has been fairly clear on this, so it's obviously something we need to look at.
• I met someone from MS Research at the MS Product Fair who pointed me to the Institute for Personal Robots in Education, a joint effort between Georgia Tech and Bryn Mawr College and sponsored by Microsoft Research. Their Myro software (myro == my robot) is written in CPython, but there's an effort underway (aka Miro 3.0) to build a .NET version that uses IronPython. Must investigate.
• Seshadri shows how easy it is to extend C# types in IronPython. It's also shows how simple it is to host DLR code in your app - it's like 6 lines of code!
• Early reviews of IronPython in Action are good.  
• If you want to run an IronPython IDE in your browser with Silverlight, check out SilverShell from Dan Eloff.
• The XNA team has announced their business plans for community games. Basically, you set a price point between 200 and 800 points (aka between $2.50 and $10) and receive a "baseline" of 70% of the revenue the game generates. More details are available in the FAQ. This is pretty excited. I'd like to build some co-op kids games.
• Speaking of XNA, Caligari is now offering TrueSpace 7.6 for free . David Weller and Glenn Wilson provide an XNA viewpoint on the announcement, Chris Pendleton shows how to upload your models to VirtualEarth.
• Congrats to the CodePlex team on their latest drop, which features that a cool new feature - Mailing Lists! IronPython has had a Mailman mailing list for years, so I'm not sure we'll use this feature on IPy, but I'll investigate it
• Two PDC notes: First, Rick Rashid - VP of MS Research - will be delivering a PDC keynote. Second, the PDC team has put up a video podcast on Producing a Ginormous Conference in 10 Minutes or Less! It's the "inaugural episode" so watch for more Countdown to PDC video podcast episodes in the future.
• I recently discovered Chris Smith's F# blog. He's got recent posts on Mastering F# Lists and Guidelines for Readable F# code. For the F# novice, check out his F# in 20 Minutes posts (part one, part two)
• Pat Helland is moving to the SQL team. Good luck Pat!
• I like Nick Malik's formal definition of use cases, but I can't help be reminded of Charlie Alfred's Value-Driven Architecture article in Architecture Journal 5 where he said use cases were "easy to teach and explain" but that "if simplicity were the only goal that counted, we'd all still be walking or riding horses to get from one place to another."

Five Minutes Past Noon Coffee 170

• Ben Hall announces IronEditor, a simple dev tool for IronPython and IronRuby. Pretty nice, though fairly simplistic (as Ben readily admits). For example, it doesn't have an interactive mode, only the ability to execute scripts and direct the output to IronEditor's output window. However, it is a good start and I'm sure it'll just get better. One thing he's apparently considering is a Silverlight version. (via Michael Foord)
• Speaking of "Iron" tools, Sapphire Steel have had an IronRuby version (in alpha) of their Ruby in Steel product for several months now. I wonder if John's had a chance to play with it.
• Speaking of John, the ASP.NET MVC / IronRuby prototype he talked about @ TechEd is now available on ASP.NET MVC Preview 4 via Phil Haack.
• Ted Neward has an article exploring the IronPython VS Integration sample that ships in the VS SDK. As I mentioned the other day, we're starting working on a production quality implementation of VS Integration for IPy.
• Ophir Kra-Oz (aka Evil Fish) blogs Python for Executives. I like his "Risk, Recruiting, Performance and Maturity" model - four boxes, perfect for keeping an executive's attention! :) Plus Ophir has some nice things to say about IronPython. (via Michael Foord)
• Ronnie Maor blogs an extension method for PythonEngine to make Eval simpler. I especially like how he uses string format syntax so you can dynamically generate the code to eval. I wonder what this would look like in IPy 2.0 with DLR Hosting API. (via IronPython URLs)
• Speaking of DLR Hosting, Seshadri has another great DLR hosting post, this time hosting IPy inside of VS08 so you can script VS08 events (document saved, window created, etc) with Python.
• Justin Etheredge has a bunch of IronRuby posts - Getting IronRuby Up and Running, Running Applications in IronRuby, Learning Ruby via IronRuby and C# Part 1. (via Sam Gentile)
• Don Syme links to several F# related posts by Ray Vernagus, though he's apparently also experimenting with IronRuby. I'm really interested in his Purely Functional Data Structures port to F#.
• Speaking of F#, Brian has a teaser screenshot of F# upcoming CTP. However, he chooses the New Item dialog to tease, which looks pretty much like the current new item dialog (the new one does have fewer F# templates). However, if you look in the Solution Explorer, you'll notice a real "References" node. No more #I/#R! Yeah!
• The interactive graphic in Kevin Kelly's One Machine article is fascinating. It really highlights that the vast vast vast majority of power, storage, CPU cycles and RAM come from personal computers on the edge. Even in bandwidth, where PC's still have the highest share but it looks to be around 1/3rd, the aggregate of all edge devices (PCs, mobile phones, PDAs, etc.) still dominates the data centers.

Morning Coffee 167

• If you're a gamer, you're probably already well aware that E3 is this week. The Too Human demo has already been released. I have a friend who's been working on "something" that will be announced today (I think).
• Live Mesh folks pushed out an update Friday. Among the new features is the ability to sync folders among peers but NOT up to the cloud. This is cool because it means I can sync my many many GB of pictures and music on my home machine backed up with Carbonite. This means I can sync them without blowing thru my 5GB Mesh storage limit.
• It looks like there's a new F# drop - 1.9.4.19 - but as usual there is no announcement or details as to what's new. Release notes guys, look into it.  UPDATE - Don Syme blogged the release, and it's pretty minor. a .NET FX 3.5 SP1 bug fix, a fix for Mono, and they removed WebRequest.GetResponseAsync to make F# work on Silverlight. And the release notes are in the readme. My bad.
• Speaking of F#, it was "partially inspired" by OCaml, so when I see papers related to OCaml, I immediately wonder if I an apply the described techniques to F#. "Catch me if you can, Towards type-safe, hierarchical, lightweight, polymorphic and efficient error management in OCaml" is one such paper. (via LtU)
• Speaking of functional programming, Matthew Podwysocki posted a bunch of FP links as well as a Code Gallery Sample on FP in C#. Good stuff.
• As per Scott Guthrie, it looks like there's a new ASP.NET MVC drop coming this week.
• Based on posts by Ted Neward, Dare Obasanjo and Steve Vinoski, Google Protocol Buffers sounds like it's going to be a dud. Note, I haven't looked at it depth personally, I'm just passing on opinions of some folks I read and trust.
• Speaking of Dare, both he and James Hamilton take a look at Cassandra and come away impressed. I wonder how easy it is to code against from Python and/or .NET?
• Bart de Smet has a cool sample of calling out to PowerShell from IronRuby via the backtick command. Pretty cool, but it would even cooler to show how to call out to PS and return .NET objects to Ruby (though that would probably not be spec compliant for the backtick command).
• Here's a MS code name I had never heard before - Zermatt. It's "a framework for implementing claims-based identity in your applications." (via Steve Gilham)

Morning Coffee 166

Yes, I realize it’s been a while. I tried in vain to catch up with my blog reading after my Hawaii vacation and finally just gave up and hit “mark all as read”.

Dynamic Languages

• There's a new version of the DLR hosting spec available (doc, pdf). The DLR implementation is still in motion, so there are some inconsistencies between the spec and the code, but the spec should give you the high level overview you need if you want to host DLR languages inside your app.
• Oleg Tkachenko recently joined the dynamic languages team. He's the creator of the Interactive IronRuby Web Shell, an IronRuby version of Try Ruby. Of course, it’s not as cool as using SL2to execute the code directly in the browser. Michael Foord has his Python in the Browser and my teammates John and Jimmy demoed a Silverlight version of Try Ruby @ TechEd.
• Jim Deville, also of the dynamic languages team, recently started blogging.
• I have a new boss, Dave Remy. He doesn't have a blog - yet - but you can follow him on Twitter as daveremy. When Twitter is actually working that is.
• There's a new homepage/wiki for IronRuby though I’m not sure why there's a picture of Matz wearing a Python shirt on the home page.
• My teammate Jimmy Schementi provides some "continued hope" for a better (heck, I'll take current) ASP.NET and ASP.NET MVC story for DLR languages.
• Via Michael Foord, sounds like IronClad is making good progress. V0.4 can run the bz2 module "in its entirity" (maybe run a spellcheck on your site, guys?) and now apparently, it's now able to load numpy.core. Very exciting!

Other Stuff

• Pat Helland, who has blogged even less than me for the past few months, has a post up about controller and doers in the IT department. After 18 months in MSIT, put me in the doer camp, please.
• The F# team has pushed out a spec for v1.9.4 of the language. Don Syme says it's not official, but it's a huge improvement over the old informal spec
• Speaking of F#, my friend Matthew Podwysocki recently published FsTest, a testing DSL for F#. I wrote about F# unit testing as part of my PEG parsing series, and I really like the direction Matthew has taken this project. You can pull it down from CodePlex.
• When I did my PEG talk @ Lang.NET, Gilad Bracha mentioned I should check out oMeta. It looks really cool, though with the job change I haven’t had the time to play with it. Now I discover that Jeff Moser is working on a version for CLR called oMeta# that I’ve got to spend some time with. And in the comments to that post, I discovered pyMeta from Allen Short, though it apparently doesn’t work on IronPython (must investigate why).
• James Kovacs introduces psake, a PowerShell based build automation tool which uses a rake-inspired internal DSL syntax similar to one I blogged last year. I'd love to see this take off, but given MSBuild's tool integration, I wonder if that's feasible.
• I upgraded my home wireless network almost exactly a year ago. I've been happy with the range and coverage, but not so happy with the Buffalo Tech firmware. The built-in DHCP server is pretty flaky. So I upgraded to the open-source Tomato firmware. Upgrade was smooth, though I did need to reset my cable modem. But even that was smooth - Comcast has an automated service for that now,

Programming Languages @ PDC08

The PDC folks pushed out a bunch of new sessions yesterday, including a bunch from my part of DevDiv. You can see the full list of sessions that have been published (so far) at the PDC site.

An Introduction to F#
Learn about Microsoft's new language, F#, a typed functional programming language for the .NET Framework. F# combines functional programming with the runtime support, libraries, tools, and object model of .Net. Understand how F# asynchronous workflows help tame the complexity of parallel and asynchronous I/O programming and how to use F# in conjunction with tools such as Parallel Extensions for .NET.

Deep Dive: Dynamic Languages in .NET
The CLR has great support for dynamic languages like IronPython. Learn how the new Dynamic Language Runtime (DLR) adds a shared dynamic type system, a standard hosting model, and support for generating fast dynamic code. Hear how these features enable languages that use the DLR to share code with other dynamic and static languages like VB.NET and C#.

Future Directions for Visual Basic
Come learn about the new capabilities in the next version of the language, including: extensions to LINQ, syntax simplifications, and improvements to the IDE. We'll provide insight into the direction of the language, including dynamic binding, meta-programming, and scripting.

The Future of C#
In this talk Microsoft Technical fellow and C# Chief Architect Anders Hejlsberg outlines the future of C#. He will describe the many forces that influence and shape the future of programming languages and explain how they fit into C#.

Visual C++: 10 is the New 6
Get more done. The next version of Visual C++ is all about improving developer productivity for large-scale applications. Learn about the IntelliSense and browsing experiences, changes to the project and build system, project-less browsing, collaboration through remote symbol indexing, and custom visualization of symbolic information.

Morning Coffee 164

• Big news since my last Morning Coffee post was the announcement of Live Mesh. I've been running it for about a month, and I'm really digging it. Make sure you check out the team blog and watch the developer tour video (be on the lookout for IPy about half way thru the video)

ALT.NET

• I had a great time @ the ALT.NET open space conference last weekend. I was somewhat distracted on Saturday as due to a family communication mixup, I had to bring my son Patrick with me. Jeffrey Palermo shot a cute video of him (3 minutes in) where he explains that he's at the conference "to be with my dad". Having a five year old is a little distracting, but everyone was amazingly cool with having him around. When he gets a little older I have no doubt he'll be attending conferences and leading open sessions.
• I did a session on F#, but it felt kinda all over the place. I hadn't touched F# in a few months and it showed IMO. Matt Podwysocki was there to help keep the session from devolving into mass chaos. Thanks Matt.
• My favorite session of the conference was Scott Hanselman's "Are We Innovating?" talk, which I think originated from a question I asked him: There are many examples of large OSS projects in other dev communities that get ported to .NET (NHibernate, NAnt, MonoRail, etc). Can you name one that's gone the other way? I can't.
• I took Matt's advice and joined the local ALT.NET Seattle group.

DyLang Stuff

• Martin Maly posts about how dynamic method dispatches are cached in three different layers by the DLR. You shouldn't care about this stuff if you're a DLR language user, but you will certainly care about it if you're a DLR language builder.
• I'm really excited to see Phil Haack (whom I met F2F @ ALT.NET) is experimenting with IronRuby & ASP.NET MVC. True, I'd rather it was IPy, but his Routes.LoadFromRuby would work with Python with very little code change.
• Note to self, take a deeper look at Twining, the IPy database DSL by David Seruyange.
• Daily Michael Foord - Ironclad 0.2 Released. Ironclad is a project to implement Python's C extension API in C# so that IronPython could load standard Python C modules like SciPy and NumPy. So far, they're able to load the bz2 module. 

Other Stuff

• Congrats to Brad and Jim for shipping xUnit.net 1.0.
• Everyone seems to be jumping on the functional C# coding bandwagon. Bart De Smet's series on pattern matching in C# is currently at eight posts. Now Luca Bolognese is in on the action, with three posts so far on functional code in C#. I like how Luca keeps writing that the C# syntax is "not terrible" for functional programming. Again, why suffer thru the "not terrible" syntax when you could be using F# instead? (via Charlie Calvert)
• I need to take a look at VLinq. Charlie and Scott Hanselman both mentioned it recently.
• I would like to have been in the conversation with Ted Neward, Neal Ford, Venkat Subramaniam, Don Box and Amanda Silver.
• I haven't had any time to play with XNA of late, which means the great list of GDC videos Dave Weller posted on the XNA team blog will remain beyond my ability to invest time for now.
• There's a new drop of Spec# from MS Research. IronRuby is using Spec# heavily as I recall.

Morning Coffee 161

• Huge perk of the new job: new hardware. I had to give up my Dell workstation but I got a Lenovo T61p dual core widescreen laptop, an HP dc7800 dual monitor quad core desktop and a Polycom CX700 IP phone. I'm really digging the Lenovo's integrated fingerprint reader - no more password login - but I'm most impressed with their integrated driver management software. Sure beats the heck out of hunting for dozens of updated drivers all over the place like most vendors for you to.
• Minor downside to all my new toys: I spent most of my first week on the job installing and configuring said new toys.
• Caps will face the Flyers in the first round of the playoffs which starts Friday. I have a feeling that I'll be feeling poorly Friday around 3pm and have to head home early. :)

DyLang Stuff

• Apparently, Michael Foord isn't getting enough exposure on this blog. :) He left a comment to remind me to mention the IronPython URLs link blog he writes along with Mark Rees and Seo Sanghyeon.
• Speaking of Michael, his employer Resolver Systems just launched a new product: Resolver One Quant.
• Still speaking of Michael, he's quoted in the InternetNews article Python Fans Take Aim at the Enterprise.
• My teammate Jimmy Schementi posts a preview of his spare time project "Silverlight on Rails". This RoR plugin lets you declaratively specify if you want your RoR controller code to be accessed remotely via AJAX and run on the server or if you want that code to be downloaded to the client and run in SilverLight. Very cool stuff.

Other Stuff

• Don Syme provides some insight into the F# producization process. There's going to be an update to the "Research release" later this month and a CTP of the "Product release" later this summer (Brian McNamara has the CTP details). I am looking forward to these releases, though I'll probably be too busy w/ IPy to experiment much with them.
• Speaking of F#, Matt Podwysocki continues his adventures with F# with a look at tuples, records and discriminated unions. Of the three, I find discriminated unions the most interesting since there isn't anything like it in other languages I've used.
• Gregori and Chris both announce the release of Unity 1.0. Congrats guys! But if I don't have time to hack around with the latest F# release, you can imagine I won't be getting to Unity any time soon...
• Jeff Atwood recommends you build your application UI first. Furthermore, he does a good job selling the value of paper prototyping as well as introducing the concept of PowerPoint prototyping. Money quote: "You don't want something too powerful."
• Via LiveSide I discovered James Hamilton's blog. Normally, hardware infrastructure isn't really my bag, but I find his ideas around using ISO standard shipping containers as modular data center building blocks fascinating. For example, check out this post that suggests sticking modular data centers in condos would be cheaper than building data centers!Subscribed.
• Speaking of ISO, you may have heard Open Office XML was ratified as an ISO standard. Obviously, there was a lot of controversy around this, but Miguel de Icaza lists of what he considers major community wins from the standardization process. Anything that "pushed Microsoft into more open directions" is a good thing IMO.

DLR Resources & Jobs

I'm back home from PyCon, but between digging out my inbox, finishing transition reports and doing my mid-year career discussion I'm a little busy. But I did want to point at a couple of recent posts from the IPy team blog:

• DLR Resources is a partial list of links to blogs (where's mine?), mailing lists, specs, whitepapers and websites related to DLR, IronRuby and IronPython. I especially want to call out Dynamic Silverlight which not only has lots of good information but also looks frakking sweet. Props to my so-to-be teammate Jimmy Schementi.
• Dev and Test Job Openings on IronPython, IronRuby & F#. We're hiring, so if you're interested in working on language development @ MSFT, drop me a line and I'll make sure it gets to the right people. BTW, I hear the IronPython PM position has been filled.