The Reese’s Peanut Butter Cups of Language Conferences

It’s been about 11 months since the last Lang.NET Symposium and we’ve been working on next year’s version. But then we discovered that we’re not the only ones inside Microsoft thinking about having a language conference. A couple of meetings later, and we’ve decided to combine them, which will assuredly lead to side conversations like this:

DSL DevCon Attendee: Hey, you got your compiler in my DSL!

Lang.NET Attendee: Hey, you got your DSL in my compiler!

[They sample the combined content]

Both Attendees: Mmmm, two great conferences that work well together!

Well, maybe not. But seriously, it should be a great combined conference.

However, there are some logistics things we need to work out, like how many days should the combined conference run? We figure the “right” answer to these questions depends on the likely overlap between the two conferences. Frankly, we don’t know what the overlap will be so we decided to simply, you know, ask.

If you are interested in attending Lang.NET, the DSL DevCon, or both next year, please head over to Chris Sells blog and make your voice heard.

PowerShell find-to-set-alias

I use Live Mesh to keep my PowerShell scripts folder synced between multiple machine. Some of those machines have different things installed on them or have things installed in different locations. For example, my laptop is x86 while my desktop is x64 so many things on the desktop get installed into c:\Program Files(x86) instead of the plain-old c:\Program Files folder. I wanted my shared profile script to be able to search a set of folders for a given executable to alias, and I came up with the following function.

It’s pretty simple to use. You pass in a folder search criteria – it must have a wildcard, or the function won’t work – the file you’re looking for and the alias you want to set. The function finds all the folders matching the $foldersearch criteria, then searches them recursively looking for the $file you specified. Set-alias is called for the first matching $file found – pipeline processing is halted via the break statement.

Here are the find-to-set-aliases I have in my profile:

Python, IronPython and F# aliases, no surprise there. Chiron is the REPL server for dynamic language Silverlight development. Typically, I use Chris Tavares’ vsvars script to configure the command shell for development purposes, but I find it’s nice to have aliases for TF and DevEnv handy at all times.

IronPython and LiveFX: Raw HTTP Access

One of the cool things about the Live Framework is that while there’s a convenient .NET library available, you can use the raw HTTP interface from any platform. LiveFX data is served up over HTTP and is available in ATOM, RSS, JSON or POX formats. As I’ve already shown, you can easily use the .NET library from IronPython, but I wanted to try working with the raw HTTP interface to get a feel for that as well.

Unfortunately, it was harder than I expected it to be. The big issue is that the documentation on how to LiveFX authorization tokens via raw HTTP is fairly sparse and occasionally contradictory. For example, there’s a whole section on Authentication and Live Framework, but it doesn’t cover this scenario. Luckily, I was able to figure it out with the help of AtomPub Project Manager LiveFX Sample, a post on Alex Feinman’s blog, a post on Emmanuel Mesas’ blog and a little groveling around with Reflector. It does appear that the auth docs are in flux –Emmanuel refers to this MSDN article as being about RPS Soap requests, but it’s actually about delegated authority. (Is MSDN reusing URLs? Bad idea.) Also, the sample code has a comment that reads “to be replaced by delegated authorization” so it looks like changes are coming. In other words, no promises on how long this code will work!

If you look at the AtomPub Project Manager sample, there’s a WindowsLiveIdentity.cs file that implements static GetTicket method that looks similar to both the code on Alex’s blog as well as the implementation of GetWindowsLiveAuthenticationToken. The upshot is that there’s a WS-Trust endpoint for Windows Live at https://dev.login.live.com/wstlogin.srf. You send it a RequestSecurityToken (aka RST) message (with a couple of extra WL specific extensions) and it responds with the security token you’ll need for accessing the LiveFx HTTP endpoints.

I ported the GetTicket function over to IronPython. I’m using .NET classes like WebRequest and XmlReader, but there’s nothing fancy here so I would expect it to be easy enough to port over to the standard Python library.

This code simply uses a WebRequest object to post the RST message to the WS-Trust enpoint then parses the result to find the token. get_RST_message uses standard Python string formatting to generate the RST message that gets posted to the WS-Trust endpoint. I’m not exactly sure why you need to convert the token value to a byte array and then Base64 encode it, but that’s what the sample code does so I did it to.

Once you have the authentication ticket, you need to download root service endpoint document in order to get the base URL and the profiles link. Then you can download all the profiles or you can download a specific one if you know it’s leet-speak identifier. LiveFX data can be downloaded in a variety of formats: ATOM, JSON, RSS or POX. You choose your format by setting the Accept and Content-Type headers.

I wrote the following functions, the generic boilerplate download function as well a specific versions for downloading JSON and POX:

Using JSON in Python is really easy, since I can simply eval the returned string and get back Python dictionary objects, similar to what you can do in Javascript.

Here’s some code that uses the get_WL_ticket and download_json functions above to retrieve the the user’s Personal Status Message

POX is also fairly easy, though a bit more verbose than JSON. The sample code, which I have stuck on my SkyDrive, includes both POX and JSON code, so you can compare and contrast the differences.

IronPython and LiveFX: Ori’s LiveOE.py

Ori Amiga is a Group Program Manager over in the Live Framework team whom you might have seen at PDC08 delivering the Lap Around LiveFX & Mesh Services and LiveFX Programming Model Architecture and Insights talks. And apparently, he’s an IronPython fan as posted a small LiveFX Python module to his blog. It’s pretty simple – it only wraps Connect and ConnectLocal - but it does cut about ten lines of path appending, reference adding and module importing code into a single import statement. Here’s the profile access script from my last post rewritten to use Ori’s LiveOE module.

FYI, make sure you update the sdkLibsPath in LiveOE.py – I’m not sure where Ori has installed the LiveFX SDK, but it’s *not* in the location suggested by the read me file.

BTW, it turns out the WL Profile information is read only which answers a question I had. However, reading the thread it sounds like they will eventually get around to making it read-write at some point.

IronPython and LiveFX: Accessing Profiles

I recently got access to both the Windows Azure and Live Framework CTP programs. Frankly, I’m very interested in Live Mesh, so I decided to start with a simple LiveFX program. Scott (aka ScottIsAFool) at LiveSide posted a “quick and dirty” console app that pulls info from a user’s profile via LiveFx. It’s not Mesh per se, but it does use the same framework and resource model so I decided to port it to IronPython. FYI, this app won’t run unless you’ve been received a LiveFx CTP token and provisioned yourself.

I did modify the app slightly, reading the WLID and password off the console – I was *sure* I would accidently post my personal credentials if I left them embedded in the app. Otherwise, it’s a straight port. First, I add references the LiveFX dlls. Since they’re not local to my script, I add the directory where they’re installed to sys.path, which lets me call clr.AddReference directly. Then I retrieve the user’s ID and password using raw_input (Python’s equivalent to Console.ReadLine). Finally, I connect to the user’s LiveOperatingEnvironment and pull their name, email address, personal status message and the number of contacts they have.

As per the original app, I use LINQ to find the right profile as well as count the number of contacts. I was able to reuse the linq.py file I wrote for my Rock Band song list screen scraper (though I did have to add the Count function since I hadn’t needed it previously). I’ve posted this script on my SkyDrive, and it includes my most recent linq.py file.

BTW, it doesn’t appear that you can set the PersonalStatusMessage programmatically, at least not currently. I was thinking it would be cool to build an app that sets your PSM via Twitter, but the set method of PersonalStatusMessage is marked internal. In fact, all the set methods of all the profile properties I looked at are marked internal. If someone knows how to update LiveFX resource objects in the current CTP, I’d appreciate it if you dropped me a line or left me a comment.

IronPython RTM News Gets Around

I just hit the MSDN home page, and what should I see?

It’s cool to see JasonZ, aka my group’s general manager, blogging about our product.

I also fired up Visual Studio, and IronPython is the top headline there too:

Not sure why the news is dated September 18th, but hey it’s really cool to see IronPython (not to mention the DLR, with the second headline) getting this kind of visibility.

IPy RTW FTW!

This is a very pretty sight. It’s a screenshot from the IronPython CodePlex home page showing that 2.0 is the “current release”. Yes that’s right, dear reader, IronPython 2.0 has officially been released!

Get it now!

This release marks the end of a very busy year for me, nine months to the day since I accepted the offer to join the dynamic languages team. Between helping ship IronPython 2.0 and helping manage the languages and tools PDC08 track, I’ve been swimming in the deep end of the pool all year. Feels good to not have any immediate deliverables for the next month or two.

Major, major props to Dino, IronCurt, Dave and Srivatsn who have done the heavy lifting on the IPy side this release. Also major props to the DLR team, who are releasing the final 0.9 version of the DLR later today in concert with IPy 2.0. (Update: the DLR 0.9 RTW bits are now available) And of course, HUGE HUGE HUGE thanks to the vibrant IPy community, many of whom are listed by name in the release notes.

Even with 2.0 finally out the door, there’s no rest for the dynamic. As per the release notes, “we’re planning on releasing IronPython 2.0.1 fairly soon” so keep those bug reports coming. Going forward, we’ve got big plans for IronPython and we rely heavily on the continued input from our community, so please keep telling us where we can improve.

On a personal note, the past nine months have been busy – very busy – but they’ve also been a blast. Frankly, I was hesitant about joining the product groups for a long time because I was worried about the grind, the culture, the overall experience. Turns out my fears were overblown, though I’m thinking that’s at least partially related to the fact that I work on a “little” project like IronPython rather than a huge project like Visual Studio.

IronPython and Linq to XML Part 4: Generating XML

Now that I have my list of Rock Band songs and I can get the right Zune metadata for most of them, I just need to write out the playlist XML. This is very straight forward to do with the classes in System.Xml.Linq.

XElement’s constructor takes a name (XName to be precise) and any number of child objects. These child objects can be XML nodes (aka XObjects) or simple content objects like strings or numbers. If you pass an IEnumerable, the XElement constructor will iterate the collection and add all the items as children of the element. If you’ve had the displeasure of building an XML tree using the DOM, you’ll really appreciate XElements’s fluent interface. I was worried that Python’s significant whitespace would force me to put all the nested XElements on a single line, but luckily Python doesn’t treat whitespace inside parenthesis as significant. 

Creating collections in Python is even easier than it is in C#. Python’s supports a yield keyword which is basically the equivalent of C#’s yield return. However, Python also supports list comprehensions (known as generator expressions), which are similar to F#’s sequence expressions. These are nice because you can specify a collection in a single line, rather than having to create a separate function, which is what you have to do to use yield. I have two generator expressions: (XAttribute(key, trk[key]) for key in trk) creates a collection of XAttributes, one for every item in the trk dictionary and (GenMediaElement(song) for song in songs) which generates a collection of XElements, one for every song in the song collection.

Once I’ve finished building the playlist XML, I need to write it out to a file. Originally, I used Python’s built in open function, but the playlist file had to be UTF-8 because of band names like Mötley Crüe. Zune’s software appears to always use UTF-8. In addition to setting the encoding, I also specify to use indentation, so the resulting file is somewhat readable by humans.

The playlist works great in the Zune software, but since it’s a streaming playlist there’s no easy way to automatically download all the songs and sync them to your Zune device. I expected to be able to right click on the playlist and select “download all", but there’s no such option. Zune does have a concept called Channels where the songs from a regularly updated feed are downloaded locally and synced to the device. However, the Zune software appears to be hardcoded to only download channels from the catalog service so I couldn’t tap into that. If anyone knows how to sign up to become a Zune partner channel, please drop me a line.

Otherwise, that’s So there you have it. As usual, I’ve stuck the code up on my SkyDrive. If I can remember, I’ll try and run the script once a week and upload the new playlist to my SkyDrive as well.