IronPython and Linq to XML Part 3: Consuming Atom Feeds

Now that I have my list of Rock Band songs, I need to generate a Zune playlist. I wrote that Zune just uses the WMP playlist format, but that’s not completely true. Media elements in a Zune playlist have several attributes that appear unique to Zune.

Because of Zune Pass, Zune supports the idea of streaming playlists where the songs are downloaded on demand instead of played from the local hard drive. In order to enable this, media elements in Zune playlists can have a serviceID attribute, a GUID that uniquely identifies the song on the Zune service. We also need the song’s album and duration – the Zune software summarily removes songs that don’t include the duration.

Of course, the Rock Band song list doesn’t include the Zune song service ID. It also doesn’t include the song’s album or duration. So we need a way, given the song’s title and artist (which we do have) to get its album, duration and service ID. Luckily, the Zune service provides a way to do exactly this, albeit an undocumented way. Via Fiddler2, I learned that Zune exposes a set of Atom feed web services on catalog.zune.net that the UI uses when you search the marketplace from the Zune software. There are feeds to search by artist and by album but the one we care about is the search by track. For example, here’s the track query for Pinball Wizard by The Who.

Since these feeds are real XML, I can simply use XDocument.Load to suck down the XML. Then I look for the first Atom entry element using similar LINQ to XML techniques I wrote about last time. If there’s no Atom elements, that means that the search failed – either Zune doesn’t know about the song or it can’t find it via the Rock Band provided title and artist. Of the 461 songs on Rock Band right now, my script can find 417 of them on Zune automatically.

Of course, since the Zune data is in XML instead of HTML, finding the data I’m looking for is much easier that it was to find the Rock Band song data. Here’s the code pull the relevant information out of the Zune catalog feed that we need.

A few quick notes:

• The code above isn’t valid Python, I added a couple of carriage returns (albumArtist and albumTitle) to get it to read well on the blog without wrapping badly.
• song.search_string returns the song title and artist as a plus delimited string. i.e. pinball+wizard+the+who. However, many Rock Band songs end in a parenthetical like (Cover Version) so I automatically strip that off for the search string
• duration in the Atom feed is stored like PT3M23S, which means the song is 3:23 long. The playlist file expect the song length in milliseconds, so I use a .NET regular expression to pull out the minutes and seconds and do the conversion. It’s not exact – songs lengths usually aren’t exactly a factor of seconds, but as far as I can understand, Zune just uses that to display in the UI – it doesn’t affect playback at all.

Now I have a list of songs with all the relevant metadata, next time I’ll write it out into a Zune playlist file.

Early Christmas from Iron Languages and DLR

Tomorrow may be Thanksgiving, but the Microsoft DevDiv dynamic language teams are trying to make it feel like Christmas with three separate pre-holiday releases.

1. IronPython 2.0 RC2 
   We were really hoping to only have one release candidate, but we ended up with a couple of significant bugs that we couldn’t push off to 2.0.1. With December holidays coming soon, RC2 has a pretty small window before we declare RTM so now is the time to download the release and try your code out.
2. IronRuby 1.0 Alpha 2 
   There’s been zero blog traffic on this, just a notice on the IronRuby mailing list. As per said notice, “Notable features” include “the inclusion of iirb.bat, igem.bat, irails.bat, irake.bat”.
3. New DLR CodePlex Project T
   he DLR source has been available as part of IronPython for over a year but now they have their own home on CodePlex. Check out the Release Notes for an overview, reads some Docs and Specs or just download their initial v0.9 beta. Their v0.9 beta is synced with IPy 2.0 RC2 (and their v0.9 final will sync with IPy 2.0 RTM) but it also includes synced versions of IronRuby and ToyScript in both source and binaries. Plus, Sesh has promised “weekly code drops”. Finally, unlike IronPython and IronRuby, DLR is using the discussion section of their CodePlex site – I’m eager to see how well the new-ish discussion/mailing list integration works.

So there you go, new versions of IronPython and IronRuby plus a whole new DLR CodePlex project to boot. Enjoy.

IronPython and Linq to XML Part 2: Screen Scraping

First, I need to convert the HTML list of Rock Band songs into a machine readable format. That means doing a little screen scraping. Originally, I used Beautiful Soup but I found that UnicodeDammit got confused on names like Blue Öyster Cult and Mötley Crüe. I’m guessing it’s broken because IronPython doesn’t have non-unicode strings.

Instead, I used SgmlReader to provide an XmlReader interface over the HTML, then queried that data via Linq to XML. I used the version of SgmlReader from MindTouch since they include a compiled binary and it seems to be the only active maintained version. I wrapped it all up in a function called load that loads HTML from either disk or the network (based on the URI scheme) into an XDocument.

I call load, passing in the URL to the list of songs. The “official” Rock Band song page loads the actual content from a different page via AJAX, so I just load the actual list directly via my load function.

Once the HTML is loaded as an XDocument, I need a way to find the specific HTML nodes I was looking for. As I said earlier, XDocument uses Linq to XML – there is not other API for querying the XML tree. In the HTML, there’s a div tag with the id “content” that contains all the song rows as table row elements. I built a simple function that uses the LINQ Single method to find the tag by it’s id attribute value.

(Side note – I didn’t like the verbosity of the “a != None and a.Value == id” line of code, by XAttributes are not comparable by value. That is, I can’t write “node.Attribute(‘id’) == XAttribute(‘id’, id)”. And writing “node.Attribute(‘id’).Value == id” only works if every node has an id attribute. Not making XAttribute comparable by value seems like a strange design choice to me.)

LINQ to objects works just fine from IronPython, with a few caveats. First, IronPython doesn’t have extension methods, so you can’t chain calls together sequentially like you can in C#. So instead of collection.Where(…).Select(…), you have to write Select(Where(collection, …), …). Second, all the LINQ methods are generic, so you have to use the verbose list syntax (for example: Single[object] or Select[object,object]). Since Python doesn’t care about the generic types, I wrote a bunch of simple helper functions around the common LINQ methods that just use object as the generic type. Here are a few examples:

Once I have the content node, all the songs are in tr nodes beneath it. I wrote a function called ScrapeSong that transforms a song tr node into a Song object (which I’ll talk about in the next installment of this series). I use LINQ methods Select, OrderBy and ThenBy to provide me an enumeration of Song objects, ordered by date added (descending) than artist name.

And that’s pretty much it. Next, I’ll iterate thru the list of songs and get the details I need from Zune’s catalog web services in order to write out a playlist that the Zune software will understand.

IronPython and Linq to XML Part 1: Introduction

Shortly after I joined the VS Languages team, we had a morale event that included a Rock Band tournament. I didn’t play that day in the tournament since I had never played before, but I was hooked just the same. I got Rock Band for my birthday, Rock Band 2 shortly after it came out in September and I’m hoping to get the AC/DC Track Pack for Christmas.

There are lots of songs available for Rock Band - 461 currently available between on-disc and downloadable tracks – with more added every week. Frankly, there’s lots of music on that list that I don’t recognize. Luckily, I’m also a Zune Pass subscriber, so I can go out and download all the Rock Band tracks and listen to them on my Zune. But who has time to manually search for 461 songs? Not me. So I wrote a little Python app to download the list of Rock Band songs and save it as a Zune playlist.

I ended up use Linq to XML very heavily in this project. Zune playlists use the same XML format as Windows playlists, Zune exposes the backend music catalog via a Atom feeds and I used Chris Lovett’s SgmlReader to expose the HTML list of Rock Band songs as XML. I realize Linq to XML wasn’t on “the list”, but I had a specific need so it got bumped to the head of the line.

BTW, for those who just want the playlist, I stuck it on my Skydrive. Unfortunately, there’s no Skydrive API right now, so I can’t automate uploading the new playlist every week. If anyone has alternative suggestions or a way to programmatically upload files to SkyDrive, let me know.

IronPython and WPF Part 5: Interactive Console

One of the hallmarks of dynamic language programming is the use of the interactive prompt, otherwise known as the Read-Eval-Print-Loop or REPL. Even though I’m building a WPF client application, I’d still like to have the ability to poke around and even modify the app as it’s running from the command prompt, REPL style.

If you work thru the IronPython Tutorial, there are exercises for interactively building both a WinForms and a WPF application. In both scenarios, you create a dedicated thread to service the UI so it can run while the interactive prompt thread is blocked waiting for user input. However, as we saw in the last part of this series, UI elements in both WinForms and WPF can only be accessed from the thread they are created on. We already know how to marshal calls to the correct UI thread – Dispatcher.Invoke. However, what we need is a way to intercept commands entered on the interactive prompt so we can marshal them to the correct thread before they execute.

Luckily, IronPython provides just such a mechanism: clr module’s SetCommandDispatcher. A command dispatcher is a function hook that gets called for every command the user enters. It receives a single parameter, a delegate representing the command the user entered. In the WPF and WinForms tutorials, you use this function hook to marshal the commands to the right thread to be executed. Here’s the command dispatcher from the WPF tutorial:

The dispatcher.Invoke call looks kinda like the UIThread decorator from the Background Processing part of this series, doesn’t it?

Quick aside: I looked at using SyncContext here instead of Dispatcher, since I don’t care about propagating a return value back to the interactive console thread. However, SyncContext expects a SendOrPostDelegate, which expects a single object parameter. The delegate passed to the console hook function is an Action with no parameters. I could have built a wrapper function that took a single parameter which it would ignore, but I decided it wasn’t worth it. The more I look at it, the more I believe SyncContext is a good idea with a bad design.

I wrapped all the thread creation and command dispatching into a reusable helper class called InteractiveApp.

The code is pretty self explanatory. The constructor (__init__) creates the UI thread, starts it, waits for it to signal that it’s ready via an AutoResetEvent and then finally sets the command dispatcher. The UI thread creates and runs the WPF application, saves the dispatcher object as a field on the object, then signals that it’s ready. DispatchConsoleCommand is nearly identical to the earlier version, I’ve just made it an instance method instead of a stand-alone function. Finally, I define __getattr__ so that any operations invoked on InteractiveApp are passed thru to the contained WPF Application instance.

In my app.py file, I look to see if the module has been started directly or if it’s been imported into another module. If the module is run directly (aka ‘ipy app.py’) then the global __name__ variable will be ‘__main__’. In that case, we start the application up normally (i.e. without the interactive prompt) by just creating an Application then running it with a Window instance. Otherwise, we are importing this app into another module (typically, the interactive console), so we create an InteractiveApp instance and we create an easy to use run method that can create the instance of the main window.

If you want to run the app interactively, you simply import the app module and call run. Here’s a sample session where I iterate thru the items bound to the first list box. Of course, I can do a variety of other operations I can do such as manipulate the data or create new UI elements.

One small thing to keep in mind: if you exit the command prompt, the UI thread will also exit since it’s marked as a background thread. Also, it looks like you could shut the client down then call run again to restart it, but you can’t. If you shut the client down, the Run method in InteractiveApp.thread_start exits, resets the Command Dispatcher to nothing and the thread terminates. I could fix it so that you could run the app multiple times, but I find I typically only run the app once for a given session anyway.

Resolver One 1.3 Released

IronPython’s biggest customer is Resolver Systems, makers of Resolver One, “a familiar spreadsheet-like interface with the powerful Python programming language to give you a tool to analyse and present your data.” While I think they have a great product on pure merit - I’ve been impressed with their product since I was introduced to it at Lang.NET this year - I’m particularly interested in Resolver One as it’s written in IronPython. They use IPy not only as the embedded language exposed to end users but as the underlying implementation language as well.

Furthermore, these guys are heavily involved in the IPy community. Resolver developer Michael Foord is writing a book on IronPython and was our first Dynamic Language MVP. Michael’s Resolver colleague Jonathan Hartley did me a solid by taking my space at ØreDev. Even the CTO and Co-founder Giles Thomas is a regular blogger and speaker at events. Let me tell you, having guys this great in the community sure makes my job easier.

I just wanted to give the Resolver folks a shout out and say major congratulations on shipping a new version of their core Resolver One product. Michael has more info on this release as well as a glance forward with their plans for their next (IPy 2.0, woot!)

Background Processing Re-Revisited

OK, here’s the last word on this whole background processing / concurrency decorators thing. I went back and re-wrote the original decorators, but using the approach I used with the SyncContext version. I don’t want to rehash it again, here are the main points:

• Instead of using a property to retrieve the dispatcher, I get it via Application.Current.MainWindow.Dispatcher (checking to be sure Current and MainWindow aren’t null…err, None). This way, I pick up the dispatcher automatically rather than forcing a specific interface on the class with decorated methods. In fact, this approach should work with pure functions as well.
• Since I don’t have a convenient function like SetSynchronizationContext, I store the dispatcher in thread local storage for later use in calling back to the UI thread.
• Unlike the SyncContext version, this version propagates the return value of @UIThread decorated functions. I don’t propagate the return value of @BGThread functions – there’d be no point farming a task to a background thread then blocking the UI thread waiting for a response.

As usual, the code is on my SkyDrive. It includes both the SyncContext and Dispatcher version of the decorators.

Introducing IronPython Article

FYI, my Introducing IronPython article from the .NET Languages issue CoDe magazine is now available online in it’s entirety. Previously, only the introduction was available online. And while we’re on the subject, major thanks to the folks who at the CoDe magazine booth at PDC, who gave me several copies of that issue.

IronPython and WPF Background Processing Revisited

Yesterday, I blogged about using decorators to indicate if a given function should execute on the UI or background thread. While the solution works, I wrote “I’m thinking there might be a way to use SynchronizationContext to marshal it automatically, but I haven’t tried to figure that out yet.” I had some time this morning so I figured out how to use SynchronizationContext instead of the WPF dispatcher.

Leslie Sanford wrote a pretty good overview, but the short version is that SyncContext is an abstraction for concurrency management. It lets you write code that is ignorant of specific synchronization mechanisms in concurrency-aware managed frameworks like WinForms and WPF. For example, while my previous version worked fine, it was specific to WPF. If I wanted to provide similar functionality that worked with WinForms, I’d have to rewrite my decorators to use Control.Invoke. But if I port them over to use SyncContext, they would work with WinForms, WPF and any other library that plugs into SyncContext.

SyncContext abstracts away both initially obtaining the sync context as well as marshaling calls back to the UI thread. SyncContext provides a static property to access  current context, instead of a framework specific mechanism like accessing the Dispatcher property of the WPF Window class. Once you have a context, you can call Send or Post to marshal the call back to the UI thread (Send blocks the calling thread, Post doesn’t).

With that in mind, here’s the new version of BGThread and UIThread. Slightly more complex, but still pretty simple clocking in at just under 30 lines.

In the BGThread wrapper, I add the current SyncContext to the parameter tuple that I pass to the background thread. Once on the background thread, I set the current SyncContext to the last element of the the parameter tuple then call the decorated function with the remaining parameters. (for the non pythonic: args[:-1] is Python slicing syntax that means “all but the last element of args”). Using a try/finally block is probably overkill – I expect the current SyncContext to be either None or leftover garbage – but the urge to clean up after myself is apparently much stronger on the background thread than it is in say my office. :)

In the UIThread wrapper, I grab the current context and invoke the decorated method via the Send method. Like QueueUserWorkItem, SyncContext Send and Post only support a single parameter, so I use the same *args trick I described in my last post. (I changed the name to unpack in the code above for blog formatting purposes)

One major caveat about this approach is that there’s no way to return a value from a function decorated as UIThread. I understand why SyncContext.Post doesn’t return a value (it’s async) but SyncContext.Send is synchronous call, so why doesn’t it marshal the return value back to the calling thread? WPF’s Dispatcher.Invoke and WinForm’s Control.Invoke both return a value. I didn’t handle the return value in my original version of UIThread, but now that I’ve moved over to using SyncContext, I can’t. Not sure why the SyncContext is designed that way – seems like a design flaw to me. Since the return value won’t propagate, I sniff the result decorated function’s return value and raise a warning if it’s not None.

I’ve uploaded the SyncContext version to my SkyDrive in case you want the code for yourself. Note, I’ll thinking I’ll revise code this one more time – I want to rebuild the WPF version so that it propagates return values and picks up an dispatcher via Application.Current.MainWindow rather than having to have a dispatcher property on my class.

IronPython and WPF Part 4: Background Processing

Like many apps today, my WL Spaces photo viewer is a connected app. The various WL Spaces RSS feeds that drive the app can take a several seconds to download. Unless you like annoying your users, it’s a bad idea to lock up your user interface while you make you make synchronous network calls on your UI thread. Typically, this long running processing gets farmed out to a background thread which keeps the UI thread free to service the user events.

.NET provides a variety of mechanisms for doing long running processing on a background thread. For example you can create a new thread, you can queue a work item to the ThreadPoool or use the BackgroundWorker component. However, none of these are particularly pythonic, so I set out to see if I could leverage any of Python’s unique capabilities to make background processing as easy as possible. This is what I ended up with:

By using the cool decorators feature of Python, I’m able to declaratively indicate whether I want a given method to be executed on the UI thread or on a background thread. Doesn’t get much easier than that. Even better, the implementations of BGThread and UIThread are only about twenty lines of Python code combined!

Decorators kinda look like custom .NET attributes. However, where .NET attributes are passive (you have to ask for them explicitly), decorators act as an active modifier to the functions they are attached to. In that respect, they’re kind of like aspects. Certainly, I would consider which thread a given method executes on to be a cross-cutting concern.

The Completed function above is exactly the same as if I had written the following:

In C#, you can’t pass a function as a parameter to another function – you have to first wrap that function in a delegate. Python, like F#, directly supports higher-order functions. This lets you easily factor common aspectual code out into reusable functions then compose them with your business logic. The decorators have no knowledge of the functions they are attached to and the code that calls those functions are written in complete ignorance of the decorators. Python goes the extra mile beyond even F# by providing the ‘@’ syntax.

Here are the implementations of my the UIThread and BGThread decorators:

BGThread defines a wrapper function that queues a call to the decorated function to the .NET thread pool.  UIThread defines a wrapper that marshals the call to the UI thread by using a WPF Dispatcher. I’m thinking there might be a way to use SynchronizationContext to marshal it automatically, but I haven’t tried to figure that out yet. The above approach does require a dispatcher property hanging off the class, but that’s fairly trivial to implement and seems like a small price to pay to get declarative background thread processing.

A couple of quick implementation notes:

• The ‘*args’ syntax used in those methods above means “given me the rest of the positional arguments in a tuple”. Kinda like the C# params keyword. But that syntax also lets you pass a tuple of parameters to a function, and have them broken out into individual parameters. QueueUserWorkItem only supports passing a single object into the queued function, so I pass the tupled arguments to the argUnpacker method, which in turn untuples the arguments and calls the decorated function.
• The System assembly includes the single parameter Action<T> delegate. The current DLR provides Action delegates with zero, two and up to sixteen parameters. However, those are in a separate namespace (remember?) and IPy seems to have an issue with importing overloaded type names into the current scope. I could have used their namespace scoped name, but instead I redefined the version from System to be called Action1.
• To interop with .NET generic types, IPy uses the legal but rarely used Python syntax type[typeparam]. For example, to create a List of strings, you would say “List[str]()”. The type parameter is a tuple, so in UIThread I build a tuple of objects based on the number of arguments passed into wrapper (with the special case of a single type parameter using Action1 instead of Action).

I haven’t uploaded my WL Spaces Photo Viewer app because I keep making changes to it as I write this blog post series. However, for this post I built a simple demo app so I could focus on just the threading scenario. I’ve stuck the code for that demo up on my SkyDrive, so feel free to leverage it as you need.

IronPython and WPF Part 3: Data Binding

Here’s the short version of this post: data binding in WPF to IPy objects just works...mostly. However, I’m guessing you are much more interested in the long version.

Typically, data binding depends on reflection. For example, the following snippet of XAML defines a data bound list box where the title property of each object in the bound collection gets bound to the text property of a text block control. WPF would typically find the title property of the bound objects via reflection.

The problem is that IronPython objects don’t support reflection – or more accurately, reflection won’t give you the answer you’re expecting. Every IPy object does have a static type, but it implements Python’s dynamic type model. [1] Thus, if you reflect on the IPy object looking for the title property or field, you won’t find it. It might seem we’re in a bit of a bind (pun intended). However, WPF does provide an out:

“You can bind to public properties, sub-properties, as well as indexers of any common language runtime (CLR) object. The binding engine uses CLR reflection to get the values of the properties. Alternatively, objects that implement ICustomTypeDescriptor or have a registered TypeDescriptionProvider also work with the binding engine.”
WPF Binding Sources Overview, MSDN Library

Luckily for us, IronPython objects implement ICustomTypeDescriptor [2]. That snippet of XAML above? It’s straight from my photo viewing app. All I had to do was define the data template in the list box XAML then set the ItemsSource property of the list box instance.

As I said, it just works. However, I did hit one small snag – hence the “mostly” caveat above.

If you look at the top level WL Spaces photos feed, you’ll see that each item’s title starts with “Photo Album:”. Yet in the screenshot of my app, you’ll notice that I’ve stripped that redundant text out of the title. Typically, if you want to change the bound value during the binding process, you build an IValueConverter class. I needed two value conversions in my app, stripping “Photo Album:” for the album list box and converting a string URL into a BitmapImage for the image list box.

IronPython objects can inherit from a .NET interface, so there’s no problem building an IValueConverter. However, in order to use a custom IValueConverter from XAML, you need to declare it in XAML as a static resource. However, as you might imagine, dynamic IPy objects don’t work as static resources. So while I can define an IValueConverter in Python, I can’t create one from XAML.

There are a few possible solutions to this. The first is to build up the data template in code. If you do that, they you can programmatically add the converter to the binding. I was hopeful that I could define the data template in XAML then manipulate the binding, but there doesn’t appear to be any way to do that. Another option would be to build some type of generic IValueConverter class in C# that loads either an IPy based IValueConverter or embedded python conversion code. That’s problematic because those IPy object would need to be created in the right ScriptRuntime, and there’s no built-in way to access that. There are also a small set of XamlReader extensions such as XamlTypeMapper that might be able to provide the right hook into the XAML parsing to allow IronPython based conversion.

In the end, I took the easiest way out – I transformed the data to be bound before binding it. It’s cheating of sorts, but given the read-only nature of this app, it was the easiest thing to do. So the actual line of code to set listbox1’s ItemsSource looks like this:

I create a Python class for each RSS item in the feed, saving the stripped title and the album RSS URL as fields. It’s kinda annoying to basically be parsing the feed twice, but at least it’s not much code. Python’s list comprehension syntax makes creating a list of Albums from a list of RSS items a single line of code. I do something very similar for data binding the second list box:

Here I’m not only converting the raw data (adding “:thumbnail” at the end of the URL) but also changing the data type from string to BitmapImage. I’m binding to an image object in the second list box, but to do that I need a BitmapImage instead of a string.

This “convert the data first” approach feels like a hack to me. After I get this series of posts done, I am planning on going back and improving this sample. Hopefully, I can find a better approach to value conversions. Any gurus out there on XAML parsing, please feel free to drop me a line or leave me a comment.

[1] you can access the underlying CLR type for any Python type via the clr.GetClrType method. You an also check out the CreateNewType method from NewTypeMaker.cs

[2] I spent the better part of an afternoon trying to make TypeDescriptionProviders work before Dino pointed out that we already support ICustomTypeDescriptor in Python objects. I didn’t realize at first because I had a case sensitivity bug in my original prototype code - it turns out that “Title” != “title”.

IronPython and WPF Part 2: Loading XAML

If we’re going to build a WPF app, we’re going to want to be able to load some XAML. Sure, you can programmatically build up your UI, but WPF and more importantly WPF tools like Expression Blend are designed to work with XAML. Luckily, loading XAML is fairly easy:

We simply open the filename provided and use XamlReader to build out the corresponding WPF object graph. Note, this is very different from the XAML approach used by C#/VB or even by IronPythonStudio. In those scenarios, the XAML is compiled into a binary format (BAML) and embedded in the compiled assembly. For my TechieWife Photo viewer, it’s all script so there’s neither a XAML to BAML compile step nor a compiled assembly to embed the BAML into, so we’re just loading raw XAML.

Since we’re using raw XAML, there are additional rules we need to follow. First, when using compiled XAML, we can specify the name of the event handler in the XAML directly. For XamlReader, that’s no allowed since there’s no C#/VB class associated with the XAML. Speaking of class, you can’t specify x:Class either. Finally, anywhere you want to use a static resource, as far as I can tell those need to be compiled in a static language. I think you could build one in C#, add a reference to that assembly via clr.AddReference, then use it from XAML and it should just work. However, since I’m trying to stick to IronPython exclusively, I didn’t try that scenario out. 

Since you can’t specify the event handlers in XAML loaded by XamlReader, you have to bind the event handlers in code. There are two listboxes in my photo viewing app, and I want to capture the SelectionChanged event of both of them. Binding event handlers in IronPython code uses the same += syntax as C# uses.

My win1.xaml file has a Window type instance as the root. You don’t need to be a deep WPF expert to realize that the WPF Window doesn’t have listbox1 or listbox2 properties. Yet, in the code snippet above, I was able to say win1.listbox1 and get back the WPF ListBox element with that name. Cool trick, eh? Well, I can’t take credit for it – I copied the code from our Silverlight integration for dynamic languages. Unfortunately, this code has to be written in C# code, but it is the only C# code in my whole solution (and it’s reusable!)

GetBoundMember is kinda like Python’s __getattr__ or Ruby’s method_missing. Of course, it doesn’t work with C#, but it does lets us trap dynamic member resolution when calling a C# object from a DLR language. Srivatsn has a great write up on using GetBoundMember and the four other special methods you can use to make your CLR objects act more dynamic.

In this case, if the standard reflection-based member name resolution fails, we try calling FrameworkElement’s FindName method to see if there’s a corresponding control with the provided name. So win.listbox1 is the equivalent to win.FindName(‘listbox1’), but with less code and a much more pythonic feel.

You’ll notice that we’re attaching this GetBoundMember method to FrameworkElement as an extension method. It’s kinda cool that we can inject a new method into an existing class to provides dynamic behavior and it all works seamlessly from Python. However, DLR uses a different mechanism to locate and bind extension methods than C# or VB. Those languages use ExtensionAttribute to mark extension methods and the assemblies and classes that contain them. However, that approach forces you to examine ever single class in marked assemblies and every single method in marked classes. Examining every class and method is no big deal to do at compile time, but it would be a significant perf issue at runtime. By using ExtensionType attribute, the DLR only has to look at assembly attributes in order to bind extension methods.

Once you’ve got the compiled FrameworkElementExtension assembly, you just need to load it via clr.AddReference. I called the assembly Devhawk.Scripting.Wpf and I load it automatically in my wpy.py module. So if you’re building a WPF app in IronPython, you can simply “import wpy” and you get the GetBoundMember extension method, the LoadXaml function, and a bunch of WPF related namespaces imported into the wpf scope. That way, you can write wpf.Button() instead of System.Windows.Control.Button() to programmatically create a new button.

IronPython and WPF Part 1: Introduction

I decided to start my IronPython and “veritable universe of cool technologies” examples with WPF. I figured that since we already have Silverlight support, there might be some overlap (there was). Futhermore, after seeing BabySmash on Surface I’m jonesing to build a Surface app of my own. Getting vanilla WPF working with IPy seems like a smart step before trying to build a Surface WPF app with IPy.

WPF is all about cool graphics, so I decided to build a photo viewing app. Kinda boring, I know. But it turns out my wife has posted hundreds of photos to her WL Space, and WL Spaces provides convenient RSS feeds of both photo albums as well as photos in specific albums. So I built out a simple WPF based photo viewer for my wife’s WL Space photos in IronPython.

As you can see, I'm not quitting my job to go pursue a career in design anytime soon. But hey, the point is demonstrate building a WPF app in IPy, not to be a great designer. Plus, don’t those cute kids make up for the ugliness of the app?

Turns out building this app in IPy was fairly straightforward, with a few pitfalls. I wasted half a day digging thru data binding before realized that data binding against IPy objects works out of the box - but only if you type the case of the property correctly (Title != title). Also, I couldn’t make TypeConverters work the way I wanted, but python list comprehensions made it enough to transform the feed data before binding it to the UI. That approach worked great for this scenario but maybe not so much for others. (I’ve got feelers out to the WPF data binding wonks, so maybe there’s still hope for type converters)

Over the next several posts, I’m going to show you all the code for this app. It’s pretty small, only about 50 lines of app-specific python code + 50 lines of XAML to describe the window. There’s also some reusable code – 50 lines of WPF module code (mostly stolen from avalon.py in the IPy tutorial), 200 lines of xml2py code which I’ve discussed before and a very small C# based assembly to make accessing WPF elements by name very pythonic.

IronPython and <Insert MSFT Technology Here>

Now that PDC08 is in my rear view mirror, I’m back to doing IronPython stuff. One of the things I’m looking at is making IronPython work with a variety of Microsoft technologies. Given the usage of dynamic languages in web scenarios, most of our focus to date has been on using Iron languages in Silverlight. Being able to program the browser with the same language you program the server is a fairly compelling scenario. We’re also starting to see new progress on ASP.NET support for Iron languages.

But those are only two out of a veritable universe of cool technologies. Now that I’m done with PDC, I can start to explore some of the others. Some ideas include:

• IPy and WPF
• IPy and Surface
• IPy and XNA (desktop only – Xbox and Zune use the Compact Framework with doesn’t support DLR)
• IPy and WCF
• IPy and WF

Any other suggestions? Please leave them in the comments.

My.Change.gov Idea

The words everyone is using to describe President-Elect Obama’s transition is “hit the ground running.” Of all the aspects of the transition, I wanted to call out Change.gov, the new website dedicated to the transition. There’s not much up there yet: Obama’s election-night Grant Park speech, Obama’s first press conference and first radio address as president-elect, the announcement that Rahm Emanuel will be Chief of Staff and transition co-chair Valerie Jarrett’s MTP appearance. Of course, it has been less than a week since the election – there’s only so much that could get done in that time, even when you hit the ground running.

Most interesting about the site is how it invites average American citizens to participate. Maybe not surprising, given the Politics 2.0 campaign Obama ran, but it’s nice to see an administration that’s interested in my story about the election, my vision for what America can be and my ideas about solving our biggest challenges. Obviously, it remains to be seen how an Obama administration will use this information, but there’s little reason to ask for everyone’s opinions if you’re not really interested in reading or acting on them.

The one criticism I’d make about the site is that it looks like these stories, visions and ideas aren’t publicly published to the site. Obama’s campaign site had My.BarackObama (which will “live on”) but there’s no corollary My.Change.gov – at least, not yet. On the other hand, with only 71 days until we drop “elect” from “president-elect”, maybe it makes more sense to hold off on deploying that kind of infrastructure to a more permanent home. Personally, I’m rooting for My.WhiteHouse.gov. My.USA.gov, My.House.Gov and My.Senate.Gov would also be nice to have. Since Change.gov doesn’t publicly publish stories, visions or ideas, I wanted to post my idea here as well.

(Side note, is it just me or is http://www.usa.gov/ a pretty crappy home page?)

Obviously, there are immediate economic issues to deal with and we need to bring the troops home from Iraq. But after those, I would like to see the Obama administration put renewable energy at the top of it’s priority list. I believe that investing in renewable energy as well as the infrastructure to deliver that energy (the so called “smart grid”) should be our top priority because I believe it provides the most bang for the buck when it comes policy.

First off, it will create jobs which is probably the most critical step for turning the economy around. We’ve lost 1.2 million jobs in the past year and unemployment is at a 14 year high.  Second, I’m a big believer in Obama’s long-stated goal of eliminating our dependency on foreign oil. This would have the dual benefit of improving our own economy while reducing the economic strength of petro-dictatorships, some of which sponsor terrorism. Third, shifting to clean renewable energy will reduce our carbon emissions. We need to do more on the climate change front, but it’s a good start. Finally, a smart grid would essentially be a new nationwide market for energy. Even though many Republicans are in the pocket of big oil, they are also big believers in free markets and individual responsibility, so maybe smart grid legislation can be passed in a bipartisan way rather than over the Republican’s wishes.

There seems to be a lot of indication already that energy is Obama’s top priority after an economic stimulus package, so maybe he doesn’t need to hear it from me. But that doesn’t change the fact that the soon-to-be President is continuing to embrace to concepts of Politics 2.0 in an effort to bring Americans together to solve our toughest challenges. Here’s hoping he keeps it up long after “elect” is dropped from his title.

My Elevate-Process Script

I used to use the Script Elevation PowerToys to provide a simple way to launch an elevated command window from a Powershell prompt. However, that required installing said PowerToys in order to work, which I invariably forgot to install when paving my machine. That got annoying, so I went in search of a pure Powershell solution, which Peter Provost conveniently provided on his blog.

However, one of the benefits of the Script Elevation PowerToys was the ability to launch an admin command prompt in a specific directory – including the current one. I wanted the ability to default to launching Powershell when the user doesn’t specify a command to run. I thought I could just set $psi.WorkingDirectory, but as I’ve described previously, I update $home in my profile script to D:\HPierson.Files (I keep my important files on my D: drive so I can pave C: with impunity) then set my current location to $home. So I can’t set the current location by using $psi.WorkingDirectory – it just gets overridden by my profile script.

However, it turns out you can pass arbitrary script code to Powershell via the –Command arguments. You also have to pass –NoExit to keep the command window around. The script passed in via -Command is executed after the profile script, so I can pass in a little script code to set the current location to the right location.

I modified Peter’s elevate-process script to launch a new Powershell command window when zero arguments or one folder argument are passed in. In those cases, elevate-process sets the location to the specified directory (current directory as the default when no arguments are provided) via the –NoExit and –Command arguments.

I’ve posted the script to my SkyDrive as well as providing it below. Feel free to steal use it as you need.

Update: I tried to run my elevate-process script from c:\Program Files and discovered a bug. The set-location scripts need the path parameter to be encapsulated in single quotes in order to handle paths with spaces. I’ve updated both the code above as well as the copy on my SkyDrive.

Election Predictions

Is it to early to start making predictions about 2010, 2012 and beyond yet? Probably, but I’ve got a few thoughts anyway.

First, if Obama runs his presidency like he ran his candidacy, he’ll be a great president and will win reelection in a cakewalk. Obviously, there are *huge* challenges ahead, but I have faith his administration will be able to handle them.

Second, assuming prediction #1 pans out, we’ll see another wide open race in 2016. I’m a big Joe Biden fan, but he’ll be 74 in 2016 – two years older than McCain is now. No way he runs. Hillary Clinton will 69 so I don’t see her running either. Well, maybe they run but I doubt they’d win the nomination. I’m guessing the Democratic nominee in 2016 will be someone who’s not top of mind today – someone who emerges in the next two or four years as “the next Obama”.

Third, we haven’t seen the last of Sarah Palin. I’ve been extremely critical of her, but my libertarian friend recommended I not underestimate her. So I won’t. I think she’ll be the Republican nominee for president in 2016. I know folks are talking about her for 2012, but I think she’d rather take her chances in a wide open race in 2016 than go up against Obama a second time. She’s only 44 now, so she’ll be 52 in 2016 – about perfect for a presidential candidate. Eight years is a long time for her to close the unqualified gap.

In fact, I’d say Ted Stevens apparent reelection is a likely silver lining in otherwise crappy election for the Republicans. Ted Stevens was found guilty on felony corruption charges and faces expulsion from the senate, regardless of his apparent victory yesterday. If he is expelled, there would be a special election to decide his successor. You have to assume Mark Begich, Stevens’ Democratic challenger, would handily win that special election. But what if Palin ran against Begich? She’s taken a hit it her approval ratings since joining the VP ticket, but she’s still very popular in Alaska and probably liked best by Stevens’ supporters. I’m not saying it’s a shoo-in, but I believe she’d have a very good chance to win a special election.

Being a senator for eight years would be long enough to shore up the concerns about her qualifications but short enough not to have much of an attackable voting record. She’d have to win reelection in 2014, but given that Stevens has won seven elections in a row dating back to 1970, I’m guessing getting reelected in 2014 would be no problem.

So my predictions are a fairly easy Obama reelection in 2012, followed by Sen Palin (R) vs. “The Next Obama” (D) in 2016. Midterms in 2010 will probably see Dems give back some of their recent gains, but not enough to lose control of either house of Congress. 2014 is to far out to predict, much less who wins in 2016. But looking at voting trends among minorities, party ID advantages, etc, I like Democrats chances for the foreseeable future.

Finally, I wonder if this is the “end of identity politics” as some have suggested. One election is not a trend, so I tend to think that Obama was immune to such attacks more than those attacks no longer viable. We may need to wait until 2016 to find out. When Palin does run again, will she still be a pitbull with lipstick? I like to think identity politics is toast, but I’m guessing the Republicans will go back to that well a few more times before giving up on it. Democrats, on the other hand, have all the proof they need that they can win against identity politics: President-Elect Obama.

Yes We Can

What a difference four years makes. In 2004, I was up until the wee hours because Ohio wasn’t called until the following morning. This year, with Pennsylvania called at 8pm EST and Ohio about 90 minutes later, it was basically over before my kids finished their dinner. We were watching SNL on the DVR from last night when my Dad called to tell us the nets had called it for Obama.

As I write this, Obama has 349 electoral votes – 79 more than he needed to win – with three states still to be called. It looks like Obama will pick up North Carolina (ahead by 12,160 votes with 100% of precincts reporting). Montana looks to be going for McCain (he leads by 7,000 votes with 77% reporting) and Missouri looks like a true tossup, with McCain ahead by a scant 1,740 votes with 99% reporting. Assuming those leads hold up, that would give Obama 364 EVs to McCain’s 174 EVs. Compared to the last two elections, that’s a good solid win on par with Clinton’s win in 1992 (370-168) but nowhere near the ass-kickings by Regan in 1980 (489-49) or 1984 (525-13) or Bush 41 in 1988 (426-111).

(BTW, check out the electoral college map from Carter’s win in 1976. It looks like Bizarro’s electoral map. The south went all blue while California went red? Can you even conceive of that happening today?)

But even though it wasn’t 500 point blowout, consider that Obama flipped all the following red 2004 states to blue in 2008: Virginia, Florida, Ohio, Iowa, Indiana, Colorado, New Mexico, Nevada and (probably) North Carolina. My parents are very excited to live in a blue state (VA) now. McCain on the other hand flipped none – goose egg – from blue to red. McCain’s only real hope was Pennsylvania and it wasn’t even close – Obama won PA by 9%.

I did like McCain’s concession speech. Obviously, with the president around 25% approval rating, the incumbent party is at a major disadvantage to start with. But of all the Republican candidates, McCain was the only one I was ever worried about. IMO, any of the other Republican possible nominees would have lost by a much worse margin. McCain was attractive to independents in a way no other Republican candidate this year was.

But in the end, McCain had the probably impossible job of pulling together the 25% of Americans who approve of Bush (politely called “the Republican base”, though I call them “wack-jobs”) with another 25% of Americans who don’t. Kinda like trying to push like charged magnets together, they just wouldn’t stick. To me, it seemed like McCain tacked hard right and prayed the independents would still support him. But as I watched McCain’s concession speech, it made me wonder if McCain could have won by running an honorable campaign, tacking to the center and hoping the conservatives would still support him. Maybe they would, maybe they wouldn’t, but he ended up running as a hate-mongering Rovian erratic demagogue and significantly damaged his personal brand. I can barely reconcile McCain from the stump with McCain from the concession speech. I’m guessing he’ll retire instead of running for reelection in two years.

(I’ve also got thoughts on Palin’s political future, but those will wait for another day.)

As for President-Elect Obama, I’m obviously excited than he won. Patrick made me promise to tell him who won in the morning (it wasn’t official until the west coast results came in after he went to bed). I’m going to show him Obama’s victory speech - it was amazing. I especially liked when he said “And to those Americans whose support I have yet to earn - I may not have won your vote, but I hear your voices, I need your help, and I will be your President too.” My libertarian friend who thought it would break for McCain today (way, way wrong dude) immediately went political, pointing out that the Democratic “trifecta” means they can’t shift the blame if things get worse. I’m guessing Obama will have to do quite a bit to earn my friend’s support but I’m hopeful that he can.

In other races I’m following, looks like Gov. Christine Gregoire will win reelection and Darcy Burner is leading in her race to unseat Rep. Reichert (though with only 21% reporting so it’s far from sure thing). In California, Prop 8 to ban gay marriage is currently leading but with only 51% reporting so I’m hoping that changes. (Amending the constitution in CA doesn’t require a super-majority? That sucks).

Finally, a quick shout out to my friends from New Zealand that I hung out with after the Opshop concert at the TechEd Attendee party back in September. They confided in me that “everyone in the world” was pulling for an Obama victory. Based on this Global Electoral College from the Economist magazine, “everyone in the world” is pretty close to spot on. Happy that my fellow Americans and I could deliver an Obama victory. Feels good, doesn’t it?

Election Day 2008

(Editor’s note: almost every time I make a political post, some yahoo, typically anonymously, suggests that I “stick to technology”. I will remind you, dear reader, that This Is Not A Technical Blog. If you don’t like my politics, you’re free to ignore the political posts or simply unsubscribe from my blog altogether. Anonymous suggestions as to what I should or should not be blogging about will be summarily ignored.)

Jules and I voted this morning, before work and school. Patrick helped Julie vote and Riley helped me vote. I’m assuming regular readers don’t need to be told who I voted for. Riley just wanted to help fill in the bubbles because it’s like coloring. Patrick kinda got what was going on – check out Jules’ post about how they cast “our” vote.

For shits and giggles, I went back and looked at my posts from two years ago. It doesn’t make me Nostradamus, but I did guess we were going to have “a very ugly race, especially from the Republicans” and that “the nomination race will be worse” than the general election. If Keith Olbermann or Rachel Maddow want to get in on this insightful political analysis, they know where to reach me. :)

I’ve been obsessed with three polling sites over the past few months:

• FiveThirtyEight.com which is projecting 6.1 point popular vote / 159.2 electoral vote Obama victory
• Pollster.com which is projecting a 7.6 point PV / 149 EV Obama victory (with 105 toss-up EVs)
• Real Clear Politics is projecting a 7.5 point PV / 138 EV Obama victory

My wingnut libertarian friend (anyone from my kids’ playgroup who reads this blog can guess who I’m talking about) thinks McCain will win with “295-305” electoral votes. This election is far from being in the bag for Obama, but the suggestion that McCain will come from six or seven points down in the polls to win a 10-20 point bigger EV victory than Bush did four years ago seems like crazy talk. As for me, I’m not willing to make a firm prediction except to say that I think the polls are wrong – I’m just not sure in what direction they’re wrong.

On the one hand, if the polls are underestimating Obama’s support by excluding cell-phone only voters and Obama’s Get Out The Vote effort is as strong as it looks, that could net Obama as much as five or six points on top of the lead he’s already got and we’ll be looking at an epic landslide with Obama maybe brushing up against 400 EV. For example, Pollster.com’s 291 solid/lean Obama EV + 105 toss up EVs would be 396 total EVs for Obama.

On the other hand, if the polls are overestimating Obama’s support due to the Bradley Effect, the undecideds break for decisively for McCain and Obama’s GOTV effort doesn’t materialize, McCain’s might net those five or six points and turn this into a toss-up squeaker decided by less than 35 EV (about the size of Bush’s victory in 2004). For example, if McCain wins all of Pollster.com’s tossups + the lean Obama states, it’s a scant 8 EV Obama victory (273-265). If McCain could also pick off any of New Mexico, Colorado, Iowa or Pennsylvania in that scenario, then he would win by anywhere from 2-34 EVs.

Of course, those effects could cancel out and leave us where we are right now, Obama with a six to seven point PV and low 100s EV victory. Frankly, I’ll be happy to be wrong about the poll inaccuracy since that would give Obama the presidency.

How do you think things are going to play out today?

I Survived October

It' felt touch and go at times, but I did manage to make it thru PDC prep, PDC itself, shipping IPy 2.0 RC1 and Halloween relatively unscathed. I was Superman for Halloween – the perils of letting the five year old pick everyone’s costume, we we’re *all* Superman (well, Jules and Riley were Supergirl) – but given how exhausted I was, I could have skipped the costume and just gone as a zombie.

My inbox is currently just under 500 messages, I’ve got a mountain of stuff to finish for IPy RTM, I’m presenting at the p&p Summit and I’m finally get some attention from the legal dept (now that we’re past PDC). In other words, it’s not the end of PDC is ushering in a golden age of zero work for Harry. But with PDC in my rear view mirror, I don’t feel quite so overwhelmed as I did.

And on top of all this, I’ve been borderline obsessed with election news. I’ve basically given up on all my non-political blog reading – if I hadn’t been working on PDC I might not have even been aware of the big announcements like Oslo and Azure. Things are looking good for Obama and the Democrats, but as this hilarious video shows “Being in a good position to win is not the same as winning.” I haven’t had the time to volunteer for Obama but my father has been volunteering for Obama in Northern Virginia – aka “communist country”. It sure would be nice to see my home state go blue.