IronPython’s biggest customer is Resolver Systems, makers of Resolver One, “a familiar spreadsheet-like interface with the powerful Python programming language to give you a tool to analyse and present your data.” While I think they have a great product on pure merit – I’ve been impressed with their product since I was introduced to it at Lang.NET this year – I’m particularly interested in Resolver One as it’s written in IronPython. They use IPy not only as the embedded language exposed to end users but as the underlying implementation language as well.

Furthermore, these guys are heavily involved in the IPy community. Resolver developer Michael Foord is writing a book on IronPython and was our first Dynamic Language MVP. Michael’s Resolver colleague Jonathan Hartley did me a solid by taking my space at ØreDev. Even the CTO and Co-founder Giles Thomas is a regular blogger and speaker at events. Let me tell you, having guys this great in the community sure makes my job easier.

I just wanted to give the Resolver folks a shout out and say major congratulations on shipping a new version of their core Resolver One product. Michael has more info on this release as well as a glance forward with their plans for their next (IPy 2.0, woot!)

OK, here’s the last word on this whole background processing / concurrency decorators thing. I went back and re-wrote the original decorators, but using the approach I used with the SyncContext version. I don’t want to rehash it again, here are the main points:

• Instead of using a property to retrieve the dispatcher, I get it via Application.Current.MainWindow.Dispatcher (checking to be sure Current and MainWindow aren’t null…err, None). This way, I pick up the dispatcher automatically rather than forcing a specific interface on the class with decorated methods. In fact, this approach should work with pure functions as well.
• Since I don’t have a convenient function like SetSynchronizationContext, I store the dispatcher in thread local storage for later use in calling back to the UI thread.
• Unlike the SyncContext version, this version propagates the return value of @UIThread decorated functions. I don’t propagate the return value of @BGThread functions – there’d be no point farming a task to a background thread then blocking the UI thread waiting for a response.

As usual, the code is on my SkyDrive. It includes both the SyncContext and Dispatcher version of the decorators.

FYI, my Introducing IronPython article from the .NET Languages issue CoDe magazine is now available online in it’s entirety. Previously, only the introduction was available online. And while we’re on the subject, major thanks to the folks who at the CoDe magazine booth at PDC, who gave me several copies of that issue.

Yesterday, I blogged about using decorators to indicate if a given function should execute on the UI or background thread. While the solution works, I wrote “I’m thinking there might be a way to use SynchronizationContext to marshal it automatically, but I haven’t tried to figure that out yet.” I had some time this morning so I figured out how to use SynchronizationContext instead of the WPF dispatcher.

Leslie Sanford wrote a pretty good overview, but the short version is that SyncContext is an abstraction for concurrency management. It lets you write code that is ignorant of specific synchronization mechanisms in concurrency-aware managed frameworks like WinForms and WPF. For example, while my previous version worked fine, it was specific to WPF. If I wanted to provide similar functionality that worked with WinForms, I’d have to rewrite my decorators to use Control.Invoke. But if I port them over to use SyncContext, they would work with WinForms, WPF and any other library that plugs into SyncContext.

SyncContext abstracts away both initially obtaining the sync context as well as marshaling calls back to the UI thread. SyncContext provides a static property to access  current context, instead of a framework specific mechanism like accessing the Dispatcher property of the WPF Window class. Once you have a context, you can call Send or Post to marshal the call back to the UI thread (Send blocks the calling thread, Post doesn’t).

With that in mind, here’s the new version of BGThread and UIThread. Slightly more complex, but still pretty simple clocking in at just under 30 lines.

def BGThread(fun):  
  def argUnpacker(args):  
    oldSyncContext = SynchronizationContext.Current
    try:
      SynchronizationContext.SetSynchronizationContext(args[-1])
      fun(*args[:-1])
    finally:
      SynchronizationContext.SetSynchronizationContext(oldSyncContext)

  def wrapper(*args):
    args2 = args + (SynchronizationContext.Current,)
    ThreadPool.QueueUserWorkItem(WaitCallback(argUnpacker), args2)

  return wrapper

def UIThread(fun):
  def unpack(args):  
    ret = fun(*args)
    if ret != None:
      import warnings
      warnings.warn(fun.__name__ + " function returned " + str(ret) + " but that return value isn't propigated to the calling thread")

  def wrapper(*args):
    if SynchronizationContext.Current == None:
      fun(*args)
    else:
      SynchronizationContext.Current.Send(SendOrPostCallback(unpack), args)

  return wrapper

In the BGThread wrapper, I add the current SyncContext to the parameter tuple that I pass to the background thread. Once on the background thread, I set the current SyncContext to the last element of the the parameter tuple then call the decorated function with the remaining parameters. (for the non pythonic: args[:-1] is Python slicing syntax that means “all but the last element of args”). Using a try/finally block is probably overkill – I expect the current SyncContext to be either None or leftover garbage – but the urge to clean up after myself is apparently much stronger on the background thread than it is in say my office. 😄

In the UIThread wrapper, I grab the current context and invoke the decorated method via the Send method. Like QueueUserWorkItem, SyncContext Send and Post only support a single parameter, so I use the same *args trick I described in my last post. (I changed the name to unpack in the code above for blog formatting purposes)

One major caveat about this approach is that there’s no way to return a value from a function decorated as UIThread. I understand why SyncContext.Post doesn’t return a value (it’s async) but SyncContext.Send is synchronous call, so why doesn’t it marshal the return value back to the calling thread? WPF’s Dispatcher.Invoke and WinForm’s Control.Invoke both return a value. I didn’t handle the return value in my original version of UIThread, but now that I’ve moved over to using SyncContext, I can’t. Not sure why the SyncContext is designed that way – seems like a design flaw to me. Since the return value won’t propagate, I sniff the result decorated function’s return value and raise a warning if it’s not None.

I’ve uploaded the SyncContext version to my SkyDrive in case you want the code for yourself. Note, I’ll thinking I’ll revise code this one more time – I want to rebuild the WPF version so that it propagates return values and picks up an dispatcher via Application.Current.MainWindow rather than having to have a dispatcher property on my class.

Like many apps today, my WL Spaces photo viewer is a connected app. The various WL Spaces RSS feeds that drive the app can take a several seconds to download. Unless you like annoying your users, it’s a bad idea to lock up your user interface while you make you make synchronous network calls on your UI thread. Typically, this long running processing gets farmed out to a background thread which keeps the UI thread free to service the user events.

.NET provides a variety of mechanisms for doing long running processing on a background thread. For example you can create a new thread, you can queue a work item to the ThreadPoool or use the BackgroundWorker component. However, none of these are particularly pythonic, so I set out to see if I could leverage any of Python’s unique capabilities to make background processing as easy as possible. This is what I ended up with:

def OnClick(self, sender, args):  
    self.DLButton.IsEnabled = False  
    self.BackgroundTask(self._url.Text)  

@BGThread
def BackgroundTask(self, url):  
    wc = WebClient()
    data = wc.DownloadString(Uri(url))
    self.Completed(data)  

@UIThread  
def Completed(self, data):  
    self.DLButton.IsEnabled = True
    self._text.Text = data

By using the cool decorators feature of Python, I’m able to declaratively indicate whether I want a given method to be executed on the UI thread or on a background thread. Doesn’t get much easier than that. Even better, the implementations of BGThread and UIThread are only about twenty lines of Python code combined!

Decorators kinda look like custom .NET attributes. However, where .NET attributes are passive (you have to ask for them explicitly), decorators act as an active modifier to the functions they are attached to. In that respect, they’re kind of like aspects. Certainly, I would consider which thread a given method executes on to be a cross-cutting concern.

The Completed function above is exactly the same as if I had written the following:

def Completed(self, data):  
    self.DLButton.IsEnabled = True  
    self._text.Text = data  
Completed = UIThread(Completed)

In C#, you can’t pass a function as a parameter to another function – you have to first wrap that function in a delegate. Python, like F#, directly supports higher-order functions. This lets you easily factor common aspectual code out into reusable functions then compose them with your business logic. The decorators have no knowledge of the functions they are attached to and the code that calls those functions are written in complete ignorance of the decorators. Python goes the extra mile beyond even F# by providing the ‘@’ syntax.

Here are the implementations of my the UIThread and BGThread decorators:

def BGThread(fun):  
  def argUnpacker(args):  
    fun(*args)

  def wrapper(*args):  
    ThreadPool.QueueUserWorkItem(WaitCallback(argUnpacker), args)

  return wrapper

def UIThread(fun):
  def wrapper(self, *args):
    if len(args) == 0:
      actiontype = Action1[object]
    else:
      actiontype = Action[tuple(object for x in range(len(args)+1))]

    action = actiontype(fun)
    self.dispatcher.Invoke(action, self, *args)

  return wrapper

BGThread defines a wrapper function that queues a call to the decorated function to the .NET thread pool.  UIThread defines a wrapper that marshals the call to the UI thread by using a WPF Dispatcher. I’m thinking there might be a way to use SynchronizationContext to marshal it automatically, but I haven’t tried to figure that out yet. The above approach does require a dispatcher property hanging off the class, but that’s fairly trivial to implement and seems like a small price to pay to get declarative background thread processing.

A couple of quick implementation notes:

• The ‘*args’ syntax used in those methods above means “given me the rest of the positional arguments in a tuple”. Kinda like the C# params keyword. But that syntax also lets you pass a tuple of parameters to a function, and have them broken out into individual parameters. QueueUserWorkItem only supports passing a single object into the queued function, so I pass the tupled arguments to the argUnpacker method, which in turn untuples the arguments and calls the decorated function.
• The System assembly includes the single parameter Action<T> delegate. The current DLR provides Action delegates with zero, two and up to sixteen parameters. However, those are in a separate namespace (remember?) and IPy seems to have an issue with importing overloaded type names into the current scope. I could have used their namespace scoped name, but instead I redefined the version from System to be called Action1.
• To interop with .NET generic types, IPy uses the legal but rarely used Python syntax type[typeparam]. For example, to create a List of strings, you would say “List[str]()”. The type parameter is a tuple, so in UIThread I build a tuple of objects based on the number of arguments passed into wrapper (with the special case of a single type parameter using Action1 instead of Action).

I haven’t uploaded my WL Spaces Photo Viewer app because I keep making changes to it as I write this blog post series. However, for this post I built a simple demo app so I could focus on just the threading scenario. I’ve stuck the code for that demo up on my SkyDrive, so feel free to leverage it as you need.