The Reese’s Peanut Butter Cups of Language Conferences

It’s been about 11 months since the last Lang.NET Symposium and we’ve been working on next year’s version. But then we discovered that we’re not the only ones inside Microsoft thinking about having a language conference. A couple of meetings later, and we’ve decided to combine them, which will assuredly lead to side conversations like this:

DSL DevCon Attendee: Hey, you got your compiler in my DSL!

Lang.NET Attendee: Hey, you got your DSL in my compiler!

[They sample the combined content]

Both Attendees: Mmmm, two great conferences that work well together!

Well, maybe not. But seriously, it should be a great combined conference.

However, there are some logistics things we need to work out, like how many days should the combined conference run? We figure the “right” answer to these questions depends on the likely overlap between the two conferences. Frankly, we don’t know what the overlap will be so we decided to simply, you know, ask.

If you are interested in attending Lang.NET, the DSL DevCon, or both next year, please head over to Chris Sells blog and make your voice heard.

PowerShell find-to-set-alias

I use Live Mesh to keep my PowerShell scripts folder synced between multiple machine. Some of those machines have different things installed on them or have things installed in different locations. For example, my laptop is x86 while my desktop is x64 so many things on the desktop get installed into c:\Program Files (x86) instead of the plain-old c:\Program Files folder. I wanted my shared profile script to be able to search a set of folders for a given executable to alias, and I came up with the following function.

function find-to-set-alias($foldersearch, $file, $alias)
{
  dir $foldersearch |
    %{dir $_ -Recurse -Filter $file} |
    %{set-alias $alias $_.FullName -scope Global; break}
}

It’s pretty simple to use. You pass in a folder search criteria – it must have a wildcard, or the function won’t work – the file you’re looking for and the alias you want to set. The function finds all the folders matching the $foldersearch criteria, then searches them recursively looking for the $file you specified. Set-alias is called for the first matching $file found – pipeline processing is halted via the break statement.

Here are the find-to-set-aliases I have in my profile:

find-to-set-alias 'c:\program files*\IronPython*' ipy.exe ipy
find-to-set-alias 'c:\program files*\IronPython*' chiron.exe chiron

find-to-set-alias 'c:\Python*' python.exe cpy

find-to-set-alias
    'c:program files*\Microsoft Visual Studio 9.0\Common7' devenv.exe vs
find-to-set-alias
    'c:program files*\Microsoft Visual Studio 9.0\Common7' tf.exe tf

find-to-set-alias 'c:\program files*\FSharp*' fsi.exe fsi
find-to-set-alias
    'c:\program files*\Microsoft Repository SDK*' ipad.exe ipad
find-to-set-alias
    'c:\program files*\Microsoft Virtual PC*' 'Virtual pc.exe' vpc

Python, IronPython and F# aliases, no surprise there. Chiron is the REPL server for dynamic language Silverlight development. Typically, I use Chris Tavares’ vsvars script to configure the command shell for development purposes, but I find it’s nice to have aliases for TF and DevEnv handy at all times.

IronPython and LiveFX: Raw HTTP Access

One of the cool things about the Live Framework is that while there’s a convenient .NET library available, you can use the raw HTTP interface from any platform. LiveFX data is served up over HTTP and is available in ATOM, RSS, JSON or POX formats. As I’ve already shown, you can easily use the .NET library from IronPython, but I wanted to try working with the raw HTTP interface to get a feel for that as well.

Unfortunately, it was harder than I expected it to be. The big issue is that the documentation on how to LiveFX authorization tokens via raw HTTP is fairly sparse and occasionally contradictory. For example, there’s a whole section on Authentication and Live Framework, but it doesn’t cover this scenario. Luckily, I was able to figure it out with the help of AtomPub Project Manager LiveFX Sample, a post on Alex Feinman’s blog, a post on Emmanuel Mesas’ blog and a little groveling around with Reflector. It does appear that the auth docs are in flux –Emmanuel refers to this MSDN article as being about RPS Soap requests, but it’s actually about delegated authority. (Is MSDN reusing URLs? Bad idea.) Also, the sample code has a comment that reads “to be replaced by delegated authorization” so it looks like changes are coming. In other words, no promises on how long this code will work!

If you look at the AtomPub Project Manager sample, there’s a WindowsLiveIdentity.cs file that implements static GetTicket method that looks similar to both the code on Alex’s blog as well as the implementation of GetWindowsLiveAuthenticationToken. The upshot is that there’s a WS-Trust endpoint for Windows Live at https://dev.login.live.com/wstlogin.srf. You send it a RequestSecurityToken (aka RST) message (with a couple of extra WL specific extensions) and it responds with the security token you’ll need for accessing the LiveFx HTTP endpoints.

I ported the GetTicket function over to IronPython. I’m using .NET classes like WebRequest and XmlReader, but there’s nothing fancy here so I would expect it to be easy enough to port over to the standard Python library.

def get_WL_ticket(username, password, compactTicket):
    req = WebRequest.Create(_LoginEndPoint)
    req.Method = "POST"
    req.ContentType = "application/soap+xml; charset=UTF-8"
    req.Timeout = 30 * 10000

    rst = get_RST_message(username, password, compactTicket)
    rstbytes = Encoding.UTF8.GetBytes(rst)
    with req.GetRequestStream() as reqstm:
      reqstm.Write(rstbytes, 0, rstbytes.Length)

    with req.GetResponse() as resp:
      with resp.GetResponseStream() as respstm:
        with XmlReader.Create(respstm) as reader:
          if compactTicket:
            name = "BinarySecurityToken"
            namespace = "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
          else:
            name = "RequestedSecurityToken"
            namespace = "http://schemas.xmlsoap.org/ws/2005/02/trust"

          if not reader.ReadToDescendant(name, namespace):
            raise "couldn't find security token element"

          reader.ReadStartElement(name, namespace)
          token = reader.ReadContentAsString()
          reader.ReadEndElement()

          return Convert.ToBase64String(Encoding.UTF8.GetBytes(token))

This code simply uses a WebRequest object to post the RST message to the WS-Trust enpoint then parses the result to find the token. get_RST_message uses standard Python string formatting to generate the RST message that gets posted to the WS-Trust endpoint. I’m not exactly sure why you need to convert the token value to a byte array and then Base64 encode it, but that’s what the sample code does so I did it to.

Once you have the authentication ticket, you need to download root service endpoint document in order to get the base URL and the profiles link. Then you can download all the profiles or you can download a specific one if you know it’s leet-speak identifier. LiveFX data can be downloaded in a variety of formats: ATOM, JSON, RSS or POX. You choose your format by setting the Accept and Content-Type headers.

I wrote the following functions, the generic boilerplate download function as well a specific versions for downloading JSON and POX:

def download(url, contentType, authToken):
  req = WebRequest.Create(url)
  req.Accept = contentType
  req.ContentType = contentType
  req.Headers.Add(HttpRequestHeader.Authorization, authToken)

  return req.GetResponse()  

def download_json(url, authToken):
  resp = download(url, 'application/json', authToken)
  with StreamReader(resp.GetResponseStream()) as reader:  
      data = reader.ReadToEnd()
      return eval(data)

def download_pox(url, authToken):
  resp = download(url, 'text/xml', authToken)
  return XmlReader.Create(resp.GetResponseStream())

Using JSON in Python is really easy, since I can simply eval the returned string and get back Python dictionary objects, similar to what you can do in Javascript.

Here’s some code that uses the get_WL_ticket and download_json functions above to retrieve the the user’s Personal Status Message

#Get user's WL ticket

uid = raw_input("enter WL ID: ")
pwd = raw_input("enter password: ")

authToken = livefx_http.get_WL_ticket(uid, pwd, True)

#download root service document

service = livefx_http.download_json(_LiveFxUri, authToken)

#download general profile document

url = service['BaseUri'] + service['ProfilesLink'] + "/G3N3RaL"

genprofile = livefx_http.download_json(url, authToken)
print genprofile['ProfileBase']['PersonalStatusMessage']

POX is also fairly easy, though a bit more verbose than JSON. The sample code, which I have stuck on my SkyDrive, includes both POX and JSON code, so you can compare and contrast the differences.

IronPython and LiveFX: Ori’s LiveOE.py

Ori Amiga is a Group Program Manager over in the Live Framework team whom you might have seen at PDC08 delivering the Lap Around LiveFX & Mesh Services and LiveFX Programming Model Architecture and Insights talks. And apparently, he’s an IronPython fan as posted a small LiveFX Python module to his blog. It’s pretty simple – it only wraps Connect and ConnectLocal – but it does cut about ten lines of path appending, reference adding and module importing code into a single import statement. Here’s the profile access script from my last post rewritten to use Ori’s LiveOE module.

import LiveOE     
from devhawk import linq

uid = raw_input("Enter Windows Live ID: ")
pwd = raw_input("Enter Password: ")

loe = LiveOE.Connect(uid, pwd)

general = linq.Single(loe.Profiles.Entries,  
  lambda e: e.Resource.Type == LiveOE.ProfileResource.ProfileType.General)

print loe.Mesh.ProvisionedUser.Name
print loe.Mesh.ProvisionedUser.Email
print general.Resource.ProfileInfo.PersonalStatusMessage
print linq.Count(loe.Contacts.Entries)

FYI, make sure you update the sdkLibsPath in LiveOE.py – I’m not sure where Ori has installed the LiveFX SDK, but it’s *not* in the location suggested by the read me file.

BTW, it turns out the WL Profile information is read only which answers a question I had. However, reading the thread it sounds like they will eventually get around to making it read-write at some point.

IronPython and LiveFX: Accessing Profiles

I recently got access to both the Windows Azure and Live Framework CTP programs. Frankly, I’m very interested in Live Mesh, so I decided to start with a simple LiveFX program. Scott (aka ScottIsAFool) at LiveSide posted a “quick and dirty” console app that pulls info from a user’s profile via LiveFx. It’s not Mesh per se, but it does use the same framework and resource model so I decided to port it to IronPython. FYI, this app won’t run unless you’ve been received a LiveFx CTP token and provisioned yourself.

#Add LiveFX References

import sys
sys.path.append('C:\Program Files\Microsoft SDKs\Live Framework SDK\v0.9\Libraries\.Net Library')

import clr
clr.AddReference('Microsoft.LiveFX.Client')
clr.AddReference('Microsoft.LiveFX.ResourceModel')

from Microsoft.LiveFX.Client import LiveOperatingEnvironment
from Microsoft.LiveFX.ResourceModel.ProfileResource import ProfileType
from System.Net import NetworkCredential

from devhawk import linq

#get username and password from the user

uid = raw_input("Enter Windows Live ID: ")
pwd = raw_input("Enter Password: ")
creds = NetworkCredential(uid, pwd, "https://user-ctp.windows.net")

#print out user's info

loe = LiveOperatingEnvironment()
loe.Connect(creds)

general = linq.Single(loe.Profiles.Entries,  
  lambda e: e.Resource.Type == ProfileType.General)

print loe.Mesh.ProvisionedUser.Name     
print loe.Mesh.ProvisionedUser.Email
print general.Resource.ProfileInfo.PersonalStatusMessage
print linq.Count(loe.Contacts.Entries)

I did modify the app slightly, reading the WLID and password off the console – I was sure I would accidently post my personal credentials if I left them embedded in the app. Otherwise, it’s a straight port. First, I add references the LiveFX dlls. Since they’re not local to my script, I add the directory where they’re installed to sys.path, which lets me call clr.AddReference directly. Then I retrieve the user’s ID and password using raw_input (Python’s equivalent to Console.ReadLine). Finally, I connect to the user’s LiveOperatingEnvironment and pull their name, email address, personal status message and the number of contacts they have.

As per the original app, I use LINQ to find the right profile as well as count the number of contacts. I was able to reuse the linq.py file I wrote for my Rock Band song list screen scraper (though I did have to add the Count function since I hadn’t needed it previously). I’ve posted this script on my SkyDrive, and it includes my most recent linq.py file.

BTW, it doesn’t appear that you can set the PersonalStatusMessage programmatically, at least not currently. I was thinking it would be cool to build an app that sets your PSM via Twitter, but the set method of PersonalStatusMessage is marked internal. In fact, all the set methods of all the profile properties I looked at are marked internal. If someone knows how to update LiveFX resource objects in the current CTP, I’d appreciate it if you dropped me a line or left me a comment.