The Brilliant Magic of Edge.js

In my post relaunching DevHawk, I mentioned that the site is written entirely in C# except for about 30 lines of JavaScript. Like many modern web content systems, Hawk uses Markdown. I write blog posts in Markdown and then my publishing "tool" (frankly little more than duct tape and bailing wire at this point) coverts the Markdown to HTML and uploads it to Azure.

However, as I went thru and converted all my old content to Markdown, I discovered that I needed some features that aren't supported by either the original implementation or the new CommonMark project. Luckily, I discovered the markdown-it project which implements the CommonMark spec but also supports syntax extensions. Markdown-it already had extensions for all of the extra features I needed - things like syntax highlighting, footnotes and custom containers.

The only problem with using markdown-it in Hawk is that it's written in JavaScript. JavaScript is a fine language has lots of great libraries, but I find it a chore to write significant amounts of code in JavaScript - especially async code. I did try and rewrite my blog post upload tool in JavaScript. It was much more difficult than the equivalent C# code. Maybe once promises become more widely used and async/await is available, JavaScript will feel like it has a reasonable developer experience to me. Until then, C# remains my weapon of choice.

I wasn't willing to use JavaScript for the entire publishing tool, but I still needed to use markdown-it 1. So I started looking for a way to integrate the small amount of JavaScript code that renders Markdown into HTML in with the rest of my C# code base. I was expecting to have to setup some kind of local web service with Node.js to host the markdown-it code in and call out to it from C# with HttpClient.

But then I discovered Edge.js. Holy frak, Edge.js blew my mind.

Edge.js provides nearly seamless interop between .NET and Node.js. I was able to drop the 30 lines of JavaScript code into my C# app and call it directly. It took all of about 15 minutes to prototype and it's less than 5 lines of C# code.

Seriously, I think Tomasz Janczuk must be some kind of a wizard.

To demonstrate how simple Edge.js is to use, let me show you how I integrated markdown-it into my publishing tool. Here is a somewhat simplified version of the JavaScript code I use to render markdown in my tool using markdown-it, including syntax highlighting and some other extensions.

// highlight.js integration lifted unchanged from 
// https://github.com/markdown-it/markdown-it#syntax-highlighting
var hljs  = require('highlight.js');
var md = require('markdown-it')({
  highlight: function (str, lang) {
    if (lang && hljs.getLanguage(lang)) {
      try { 
        return hljs.highlight(lang, str).value;
      } catch (__) {}
    }

    try {
      return hljs.highlightAuto(str).value;
    } catch (__) {}

    return ''; 
  }
});

// I use a few more extensions in my publishing tool, but you get the idea
md.use(require('markdown-it-footnote'));
md.use(require('markdown-it-sup'));

var html = return md.render(markdown);

As you can see, most of the code is just setting up markdown-it and its extensions. Actually rendering the markdown is just a single line of code.

In order to call this code from C#, we need to wrap the call to md.render with a JavaScript function that follows the Node.js callback style. We pass this wrapper function back to Edge.js by returning it from the JavaScript code.

// Ain't first order functions grand? 
return function (markdown, callback) {
    var html = md.render(markdown);
    callback(null, html);
}

Note, I have to use the callback style in this case even though my code is syncronous. I suspect I'm the outlier here. There's a lot more async Node.js code out in the wild than syncronous.

To make this code available to C#, all you have to do is pass the JavaScript code into the Edge.js Func function. Edge.js includes a embedded copy of Node.js as a DLL. The Func function executes the JavaScript and wraps the returned Node.js callback function in a .NET async delegate. The .NET delegate takes an object input parameter and returns a Task<object>. The delegate input parameter is passed in as the first parameter to the JavaScript function. The second parameter passed to the callback function becomes the return value from the delegate (wrapped in a Task of course). I haven't tested, but I assume Edge.js will convert the callback function's first parameter to a C# exception if you pass a value other than null.

It sounds complex, but it's a trivial amount of code:

// markdown-it setup code omitted for brevity
Func<object, Task<object>> _markdownItFunc = EdgeJs.Edge.Func(@"
var md = require('markdown-it')() 

return function (markdown, callback) {
    var html = md.render(markdown);
    callback(null, html);
}");
  
async Task<string> MarkdownItAsync(string markdown)
{
    return (string)await _markdownItFunc(markdown);
}

To make it easier to use from the rest of my C# code, I wrapped the Edge.js delegate with a statically typed C# function. This handles type checking and casting as well as provides intellisense for the rest of my app.

The only remotely negative thing I can say about Edge.js is that it doesn't support .NET Core yet. I had to build my markdown rendering tool as a "traditional" C# console app instead of a DNX Custom Command like the rest of Hawk's command line utilities. However, Luke Stratman is working on .NET Core support for Edge.js. So maybe I'll be able to migrate my markdown rendering tool to DNX sooner rather than later.

Rarely have I ever discovered such an elegant solution to a problem I was having. Edge.js simply rocks. As I said on Twitter, I owe Tomasz a beer or five. Drop me a line Tomasz and let me know when you want to collect.


  1. I also investigated what it would take to update an existing .NET Markdown implementation like CommonMark.NET or F# Formatting to support custom syntax extensions. That would have been dramatically more code than simply biting the bullet and rewriting the post upload tool in JavaScript.

Ambiguous ExtensionAttribute Errors

I was recently contacted by Nathanael Jones of the ImageResizer project about a question he had posted on Stack Overflow:

How can a single .NET assembly, targeting 2.0, 3.0, 3.5, 4.0, and 4.5 concurrently, support extension methods for both C# and VB.NET consumers?

Short Answer: You can’t. You think you can, but if you’re serious about targeting .NET 2.0/3.0 and 3.5+ as well as that whole C# and VB support thing, you can’t. Not really.

Long Answer: People love extension methods. Seriously, I think some people want to marry extension methods they love them so much. They just can’t stand to be without their extension methods, even when they’re using .NET 2.0.

Rather than go without, some people figured out how to get extension methods support on older versions of the .NET Runtime. Extension methods are essentially a compile time technology – the IL that gets emitted for calling an extension method is identical to the IL for calling a normal static method. The only runtime dependency for extension methods is the ExtensionAttribute which is used to mark methods intended to be used as extension methods (as well as classes and assemblies that contain them). ExtensionAttribute is defined in System.Core from .NET 3.5, but it’s just a marker. If you define your own copy of ExtensionAttribute and use the VS 2008 or later version of the C# compiler, you can get extension methods to work on .NET 2.0.

Back when I was working on IronPython, we ran into this exact issue when we merged DLR expression trees with LINQ expression trees. LINQ trees used extension methods all over the place, but we still needed to support .NET 2.0 in IronPython. We were already using the VS08 compiler so all we had to do was add our own copy of ExtensionAttribute to the DLR and we were good to go…or so we thought. Instead, we discovered that this approach doesn’t work as advertised – at least not if you care about VB support.

The problem stems from having multiple copies of ExtensionAttribute. IronPython and DLR had no problem – they were compiled for .NET 2.0 and thus had only the one copy of ExtensionAttribute that we added to the DLR. But people who used IronPython or DLR in a .NET 3.5 project ended up two copies of ExtensionAttribute – the copy we added to DLR and the official System.Core version. Two copies of a system provided type == start of a big problem.

Actually, if you’re only using C#, having multiple copies of ExtensionAttribute isn’t that big a deal. The C# compiler raises a warning when it find multiple definitions of a type in the System namespace. Because ExtensionAttribute is in the System namespace, C# has to pick one of the colliding type definitions to use. However, since the copies of ExtensionAttribute are identical it doesn’t matter which version the C# compiler picks.

Unfortunately, Visual Basic is much less forgiving when it encounters multiple versions of the same type. Instead of a warning like C#, the VB compiler raises an error when it encounters multiple definitions of ExtensionAttribute. So the “define your own ExtensionAttribute” approach leaves you with a DLL that won’t work from VB on .NET 3.5 or later.

Excluding VB on what was the most recent version of .NET at the time was a non starter for us, so we investigated other options. We discovered that we could “solve” this issue (again “or so we thought”) by having an internal definition of ExtensionAttribute in every assembly that needed it. Since the types weren’t public, VB stopped complaining about type collisions. C# still had the compiler warning, but we had already decided not to care about that.

I said at the time “It seems counterintuitive, doesn’t it? To solve a multiple type definition problem, we defined even more copies of the type in question.” Yeah, turns out I was kinda way wrong about that. We discovered later that having an internal ExtensionAttribute per project solved the VB ambiguous type error but introduced a new “break all the other extension methods in the project error”.

Remember earlier when I wrote it didn’t matter which copy of ExtensionAttribute the C# compiler picks because they are “identical”? Remember when I wrote we could solve the VB ambiguous type error by changing the visibility of ExtensionAttribute? Woops. Changing the visibility of our ExtensionAttribute meant it was no longer identical which meant it kinda mattered which copy of ExtensionAttribute the C# compiler choose. If the C# compiler picked one of our internal ExtensionAttributes, it would break every use of extension methods in the project referencing IronPython or the DLR!

We investigated a bunch of ways to control which version of ExtensionAttribute was selected by the C# compiler, but we couldn’t find an easy, obvious way in MSBuild to control the order of references passed to the compiler. In the end, we moved the custom ExtensionAttribute into its own DLL. That way, we could reference it from our IronPython and DLR projects to get extension method support but .NET 3.5 projects referencing either IronPython or DLR could reference System.Core instead. We still got the C# warning, but since we were back to identical ExtensionAttribute  definitions, the warning could be ignored.

Believe me, if there had been any way to remove the extension methods from the DLR and IronPython, we would have done it. Having a separate assembly with just a single custom attribute definition is an ugly hack, pure and simple. But the DLR was essentially the .NET 4.0 version System.Core – getting it to run along side the .NET 3.5 version of System.Core was bound to require hacks.

My advice to Nathanial on SO was the same as I gave at the top of this post: don’t use the “define your own ExtensionAttribute” hack to try and get extension method support on .NET 2.0. Extensions methods are nice, but they aren’t worth the headache of dealing with the errors that stem from multiple definitions of ExtensionAttribute when you try to use your library from .NET 3.5 or later.

My //build Talk

I just realized that while I posted the demo steps from my //build talk, I never posted the talk itself here on DevHawk. Consider that oversight rectified with this post.

(Note, the static image below appears cut-off, but the video should scale to the width of my blog automatically. If not, head on over to the official page for the talk over on Channel 9)

Using WinRT from C# //build Demo

Yesterday at//build, Jesse Kaplan and I delivered the Using Windows Runtime from C# and Visual Basic talk. In the talk, I demonstrated how natural and familiar it is to use WinRT from C# by building a simple Metro style app. This app  takes a picture with a webcam and implements the share charm contract in less than 15 lines of C# code.

Instead of making you try and read code off the recorded video stream that should be published soon, I’ve written this walkthru to explain exactly what I did in that demo. In addition, I’ve started from scratch (i.e. File->New Project) so that you can follow along at home if you wish.

First, you need to install the Windows Developer Preview. I recommend the x64 version with tools. Scott Hanselman has a great write up on using boot to VHD to run the preview. (though I do disagree w/ his assessment of dual boot. I’ve been dual booting Win7 and Win8 on my laptop for months and it’s never ended in tears or blood). Also, you’re going to need a webcam in order to run the app yourself.

Once the Windows Developer Preview is up and running, run the Socialite app and login with your Facebook credentials. We’re going to use Socialite to share the picture we take with the webcam. Giving it your credentials up front makes the demo run smoother!

Next, fire up VS11 (aka Microsoft Visual Studio 11 Express for Windows Developer Preview). Create a new project and select the Visual C# -> Windows Metro Style -> Application template.

Once the new project has been created, you should be looking at the MainPage.xaml file. Update the Grid element to contain a button and an image.

<Grid x:Name="LayoutRoot" Background="#FF0C0C0C">
    <Button x:Name="ClickMe" Click="ClickMe_Click">Click Me</Button>
    <Image x:Name="Photo" Width="800" Height="600"
           HorizontalAlignment="Center" VerticalAlignment="Center"/>
</Grid>

Next, hover over the Click=”ClickMe_Click” attribute of the button, right click and select “Navigate to Event Handler”. VS11 will take you to MainPage.xaml.cs and automatically generate a skeleton event handler for you.

In my //build session, I demonstrated that VS11 can automatically resolve WinRT namespaces the same way that it resolves managed namespaces. But for the purposes of this blog post, it’s easier if you just add the additional using statements we’re going to need at the top of MainPage.xaml.cs now.

using Windows.Media.Capture;
using Windows.Storage;
using Windows.UI.Xaml.Media.Imaging;
using Windows.ApplicationModel.DataTransfer;
using Windows.Storage.Streams;

Now, we add the code for ClickMe_Click:

private async void ClickMe_Click(object sender, RoutedEventArgs e)
{
    var ui = new CameraCaptureUI();
    ui.PhotoSettings.CroppedAspectRatio = new Size(4, 3);

    var file = await ui.CaptureFileAsync(CameraCaptureUIMode.Photo);

    if (file != null)
    {
        var stream = await file.OpenAsync(FileAccessMode.Read);

        var bitmap = new BitmapImage();
        bitmap.SetSource(stream);
        Photo.Source = bitmap;
    }
}

A few things to note about this code:

  • Even though it’s using native WinRT libraries, the C# feels natural and familiar – as if you were calling into traditional managed libraries. We’re newing up classes, we’re passing in constructor parameters, we’re using primitive numbers and enums, we’re assigning properties, etc. That is very much by design.
  • We’re using a couple of async WinRT methods (CaptureFileAsync and OpenAsync). C# 5.0′s new await keyword to make it extremely easy to write linear looking code that doesn’t block on async operations.
  • No P/Invoke or COM Interop attributes anywhere to be seen!

Finally, before we can run this code we need to declare our intent to use the webcam. Double click on the Package.appxmanifest file, click on the “Capabilites” tab, and then check the Webcam checkbox.

With the capability declared, now we can run the app. Hit F5 and VS11 will compile and deploy the Metro style app you just built. Click the button, acknowledge that you want to let the program use the webcam, take a pic, crop it, and there it is in your UI!

For the second part of the demo, I added share contract support. Here’s how to do that.

First, we need to pull the stream variable into class instance scope so that we can access it in the share contract event handler. We do that by adding a private IRandomAccessStream variable named stream and removing the var declarations from the line where we call OpenAsync. The updated click event handler looks like this:

//here's the instance scope stream variable
IRandomAccessStream stream;

private async void ClickMe_Click(object sender, RoutedEventArgs e)
{
    var ui = new CameraCaptureUI();
    ui.PhotoSettings.CroppedAspectRatio = new Size(4, 3);

    var file = await ui.CaptureFileAsync(CameraCaptureUIMode.Photo);

    if (file != null)
    {
        //the only change from the code above was to remove
        //the var declaration from the following line
        stream = await file.OpenAsync(FileAccessMode.Read);

        var bitmap = new BitmapImage();
        bitmap.SetSource(stream);
        Photo.Source = bitmap;
    }
}

Next, we need to wire up the share event handler in the XAML page’s constructor. That’s a single line of code and VS11 intellisense writes most of  it for you

public MainPage()
{
    InitializeComponent();
    DataTransferManager.GetForCurrentView().DataRequested +=
        new TypedEventHandler<DataTransferManager, DataRequestedEventArgs>(MainPage_DataRequested);
}

If you’ve ever wired up an event handler in C# before with VS, you’ll be familiar with the “Press TAB to insert” the correct event handler type followed by “TAB to generate handler”. Even though hthis is a WinRT event, VS11 helps you wire it up just the same as it does for managed events.

Now we implement the share contract event handler. That’s just a simple if statement – calling args.Request.Data.SetBitmap if the user has taken a picture and calling args.Request.FailWithDisplayText with an error message if they have not.

private void MainPage_DataRequested(DataTransferManager sender,
    DataRequestedEventArgs args)
{
    if (stream == null)
        args.Request.FailWithDisplayText("No picture taken!");
    else
        args.Request.Data.SetBitmap(stream);
}

This part of the demo shows off static methods and event handlers. Again, note how natural and familiar it feels to use WinRT from C#.

And we’re done, so hit F5 to build, deploy and run the app again.

I didn’t remember to do this in the //build talk, but first try selecting the share contract before taking a picture. Windows will display the “No picture taken” text in share contract window since the user taken a picture to share yet. That’s pretty boring so dismiss the share contract and take a picture like you did before. Then select the share contract, select Socalite, write a pithy message and press “Share in Facebook”.

That’s the entire demo! Taking a picture with the webcam, uploading to facebook, calling native WinRT APIs from C# in a natural and familiar way and all in just under 15 lines of code!

With our talk and demos, Jesse and I wanted to communicate just how important C# and VB are in the overall developer story for Windows 8. This demo shows off the hard work our two teams have done in order to make sure the managed developer’s experience with Windows 8 was the best that it could be. As I said in the talk – if you’re a managed developer, you already know how to build these Metro style apps.

I know I said it before, but I really can’t wait to see what you guys build with Windows 8!

The Windows Runtime

After nearly 2 years of not being able to tell anyone what I was working on – or even the name of the team I was on! – //build is finally here and the Windows 8 developer preview is finally out there in the open for everyone to start building applications for. You have NO idea how hard it’s been for me to keep my mouth shut and blog quiet about this!

I am a program manager on the Runtime Experience team, one of many teams in the Windows division building Windows 8. Our team is responsible for building the underlying infrastructure that powers the Windows Runtime (or WinRT for short). In particular, I work on the WinRT metadata infrastructure. I also work closely with our partners in Developer Division that use the metadata to project WinRT APIs into multiple languages.

In a nutshell, WinRT is the new API surface area for Metro style apps in Windows 8. WinRT APIs are available across multiple languages – C#, Visual Basic, C++ and JavaScript – enabling developers to build Metro style apps using the language and frameworks they are most familiar with. Much, much more info is available on the new Windows Dev Center.

In addition to the developer preview docs for WinRT, there are several sessions at //build focusing on WinRT – what it is, how it works under the covers, and how you use it from the various languages. Here’s a handy list of all the //build sessions you should check out if you want to know more about WinRT:

As I write this, not all the sessions have been delivered and none of them are available online yet. But they should all be online within a couple of days. Also, you can also get more information as well as ask questions over at the Windows Dev Center Forums. Our dev manager has already been very busy answering questions!

I am so excited that you can finally see what we’ve been working on and I can wait to see what you build with Windows 8!