Weakly Typed Dynamic Languages and Natural Selection

I’m not reading much in the way of blogs or twitter these days – way to heads down in my new job for that right now. But I did see Scott Hanselman’s post on method overloading and dynamic types and Ted Neward’s follow-on post static-typing fundamentalism. Even though I’ve moved on from the IronPython team, dynamic typing is a topic that’s still near and dear to my heart so I can’t resist throwing in my 2¢.

First off, I agree 100% with Ted’s post - though not the over-the-top mocking tone. These static > dynamic flame bait comments are so tired that they’ve literally become cliché. I agree with Ted’s points, but by answering fire with fire he’s just perpetuating the flame war that he claims to be so tired of. I really am tired of it, so I’m not going to bother to address any of the original anti-dynamic typing faux-arguments (fauxguments?) nor Ted’s artful and devastatingly mocking takedown of them.

But I do have a question for any static-typing fundamentalists in the audience: if static typing is so much better than dynamic typing, then how come dynamically typed languages are so popular? Doesn’t natural selection apply to type systems?

Those aren’t rhetorical questions. Building software takes time and effort. While developers often donate time and effort to projects (see: open source) typically they work for money. That money has to come from somewhere – usually it comes from someone who needs the software built for some business reason. And the people footing the bill for software construction demand the highest return on investment they can get.

If dynamic typing or VARIANT (which is actually weak not dynamic typing, but I digress) really did create “horrific devastation”, wouldn’t that have caused a negative feedback loop where the business people who actually foot the bills for creating software became wary and untrusting of using VB as the language of choice for their projects in favor of strong and statically typed languages that helped developers “make good choices”?

Yet the opposite happened. VB was the most popular programming language in the world for the better part of a decade. And while VB’s reign at the top is over, I’d argue that these days the most popular programming languages are PHP and JavaScript, both of which are weakly typed dynamic languages too.

Now clearly, popular != better. However, static-typing fundamentalism isn’t an argument about which way is “better” so much as an argument about which way is “worthy”. But how can you argue that you’re approach is the only worthy path when the opposite approach has been so successful? Remember, one developer’s “horrific devastation” might be another businessman's “successful project because it helped me enter a new market faster than my competitors”. 

DevHawk World Tour FY2010

As I’ve done the past two years, here’s a list of all the places I’m going in the next fiscal year. Traditionally, I’ve done this post by calendar year, but all MSFT planning is done by FY and so invariably I miss events early in the calendar year but late in the fiscal (like PyCon last year). I'll be updating this post periodically as I get tapped for more presentations. There are several other conferences I'm considering, submitting sessions for, in discussions with, but these are the ones that are confirmed.

Danish University Tour, Sept 7-11

My FY10 travels first take me to Copenhagen, where I was invited by the local subsidiary to present at four different universities in a single week. Don’t know how much sightseeing I’ll get done, but I’ll sure be talking a lot. My host Martin Esmann writes Stud.blog for Danish ComputerWorld and has a post (in Danish) about my visit. Personally, I am just excited about being featured in something called “Stud.blog”! :) Actually, Stud here means “Student” not “slender, upright members of wood” or any other definition of the term “stud”.

I’ll be visiting Aalborg University, Aarhus University, University of Southern Denmark and University of Copehhagen as well as delivering a TechTalk at the Microsoft Development Center Copenhagen, which is Microsoft’s biggest development center in Europe. I’ll primarily be delivering my Iron Languages introductory talk “Pumping Iron”, but there’s also some interest in language development on the DLR so I’ll be talking on that topic as well.

patterns & practices Summit Redmond 2009, Oct 12-16

This will be my third p&p Summit in a row and fourth in five years. This year, I’m doing a talk called “Not Everything is a new Nail() : How Languages Influence Design”. I was supposed to deliver this talk last year, but got side track with my day job and ended up talking about IronPython instead. Keith has made it VERY clear he doesn’t want another last minute substitution again this year.

Turing award winner Alan Perlis is credited with saying 'A language that doesn't affect the way you think about programming is not worth knowing.' Yet, most programmers rarely venture outside of the comfort zone of statically-typed object-oriented languages. Our heavy use of object-oriented languages influences our thinking to the point that we can?t see alternative approaches at all. This isn?t to say the object-oriented languages are bad, but as is typical in most things, there is no one 'best' way for all situations. In this talk, VS Languages PM Harry Pierson will look at a given software development scenario from both the object-oriented and functional perspectives, in order to see how much on an influence language really has on our engineering efforts.

Tech·Ed Europe 2009, Nov 9-13

I knew I was going to be updating this post over time, but I didn’t expect to have to update it so soon! Literally the day after I posted this, I got the speaker invite for Tech·Ed Europe 2009. My session hasn’t been posted yet, but this is the abstract we submitted:

Dynamic Languages on the Microsoft .NET Framework
The Dynamic Language Runtime (DLR) adds a shared dynamic type system, a standard hosting model, and support for generating fast dynamic code to the CLR. IronPython and IronRuby are Microsoft’s dynamic language implementations on .NET. In this talk, we’ll show you how to interactively create great .NET applications using dynamic languages. You’ll walk away knowing why dynamic languages deserve a spot in your toolbox!

It’s kind of generic, but given that most of the audience probably hasn’t seen IronPython or IronRuby, having broad latitude in my presentation topic is a good thing. I’ll probably deliver a variant of my standard “Pumping Iron” talk like I’m doing in Denmark. I delivered it recently at an internal event with Jimmy, so there’s lots more IronRuby content than there used to be.

The only bummer about doing Tech·Ed Europe is that I’m only doing one measly talk. I’m asking around – I’d love to do a .NET user group or university talk while I’m in town. Any takers?

Microsoft Professional Developers Conference 2009, Nov 17-19

Update: Tech·Ed Europe and PDC are on back-to-back weeks this year so we’ll be sending a teammate-to-be-determined to PDC in my stead. My family is very pleased I won’t be gone for two weeks straight.

Last year, I was on the content team for PDC. This year, that PITA responsibility belongs to someone else so I might actually get real work done in the four weeks leading up to PDC. My team will tell you, last year PDC sucked up 100% of my time for a month as we were driving towards our 2.0 release.

Technically, I haven’t had a talk for PDC accepted yet. But I submitted three and two are looking good (though I assume only one will make it to the actual show) so I thought I’d just go ahead and include it on this post. If/when my talks get accepted, I’ll post links and abstracts. Also, if one of my PDC talks is accepted, I’ll probably submit a talk for SoCal Code Camp as well.

PyCon 2010, Feb 19-21

This will also be my third PyCon in a row, though PyCon last year was a bit of a whirlwind since I had literally just joined the IronPython team. I finally feel like I might have something interesting to present at PyCon this year. Last year Dino and Jim handled the presentation duties from our team (with Michael Foord and Jonathan Hartley delivering a tutorial and Sarah Sutkiewicz speaking on FePy). We already have one announcement that I think is pretty significant lined up and might have a second depending on how hard I can push LCA and management between now and then. Talk proposals are due October 1st, so any suggestions would be appreciated!

VB Dev Lead Position Open

In case you’re job hunting, the VB team has a position open for a dev lead:

The Visual Basic team has a long history of delivering great value to our customers, and we are continuing that in the Dev10 release of Visual Studio. We’re looking for a Development Lead to help guide these efforts as well as shape future versions of the compiler.

The Visual Studio Languages group (VSL) develops VB, C#, F#, IronPython and IronRuby. As a member of this product unit, you’ll have the opportunity to work with others developing compilers and IDEs targeting the .NET runtime. You’ll benefit from their experience and contribute best practices and methodologies of your own. In VSL, developers work closely with their QA team, and we are committed to delivering the best value for our customers at very high quality.

As a Development Lead on the Visual Basic compiler, you’ll be the hand at the tiller of VB.NET compiler development. Specifically, you will:

• Manage the day-to-day duties of the compiler and runtime development team, ensuring on-schedule delivery of high quality components.
• Help chart the direction the compiler team takes by prioritizing efforts in coordination with your counterparts in QA and PM.
• Contribute to the design of the Visual Basic programming language.
• Mentor your team of developers to continue their career growth.
• Help shape the engineering environment and procedures in Visual Studio Languages.
• Work closely with the IDE team to help them provide a top notch editing and debugging experience.

To be successful, you’ll need the following:

• A demonstrated aptitude for managing a team of high-caliber developers.
• Excellent communication, collaboration and negotiation skills and the ability to drive open issues to closure.
• Strong architectural sense and a working knowledge of the fundamentals of compiler design.
• Passion for delivering customer solutions and quality software in general.
• Working knowledge of the managed runtime environment is a strong plus.

Register now for Lang.NET Symposium 2009

We’re now accepting registrations and talk proposals for the 2009 Lang.NET Symposium, which will be held on the Microsoft campus April 14th-16th.

For those who aren’t familiar with this event, the Lang.NET Symposium is an opportunity for programming language designers and researchers from industry and academia to get together and basically geek out for two and a half days. And it’s not just Microsoft language wonks like Anders Hejlsberg or platform wonks like Don Box. For example, last year we had Dan Ingalls from Sun, Gilad Bracha from Cadence, John Rose from OpenJDK and Wez Furlong from the PHP Project among many others.

If you’re curious about the kind of talks we typically have at Lang.NET, the talks from 2006 and 2008 are available online. 

I’ve attended both previous Lang.NET Symposiums and spoke at last year’s event, but this will be my first as a part of the team driving the event itself. We’ve expanded a little this year, though we’re still a small conference so the signal-to-noise ratio should still be very high. We barely fit in the Platform Lab last year, so we moved this year’s event to Microsoft Research’s sweet new building #99.

This year, we’re very excited to be joined at the hip with Chris Sells’ DSL Developers Conference. Those guys over in CSD are working on some cool language stuff of their own, so it just made sense to hold the two events in the same space, one right after the other. Note, registration for each event is separate. You’re welcome to attend both – we’d love it if you did - just make sure you register for both.

As in the past, there is no cost to attend Lang.NET. You get yourself here and find place to stay and we’ll take care of the content, meals and beers in the evening. Just make sure you let us know you’re coming and what, if anything, you’re interested in getting up and talking about by registering at the Lang.NET Symposium website.

The Reese’s Peanut Butter Cups of Language Conferences

It’s been about 11 months since the last Lang.NET Symposium and we’ve been working on next year’s version. But then we discovered that we’re not the only ones inside Microsoft thinking about having a language conference. A couple of meetings later, and we’ve decided to combine them, which will assuredly lead to side conversations like this:

DSL DevCon Attendee: Hey, you got your compiler in my DSL!

Lang.NET Attendee: Hey, you got your DSL in my compiler!

[They sample the combined content]

Both Attendees: Mmmm, two great conferences that work well together!

Well, maybe not. But seriously, it should be a great combined conference.

However, there are some logistics things we need to work out, like how many days should the combined conference run? We figure the “right” answer to these questions depends on the likely overlap between the two conferences. Frankly, we don’t know what the overlap will be so we decided to simply, you know, ask.

If you are interested in attending Lang.NET, the DSL DevCon, or both next year, please head over to Chris Sells blog and make your voice heard.

Programming Languages @ PDC08

The PDC folks pushed out a bunch of new sessions yesterday, including a bunch from my part of DevDiv. You can see the full list of sessions that have been published (so far) at the PDC site.

An Introduction to F#
Learn about Microsoft's new language, F#, a typed functional programming language for the .NET Framework. F# combines functional programming with the runtime support, libraries, tools, and object model of .Net. Understand how F# asynchronous workflows help tame the complexity of parallel and asynchronous I/O programming and how to use F# in conjunction with tools such as Parallel Extensions for .NET.

Deep Dive: Dynamic Languages in .NET
The CLR has great support for dynamic languages like IronPython. Learn how the new Dynamic Language Runtime (DLR) adds a shared dynamic type system, a standard hosting model, and support for generating fast dynamic code. Hear how these features enable languages that use the DLR to share code with other dynamic and static languages like VB.NET and C#.

Future Directions for Visual Basic
Come learn about the new capabilities in the next version of the language, including: extensions to LINQ, syntax simplifications, and improvements to the IDE. We'll provide insight into the direction of the language, including dynamic binding, meta-programming, and scripting.

The Future of C#
In this talk Microsoft Technical fellow and C# Chief Architect Anders Hejlsberg outlines the future of C#. He will describe the many forces that influence and shape the future of programming languages and explain how they fit into C#.

Visual C++: 10 is the New 6
Get more done. The next version of Visual C++ is all about improving developer productivity for large-scale applications. Learn about the IntelliSense and browsing experiences, changes to the project and build system, project-less browsing, collaboration through remote symbol indexing, and custom visualization of symbolic information.

Morning Coffee 156

• My hockey team won last night 4-2. No points for me, but I was even on the night. I did spend some time in the penalty box, but I was serving a two many men on the ice bench minor. We only had nine skaters, not enough for two full lines, so I'm pretty tired today. However, I'm not as tired as I was two weeks ago - that's a good sign.
• Politics 2.0 watch: The Obama campain announced yesterday that they raised $55 million in donations in the month of February. That's significantly more than Clinton ($35 million) and McCain ($12 million) combined. Even more impressive is that $45 million of that was raised online, of which $40 million were from donations of $100 or less and $22.5 million were from donations of $25 or less. I guess in Politics 2.0, individuals contribute more than online punditry and video parodies of political commercials.
• TextGlow is a Sivlerlight 2 based Word docx file viewer, created by James Newton-King. Nice, but what I really want is "SlideGlow", a SL2 based PPTX file viewer. (via DNK)
• Speaking of Silverlight, Windows Live launched an experimental site called PhotoZoom which will let you create DeepZoom photo albums. (via LiveSide)
• Charlie Calvert has created a home for Language Futures discussion on MSDN Code Gallery. If you'll recall, back in January he asked for input on Dynamic Lookup. Now he's looking for feedback on Call Hierarchy, a proposed VS IDE feature to help you visualize how your code flows. Great idea, but the Call Hierarchy dialog mockup isn't very intuitive. Couldn't we put these visualizations into the code editor window directly, like CodeRush does?
• John Lam continues his Dynamic Silverlight series, first building a Flickr image browser in Managed JScript then showing how to integrate an IronRuby version of the Flickr image browser with an ASP.NET MVC app.
• EdJez is inspiring. Subscribed. (via Brad Wilson)

Morning Coffee 149

• Big news yesterday was Microsoft announcing "Strategic Changes in Technology and Business Practices to Expand Interoperability". More details available at the new Interoperability website and this interview with Bob Muglia.
• The videos from Lang.NET 2008 are now available, including mine.
• Mix08 is just under two weeks away, and the news is ramping up. Scott Guthrie provides a "first look" at the Silverlight 2.0 beta (aka the one with .NET) that will release @ Mix. He also provides a set of eight tutorials where he demonstrates building a Digg client for Silverlight 2.0. I've been doing a little Silverlight 1.0 experimentation recently, but I think I'm going to scrap it in favor of waiting for 2.0.
• Shorter Nick Malik: IoC is cool, but isn't a silver bullet. Go read the whole thing.
• Brad Wilson has a new laptop with a solid-state hard drive. VS08 installs in under 10 minutes? I gotta get me one of those...
• In addition to the XNA news, Microsoft also announced Gears of War 2, Fable 2 and Ninja Gaiden II and Too Human will all be available this year exclusively on Xbox 360. Of those, I'm most interested in Fable 2, though Too Human is looking interesting. But what about Halo Wars? When is that shipping?

Morning Coffee 142 - Wishful Catchup Edition

• After spending most of the last four days away from my desk, I was planning on a quiet day to catch up on a variety of things. Then I heard the oh-so-minor news that Microsoft is offering to buy Yahoo for almost $45 billion. Hasn't been much reaction on the dev, architecture, politics and hockey-oriented blogs I read, but you can get a ton of reactions on TechMeme.
• Lost is back. Finally. I stayed up late last night reading Lostpedia, catching up on Lost Missing Pieces and the Find 815 ARG.
• Alex The Great had four goals and an assist in last night's victory. Coughing up three goal lead and letting the Canadiens tie the game in the last 30 seconds isn't encouraging, but a win is a win. The Caps are currently one game behind the SE leading Hurricanes and two games behind the current eight seed Rangers. Alex was named first star for January.
• Ted Neward has a nice summary of Lang.NET by day: one, two and three. I wonder if my talk qualifies for the exception to Ted's rule that "A blog is not a part of your presentation, and your presentation is not part of your blog". I had 15 minutes to discuss something I've written about over ten posts  (so far).
• John Lam points to the latest DLR hosting spec. I'm much more interested in the DLR code generator, but at least the hosting interface is documented.
• Scott Hanselman has a nice post on fluent interfaces. Note to self, find out if Beautiful Soup works with IronPython.
• I wonder if the VS Source Code Outliner PowerToy works with F#? (via Sam Gentile)
• Chris Tavares has an extensive post Deconstructing ObjectBuilder? I've poked around inside OB before, but I'm really looking forward to Unity (also via Sam Gentile)
• NVIDIA finally updated the drivers for the video card in my Tecra M4. That only took a year.

Morning Coffee 141 - Lang.NET '08 Edition

I was hoping to blog my thoughts on Lang.NET as the event went along. Obviously, that didn't happen, though I was pretty good about dumping links into my del.icio.us feed. The talks were all recorded, and should be up on the website in a week or two. Rather than provide a detailed summary of everything that happened, here are my highlights:

• The coolest thing about conferences like this is what John Rose called "N3" aka "Nerd-to-Nerd Networking". It was great to meet in person, drink with and geek out with folks who's blogs I read like Tomas Petricek, Wesner Moise and Larry O'Brien. Plus, I got to meet a bunch of other cool folks like Gilad Bracha, Stefan Wenig and Wez Furlong. That's worth the price of admission (which was admittedly nothing) right there.
• Coolest MSFT talk: Martin Maly "Targeting DLR". I was wholly unaware that the DLR includes an entire compiler back end. Martin summarized the idea of DLR trees on his blog, but the short version is "you parse the language, DLR generates the code". That's pretty cool, and should dramatically lower the bar for language development. Of course, I want to write my parser in F#, so I'm going to port the DLR ToyScript sample to F#.
• Runner-up, Coolest MSFT talk: Erik Meijer "Democratizing the Cloud with Volta". Erik is a great speaker and he really set the tone of his session with the comment "Division by zero is the goal, not an error". He was referring to an idea from The Change Function that user's measure of success is a function of perceived crisis divided by perceived pain of adoption. Erik wants to drive that adoption pain to zero. It's a laudable goal, but I remain unconvinced on Volta.
• Coolest Non-MSFT talk: Gilad Bracha "Newspeak". Newspeak is a new language from one of the co-authors of Java. It's heavily smalltalk influenced, and runs on Squeak. He showed developing PEGs in Newspeak, and they were very compact and easy to read, easier even than F#. He calls them Executable grammar, and you can read his research paper or review his slides on the topic. Unfortunately, Newspeak isn't generally available at this time.
• Runner-up, Coolest Non-MSFT talk: Miguel de Icaza "Moonlight and Mono". The talk was kinda all-over-the-place, but It's great to see how far Mono has come. Second Life just started beta testing a Mono-based script runner for their LSL language (apparently, Mono breaks many LSL scripts because it runs them so fast). He also showed off Unity, a 3D game development tool, also running on Mono.
• Resolver One is a product that bridges the gap between spreadsheets and applications, entirely written in IronPython (around 30,000 lines of app code and 110,000 lines of test code, all in IPy). Creating a spread-sheet based app development environment is one of those ideas that seems obvious in retrospect, at least to me. If you do any kind of complicated spreadsheet based analysis, you should check out their product.
• If you're a PowerShell user, you should check out PowerShell+. It's a free console environment designed for PowerShell and a damn sight better than CMD.exe. If you're not a PowerShell user, what the heck is wrong with you?
• Other projects to take a deeper look at: C# Mixins and Cobra Language.
• I thought my talk went pretty well. It's was a 15 minute version of my Practical Parsing in F# series. Several folks were surprised I've been coding F# for less than a year.

Morning Coffee 140

• I only posted one Morning Coffee post last week. It wasn't a lack of content, it was a lack of drive on my part. I had 20-30 items flagged in my news reader, but for some reason I couldn't work up the interest in posting them. So some of these are a bit old.
• I'm at the Language.NET Symposium this week, so look for lots of language blogging. I've already chatted with Tomáš Petříček and John Lam. If someone kicks Ted Neward's ass because he hates Perl, I'll try and liveblog it.
• Speaking of Ted Neward, he discusses the question "Can Dynamic Languages Scale?" without devolving into a flame-fest. I agree 100% with his point about the difference between performance scaling and complexity scaling. Personally, I tend to err on the side of better complexity scaling, since buying hardware is easier than hiring developers.
• Nick Malik responds to me calling his shared global integration vision flawed. He points to NGOSS/eTOM as an example of a shared iterative model that works. I know squat about that shared model, so I'll refrain from commenting until I do a little homework on the telco industry.
• Speaking of shared interop models, Microsoft is joining DataPortability.org. Dare Obasanjo and Marc Canter are skeptical that so far this effort is all hype and no substance. Reminds me a bit of AttentionTrust.org. But if DataPortability.org can get off the ground, maybe there's hope for Nick's vision (or vis-versa).
• Don Syme lists what's new in the latest F# release. As I said, this release is pretty light on features. Hopefully, I'll get some details
• Tomas Restrepo shows how to change your home folder in PowerShell. I need to do this.

Morning Coffee 98

• Morning Coffee was canceled on Thursday and Friday on account of a kidney stone. So not fun. Luckily, it was a little one and it was alone, but I will be listening very closely to my doctor's advice to avoid another.
• Took the kids to see Ratatouille last Tuesday and saw Transformers yesterday with my wife due to fluke babysitter luck. I liked Ratatouille, but I'm not sure it's the 51st best movie of all time. On the other hand, major props for making a kid movie with a significant lack of toy tie-ins. Ratatouille is a better movie that Cars, but I don't see my four year old boy trading in is Lightning McQueen toy car for a Remy the Rat. Transformers on the other hand obviously did not forgo the toy tie-ins! Still, it wasn't bad. Kinda reminded me of The Rock with a bigger budget.
• Micahville listed DevHawk on it's list of 69 Tech Blogs That Don’t Suck. Thanks!
• David Ing boldly writes that C# is getting fat. Or maybe it's just big-boned. My take: no question that integrated query is a big feature that covers a lot of surface area. But given the prevalence of databases and other queriable stores, it's critical to improving programmer productivity. Go read Todd Proebsting's talk on Disruptive Programming Language Technologies. Two of his candidates for disruptive language technologies were Database Integration and Manipulating XML. LINQ neatly covers both.
• According to John Shewchuck, the new BizTalk Services release is available. However, when I click on the "what's new" page, it tells me they're experiencing technical difficulties. (Their error page is Oops.aspx. Funny!)
• Scott Hanselman has Programming Personas 2.0. Who are you? I thought I was and "Order n" Architect (the quote "Where's the whiteboard" is spot on) but my CS background isn't as strong as the persona's.
• Sam Gentile is starting to dig into Concurrency and he has a great list of links that have influenced his design.

Morning Doughnuts 7

Once again I get the chance to fill in for Harry while he is gone. I will attempt to keep up with the high standard he has set.

• Be careful with trampolines. I have a four year old that was bouncing on a trampoline this weekend. He twisted his knee slightly on landing, and decided that the fastest way to get comfort was to jump off the edge. His knee gave out and he fractured his tibia. I don't think this was how he planned to start his summer. At least he is pretty calm about the whole thing.
• It looks like there is a good product out there named CliSecure to obfuscate .net code. From what I was able to read it looks like a pretty decent product, even hiding the code while its in memory. (via Larkware)
• TechEd started this morning. While I am sure Harry will be giving some on-site reports there is a link to the virtual site here.
• There is a great video showing Gregor Hohpe talking about SOA, and the many unrealistic claims in the industry. If you have read any of Hohpe's work it is clear that he has a great understanding of the topic. (via Nick Malik)
• The NBA Finals will begin this week with the Spurs playing the Cavaliers and King LeBron. I wonder if this will help rescue the NBA from what seems to be a real apathy on the part of the average fan. Other than the series between Dallas and Golden State earlier in the play-offs it really seems there haven't been any compelling stories. The NBA really needs a shot in the arm to become relevant again.

Morning Coffee 86

• Google announces Gears, a browser plugin for taking your web application offline. Developer docs are also available. TechMeme has lots more, but obviously this is yet another significant bow shot in the upcoming unified client platform war. By my count, there are four horses in this race: Microsoft with .NET and Silverlight, Adobe with Flash and Apollo, Google with AJAX and Gears and Sun with Java and JavaFX. Did I miss anyone? (via Dare Obasanjo and Scott Hanselman)
• Alex James writes that REST is about intent and shows a pseudo-code sample posting multiple changes to a single endpoint as a way of demonstrating your intent that they be applied atomically. Andres Aguiar left a comment saying that Astoria does something similar. Personally, I like that model for transactions better than the transaction factory approach Jon Udell describes. But either way, you've moved beyond simple CRUD style services and into the world of protocol. Surfacing intent via protocol aligns with what Tim described as making the protocol explicit
• Windows Live posted new beta versions of Writer, Mail and Messenger. I've been on an internal build of the new Writer for a while and I've really been impressed. There's also a new Provider Customization API, so I can't wait to see what the DasBlog folks do with that.
• Scott Guthrie's LINQ series continues, this time covering how to build the LINQ to SQL data model. Looks like they used the DSL toolkit to build the LINQ to SQL data model designer, cool! 
• Martin Fowler digs into racc, a yacc-esque compiler compiler for Ruby. Looks interesting as a internal DSL example (better than the now-canonical rake example). But why is the sexy new language on the block using old school CFG's instead of new hotness PEG's?
• Speaking of Martin, he writes about the opportunity Ruby presents to Microsoft, building on Scott Hanselman's concerns that Microsoft is losing the Alpha Geeks. Sam Gentile also weighs in, suggesting that Microsoft is at the crossroads. Frankly, I don't work in evangelism anymore so I'm going pass these links along without comment except to say that Scott, Martin and Sam are all folks I have much much respect for.

Morning Coffee 82

• Dare reached a similar conclusion about Silverlight and Apollo that I did. But newly minted Adobe RIA evangelist Ryan Stewart writes that calling Silverlight vs. Apollo the next platform competition is overblown. However, he then reiterates my point that MSFT is moving in on Adobe's traditional stronghold (aka the web) while Adobe is moving in on MSFT's traditional stronghold (aka the desktop). The upshot is that both companies are trying to deliver a unified "client" platform that spans desktop, devices and browser. That sounds like "platform competition" to me.
• Apparently I'm missing the this years compiler dev lab. :( John Lam is there, with slides from his talk on the DLR. Miguel de Icaza is there, with fairly extensive notes. Hopefully they will post talks like they did last year.
• Speaking of languages, Scott Hanselman explains why he thinks Ruby is "the tits". Personally, on the scale between "I can't see where Ruby is such hot shit" and "Ruby is the tits", I'm somewhere near the middle, leaning towards tits. I love the expressiveness of Ruby's syntax, but I miss the static typing. Call me over the hill, but I like the compiler catching mistakes at compile time. I realize it's not for everyone, but I like what I like.
• This talk about Ruby's expressiveness reminds me of something Larry O'Brien said in the wake of the IronRuby announcement: "I am surprised by the IronRuby announcement. I really thought we were going to see some form of Ruby#:Ruby::C#:Java. Although I'm happy, I was actually hoping to see a new language." The expressiveness of Ruby that Scott describes (which is to say, not all of Ruby's expressiveness) would be completely achievable in a statically typed language. Personally, I'd like to see that language...
• My birthday is past, but I want a Lego Ice Cube Tray. (via Geekdad)

Morning Coffee 79

• Soma announces PopFly, the "fun, easy way to build and share mashups, gadgets, Web pages, and applications" from the Non-Professional tools team. The PopFly team blog has some videos. Sounds vaguely like Yahoo! Pipes, but cooler. While most of the focus is on their browser-based mashup creator, they also have VS support for the non-non-professionals among us.
• Eric Nelson suggests that the new Dynamics CRM systems is actually a LOB application platform in it's own right. More details in Ben Riga's MIX session. (via Gianpaolo)
• Sam Gentile is worried that C# is becoming to complex, especially when you also consider how fast the platform is moving underneath. When you get your head out of the debugger for a second and look at the Big Picture, it certainly seems overwhelming. Is it just a question of getting used to it? The first time I fired up the VS.net 2002 alpha and looked at all the classes in the BCL, I had the same overwhelmed feeling, but eventually I got over it. Or have things just gotten too big and move to fast now? If so, it's time for some new layers of abstraction...
• Udi Dahan writes about building testable services. Testability has to be a core consideration when building anything, but especially a reusable framework. I've had similar thoughts about language design. How do you unit test a DSL?
• Roberto Medrano of SOA Software thinks "maybe 20 percent of IT folks understand SOA and half of the rest think they do". Personally, I think most IT folks don't agree on what SOA is or should be. Furthermore, we don't even have a common lexicon to discuss it, so we end up talking past each other and arguing about topics we agree on. I think Roberto is really saying is "most people are wrong because they don't agree with what I think SOA is". (via Jack van Hoof)
• Jeffrey Snover talks about the virtuous cycle of .NET language support. His point is that time spent learning .NET pays off as you transition between system programming (C#, VB.NET), shell programming (PowerShell) and script programming (IronPython, DLR). I'm not sure I would break them down that way, but his point spot on.
• Clemens Vasters experiments with the new BizTalk Services with a sample called TweetieBot. I agree 100% with his point about the assumption of centralization will be challenged by the federation of personal services.

Morning Coffee 65

• My brother is a VaTech alumni, so the shock of the deadly shootings there yesterday hits very close to home. My heavy heart is with the grieving Hokie nation today.
• Jeff Atwood has a couple of great posts on Language vs. Platform. Earlier in my MSFT career, I spent a significant amount of time explaining .NET, often to companies that had made a significant investment in Java. Picking the Java platform is fine (it's almost the best platform around!), but it seemed many people I spoke to didn't understand the fact that "[w]hen you choose a language, like it or not, you've chosen a platform".
• Ian Thomas riffs on my When is a Service Not a Service post. I like Ian's thinking about SaaS as an analogy for SOA adoption - if for no other reason that SaaS is easier to "get". But trying to realize SOA via SaaS inside the enterprise is a mistake in my opinion (and I think Ian would agree with that). SaaS is a business model, and I don't think you want to turn your enterprise into an internal service marketplace. Instead, this ties back with Nick Malik's points about central planning. Regardless if I'm right or wrong, I subscribed to Ian's blog (and not just because he linked to me - check out his Elements of the Future Business Ecosystem)
• TLA Watch: Oracle coins Application Integration Architecture (aka AIA). Joe McKendrick calls it "Big SOA". Isn't this the market segment that BizTalk has been in for seven years?

Morning Coffee 56

• I survived the weekend no problem. My wife has the details of what she did for the weekend while I played Mr. Mom. The kids were great, we even went to see the Easter Bunny on Sunday. Wish the weather had been better, but we did get to go on a little walk around the neighborhood between hailstorms Sunday after naps.
• Between taking the kids all morning until Jules got home from the airport and going to opening day for a team morale event, I worked about 30 minutes yesterday. In case you're wondering, that's way below average. I typically work at least twice that every day. :)
• After maintaining a post a day average for January and February, I slipped a bit in March. Twenty seven posts in thirty one days. So that puts me five posts behind for the year as of this one.
• Dale let me borrow Madden 07 for the weekend so I could pump my gamerscore (a practice called gamerscore whoring). I still need 255 points by April 22nd to complete the Old Spice Experience Challenge. I'm not proud of it, but it's not like I have much time to play these days.
• Mads Kristensen has a new .NET blog engine intuitively called BlogEngine.NET. I wonder how it compares to dasBlog, which powers DevHawk. (via DotNetKicks)
• I wrote a last week that unit test support should be in the Express editions of VS. Thanks to Jamie Cansdale, it is. (via Larkware)
• Scott Hanselman saved his C# Tiny OS project from the impending shutdown of GDN and reposted it to his blog. I first met Scott at TechEd Malaysia 2002, so I remember seeing him present this "back in the day".
• EMI is going to start offering songs sans DRM @ $1.29 a pop. Assuming other labels follow suit, this is gonna be huge. (via Loke Uei) 
• Jomo Fisher writes about using LINQ as a string switch compiler that's about 900% faster than using a hash table. Money quote: "Any time I see a data structure with a capability I’m not using it makes me wonder whether I can trade that capability for something I do need—in this case a speed boost." LINQ is turning out to be much more interesting than just a (much) better way to query databases. (via DotNetKicks)

I'm Wrong Because Ruby and Powershell Are Mainstream

Brad Wilson and Scott Hanselman took me to task for my comment the other day that no "mainstream" language had implemented extension methods:

How mainstream is Ruby on Rails for you? Ruby is a full fledged dynamic language. No hacks for "extension methods" (Brad)

Ya, I kind of blanched at that statement too...method_missing is pretty mainstream... (Scott)

They're right, Ruby does support the addition (and redefinition I think) of methods on a class at any time. There's a sample of this in the Classes and Objects chapter of Programming Ruby (aka the pick-axe book) where they add a basic documentation facility "available to any module or class" in Ruby by adding a doc instance method to the Module class.

Given how Ruby classes are defined, I think the newly added methods have access to the private data of the class. Extension methods in C#3/VB9 only have access the public interface of the object. But that's a fairly minor difference.

FYI, Powershell can do this as well, though not as succinctly as Ruby. Scott has an example how you can add a DatePhotoTaken property to System.IO.FileInfo using Omar Shahine's PhotoLibrary project.

Chalk this up to my continuing ignorance of dynamic languages. I'm working on it, albeit slowly.

Morning Coffee 38

• As predicted, the XNA guys had a bunch of news to announce at GDC. They launched the XNA Creators Club Online with samples, forums and a new starter kit. Also they announced some cool partnerships for Creators Club Premium members (aka the folks who paid $99 a year to be able to run code on their Xbox 360) including access to the highly anticipated Torque X Engine.
• From Don Syme, we here that the new 1.9 version of F# is almost ready. When I shifted from learning F# to learning PowerShell, it wasn't because of a sudden lack of interest in F#, so I'm glad to see them still chugging along. And by the time they release 2.0 later this year, I hope to have learned enough PowerShell that I can spend some time focusing on learning F#.
• Apparently Jenny Lam at Microsoft reviewed more than 10,000 images to pick the new Vista Wallpapers. Not sure if I would love or hate that job. But some that didn't make the cut are available online. (via DotNetKicks)
• Tomas Restrepo reviews a number of lightweight, syntax-highlighting text editors. Personally, I like Notepad2, but as Tomas mentions there's no way to save your settings. Also, there's no way to add new syntax highlighting without recompiling it. For example, Wesner Moise compiled a version that added Ruby syntax. I really want a version that that supports PowerShell. Maybe it's time to give Notepad++ a closer look
• Dale has some new SOAhlocis Anonymous shirts available.
• Jeff Atwood discovers that the French acronym for object oriented programming is POO.

Morning Coffee 15

• I was checking out the ASP.NET community site, and I noticed a small "Microsoft Communities" toolbar across the top. There's a little menu that links to other MS community sites like Channel 9 and MSDN Blogs. I'm surprised the NetFx3 community site isn't included.
• My teammate Dale is blogging about Proper SOA. He lays out 6 Proper SOA principles, and then drills into the first three: meets business needs, requires governance and responds to changing business drivers. I expect to see posts on the remaining principles this week. Maybe Dale should turn this series into an article.
• Speaking of articles about architecture, Architecture Journal 10 is online as a PDF. This issue's topic is Composite Applications.
• Malbolge is a programming lanugage that is "specifically designed to be difficult to program in." Here's Hello World in Malbolge:
  (=<`$9]7<5YXz7wT.3,+O/o'K%$H"'~D|#z@b=`{^Lx8%$Xmrkpohm-kNi;gsedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543s+O<olm
  Seriously. Actually, it's worse than it looks. The effect of any instruction depends on where it is located in memory.  Malbolge is so difficult, it took a month to write a Lisp program to generate that program. However, Lou Scheffer thinks we should think about Malbolge as a cryptosystem. I wonder if it could be used for obfuscation? (via Good Math, Bad Math)
• Nat Torkington blogs about teaching kids to program. He makes the point about "them to think in terms of small steps". I was lucky to have a computer teacher in elementary school who did something similar. She had us write down instructions for making a peanut butter and jelly sandwich and she then followed them to the letter. For example, if you wrote, "spread peanut butter on the bread" with out first instructing her to take out a slice, she'd happily spread peanut butter on the entire loaf. (via reddit)
• To this day, my wife thinks the peanut butter and jelly lesson negatively affected my ability to communicate with "normal" people. She'll even say "peanut butter and jelly" when she thinks I'm being particularly obtuse in my communication.

Managed Lex and Yacc

Buried deep in the docs for v3 of the Visual Studio SDK is something called the Managed Babel System. Babel - in the context of Visual Studio - is the framework for creating language services for stuff like syntax highlighting, brace matching, and IntelliSense completion. As part of the Managed version of Babel, the VS SDK includes the Managed Package Lex Scanner Generator (MPLEX) and the Managed Package Parser Generator (MPPG). Online docs are pretty thin, but there are a few white papers in the additional documentation of the VS SDK install.

Interestingly enough, the white papers were written by John Gough of QUT's Programming Languages and Systems group. John wrote Compiling for the .NET Common Language Runtime and PLAS has several interesting language projects including Metaphor and Ruby.NET. As you might expect for a group that focuses on languages on .NET, they have a managed version of YACC called GPPG available and John said at Lang.NET that they would be releasing a managed version of LEX "soon". I'm thinking that QUT's GPPG/GPLEX combo has been absorbed into the VS SDK and renamed MPPG/MPLEX. Unfortunately, my laptop power connector is broken, so I can't verify this until Monday.

It's not a Rosetta stone and the lex/yacc model is getting pretty long in the tooth, but I'm thinking at least Larry O'Brien will be interested in these tools.

UPDATE - Apparently, this is somewhat old news, as per Aaron Marten:

The tools we're including are called MPPG & MPLex (which stand for Managed Package Parser Generator and Managed Package Lexer). They are derivative works from the open-source GPPG/GPLex tools developed at the Queensland University of Technology.
[Managed Language Tools in Visual Studio 2005 SDK]

Python and DSLs

When I read Larry O'Brien's post IronPython as a Foundation for DSLs, I had a strong sense of deja-vu. I went thru a very similar thought process when I discovered the early versions of IronPython were written in Python. I figured that meant Python must have some support that makes building compilers easier. Unfortunately, that doesn't turn out to be the case. Python provides a standard module called parser which "provides an interface to Python's internal parser". That means that compiling Python is easy, but there's no support for compiling any language other than Python.

I've been doing a little experimenting in this space. First, as I've written before, I've been playing with parsing expression grammars. Second, I want to take a close look at Metaphor from QUT:

Metaphor is a programming language with support for type-safe run-time code generation -- a form of meta-programming. Metaphor is based on a subset of C# or Java and combines the imperative, object-oriented nature of these languages with the multi-stage programming constructs from MetaOCaml. Metaphor uses the static type system of multi-stage languages to achieve compile-time safety of run-time generated code...

Metaphor is implemented as a compiler on the Microsoft CLR.

Finally, I need to take a closer look at ANTLR. Don't know how I missed it, but I had never seen ANTLRWorks until Larry linked to it.

Language Features I Wish C# Had - Symbols

Ruby's symbols are often talked about in terms of efficiency - taking up less memory and executing faster. While these are both laudable goals, symbols are more than just performance improvers. The ability to name things is valuable semantically. Take a look thru p&p's Composite UI AppBlock and you'll see strings used as names for things all over the place. It's great for loose coupling, you see. But how do you tell the difference between a string used as a name and a string used for some other reason like user input? You can't.

Rails makes extensive use of symbols - anyone who has Rolled on Rails has seen "scaffold :recipe". That's just the tip of the iceberg. Rails uses symbols extensively across both ActionPack and ActiveRecord (and probably others I'm not familiar with). It's a great approach, but one that's unique to Ruby as far as I'm aware.

Language Features I Wish C# Had - Tuples

Several languages, such as Python, have the concept of a Tuple built into the lanugage. One of things it's used for in Python is multiple return values. So you can call "return x,y" to return two values. Of course, C# can only return one. If you need to return more values, you have to define out parameters.

LINQ / C# 3.0 / VB 9 support the idea of anonymous types, which is similar to a tuple. The big difference is that, because they're anonymous, they can't leave the scope they're defined in. In other words, they're great within a function, but if you want to pass them out of your function type-safely, you have to define a non-anonymous type for them.

Interestingly enough, F# supports tuples, though it a bit of a hack. Since the CLR doesn't support tuples, F# basically defines different Tuple classes for up to seven tuple parameters (i.e. Tuple<t1,t2,t3,t4,t5,t6,t7>), For .NET 1.x, it's even worse - they have to define different type names (Tuple2, Tuple3, etc). Ugh.

Update - Robert Pickering pointed out that F#'s tuple implementation is entirely transparent inside of F#. He's right - I was writing from the perspective of a C# developer using F#'s implementation of tuples. Maybe I need to be looking closer at F#?

Modular Compilers

During Lang.NET, I ended up sitting next to Hua Ming, who's been working on the .NET Classbox project I wrote about previously. .NET Classbox introduces a new syntax for "using" to C# - basically, you can use individual classes as well as whole namespaces, and you can extend the individual classes you use. Obviously, that meant having a custom compiler that was 99% vanilla C# + the extra classbox syntax. Rather than building a C# compiler from scratch, the Classbox project extended the Mono Project C# compiler. Hua described the process as taking a "huge amount of time" and he described the compiler as "a monster". Now, I'm not trying to knock Mono here, I imagine our C# compiler is just as hard to work with. SSCLI's C# compiler directory is 5.5MB of source code alone spread across 126 .h and 68 .cpp files.

Is it just me, or does it seem crazy to have to muck about with such a large code base in order to add a relatively simple language feature? What I'd like to see is a more modular way of building compilers, so that integrating a small language feature like classbox would be a small amount of effort.

Of course, there is some work that's been done in this space. MS Research had a Research C# compiler paper, but it's three years old and one of the two authors has moved on to a cool product group job. I also discovered SUIF and the National Compiler Infrastructure Project, but these don't look like they've been updated in a while.

I like the model that the Research C# compiler proposes. Basically, it looks like this:

1. Specify the grammar in a modular way. In the paper, the grammar is specified in an Excel file, and you can use multiple files in a modular fashion. i.e. have one file for the core language and another for the extensions.
2. Late bind a grammar production to an action. Typically, in a lex/yacc style scenario, you embed the action code for a given production directly into the grammar, which makes it extremely hard to extend the existing syntax. In the paper, each production is linked with an instance of a type, so swapping out a new type would seem to be possible.
3. Generate an abstract syntax tree, that gets processed by multiple visitors. From the paper, the compiler has broken the "traditional" compiler steps - bind, typecheck, rewrite and generate binary (in this case IL) - into separate visitors. That makes adding extra steps or chaning existing steps fairly straightforward.

The only think I don't like about this specific approach is their Excel file based parser generator. It's a huge step beyond the LEX/YACC approach as it is scanner-less (having separate scanner and parser steps kills any chance of modularity) but it still has to deal with ambiguous grammars. Personally, I've been looking at Parsing Expression Grammars in part because they aren't ambiguous. For programming lanugages, support ambiguity in the grammar is a bug, not a feature.

Lang.NET Is Helping Game Developers

Back at POPL 06, Tim Sweeny of Epic Games delivered a talk titled "The Next Mainstream Programming Language: A Game Developer’s Perspective". I imagine he was a little too busy getting Gears of War out the door to attend the Lang.NET Symposium. Too bad, as there were interesting solutions presented that solved two of the issues Tim identified in his his POPL talk.

One of the issues Tim identified is one of Modularity. Gears of War uses the Unreal Engine 3. In other words, UE3 is a game framework and GoW uses that framework. As you might expect, this framework is exposed as a hierarchy of objects. Tim's example had "Actor" as the base class in the framework hierarchy, with classes like "Player", "Enemy" and "InventoryItem" inheriting from "Actor". Then he had game-specific classes like "Dragon" and "Sword" inheriting from the generic "Enemy" class. The problem is that game developers also need to extend the functionality of the framework's base classes. That is, they need a game-specific version of "Actor" or "InventoryItem" in addition to the game specific subclasses like "Dragon" and "Sword". Unfortunately, the current generation of languages don't support this, so game developers often clone the entire framework, which is error-prone and hard to support.

At Lang.NET, Professor Markus Lumpe demonstrated an implementation of the Classbox concept for .NET. Classbox is essentially a solution to the modularity problem Tim identified. They've modified C#'s using syntax to apply to individual classes. When using a class in this fashion, you can add extensions to it like new methods and new fields. I'm not sure the scope of these extensions - whether it's the file with the using clause or the containing assembly - but it's key to realize this is a local extension. The original framework isn't modified at all. Within you assembly, the metadata for the extended classes is re-written to include the new extension. So to use Tim's example, if you extended the framework's "Actor" class, it would create a YourGame.Actor class that inherited from the Framework.Actor and contained your extensions. Then it would re-write the inheritance metadata (again, only for your assembly) so classes that inherited from Framework.Actor such as Framework.Enemy and Framework.InventoryItem now inherit from YourGame.Actor.

Now I'm sure there are some nefarious uses of this type of inheritance tree hacking. But there are scenarios such as Tim's Gaming Framework example where this behavior is exactly what you want. I spoke briefly to Markus and at length with Hua Ming, one of Markus' grad students, about perhaps having a keyword indicating that a class is "classbox enabled" rather than allowing any class to be classboxed in this way. Looking forward to their future work.

Another issue Tim identified was Reliability. He called this problem "If the compiler doesn’t beep, my program should work". He showed a very simple method to iterate an index array and transform the associated vertex from a vertex array by a provided matrix. A simple function - four lines of code. Yet, the compiler can't detect null pointer or out-of-bound array access. Adding code to check those runtime conditions would easily double or triple the length of the function. While modern managed languages (C#/VB/Java) have made great strides in eliminating "random memory overwrites" (via type safety) and "memory leaks" (via garbage collection) they don't help you with these other types of dynamic failures.

At Lang.NET, Microsoft Researcher Mike Barnett demonstrated Spec#. Spec# is a superset of C# that solves these and other types of dynamic errors. The idea, in Mike's words, is to better bridge developer intent and code by embedding certain specifications into the code itself. Furthermore, it uses a combination of static and data flow analysis to detect the types of dynamic errors Tim described in his talk. So if you took Tim's simple transform function and fed it into the Spec# compiler, it would warn you of the possible null pointer dereferences. Furthermore, you can eliminate this warning by specifying the caller never pass you a null pointer. This is simply accomplished by adding an exclamation point to the type declaration. In other words, the vertex array method parameter would be declared "Vertex[]! vertices" to indicate you can't pass in a null array. With Spec#, you can also specify method pre and post conditions, which can solve the out-of-bound array access issue, as well as object invariants, which can specify the valid states an object instance can be in.

I didn't see Tim give this presentation, I only saw the slides after the fact. But I get the feeling that one of Tim's points is that game development is extremely cutting edge, and the issues they're running into now will be mainstream issues in a few years. Good to see language researchers are already well on their way to solving these issues.

The only thing I worry about is when will these ideas make it into mainstream languages? And will they be extensions to existing languages - i.e. will C# 4.0 and VB 10 include classboxing and specifications - or will they be entirely new languages? How much can you improve a language by adding features until it collapses under it's own weight?

More on Lang.NET

Jason Bock left me a comment that he's covering Lang.NET over at his .NET Languages site. His coverage of day one is here. Looking forward to his coverage of day two and three!

Lang .NET 2006 Symposium

Yesterday, I attended the Lang .NET 2006 Symposium - basically a public version of the CLR Compiler Lab I went to back in March. Unfortunately, with my new job, I couldn't attend all three days, but I did attend day one. Here we're my thoughts on the various sessions.

Anders Hejlsberg - LINQ and C# 3.0

• This was basically a rehash of his talk from the March Compiler lab. Makes sense as it was a new audience, but the "Query the Running Processes" demo is getting pretty old. Check out my notes from March for more details.

John Gough - Dealing with Ruby on the CLR

• John is a professor from the Programming Languages and Systems group at Queensland University of Technology. They're the ones building Ruby.NET. He's also the author of Compiling for the .NET Common Language Runtime, a great if somewhat dated (i.e. .NET 1.0) book.
• Much of John's talk covered the ground that Jim Hugunin covered back in March around the difficulties of mapping dynamic languages to the static CLR. For example, most Ruby.NET objects are instances of Ruby.Object, with their link to a class - a Ruby.Class - managed by the Ruby.NET runtime rather than leveraging the CLR's built-in class structure.
• He didn't spend much time talking about the really hard problems like continuations, which I was really hoping he would.
• There are a series of "allied" tools coming out of this project which look really interesting in their own right:
• PE File Reader/Writer - a managed component for reading writing DLL and EXE files.
• Gardens Point Parser Generator (GPPG) - a Yacc/Bison style parser generator, written in and generating C#
• Gardens Point LEX (GPLEX) - companion to GPPG for generating C# scanners, a la LEX or Flex. Not released yet, but John indicated it would be available in the next couple of weeks.

Christopher Diggins - Cat Programming Language: A Functional Optimization Framework for the CLI

• I'm fairly sure Christopher doesn't present often. Otherwise he would have know that there's no way to present 107 slides in 30 minutes.
• Christopher had a hard time expressing why someone would use Cat, even when asked point blank by an audience member. Most of his 107 slides were describing various features of the language. I don't know about the rest of the audience, but I got lost pretty quickly.
• It's too bad Christopher was so obtuse as a speaker, as Cat seemed pretty interesting. If you skip the first 78 slides (!) of his deck, you get to a slide named "Transformation Engine" which seems to be the primary reason for Cat's existence. The idea seems to be to build a large number (Chris said potentially thousands) of little optimization transformations which are used to "prune" the tree during the binary generation stage of a compiler.
• The only problem with this (other than the difficulty of following the presentation) is that I don't think compiler optimization is a particularly useful area of study. I subscribe to "Proebsting's Law" on this one: "Advances in compiler optimizations double computing power every 18 years." This implies that programmer productivity is far more important than compiler optimization. Ruby is the latest great example of this phenomenon.

Mark Cooper - Page XML : An XML based domain specific language for developing web applications

• Page XML is a DSL for building web apps. Unfortunately, it isn't released yet and it was hard to get a sense of what a solution built with Page XML would look like from the individual features described on slides. But I was certainly intrigued.
• As a DSL, Page XML needs to encode domain-specific abstraction. One example they provided that I thought was cool was their URL handling. Good URL design is an important usability feature. URLs in PageXML are split into constant and variable parts, so in a URL like mysite.com/news/somechannel/4, the "somechannel" and the "4" would be variable parts that would map into parameters that are passed to a page handler. Very cool.
• There were a large number of what felt like small and simple yet eminently usable features. Too many for me to write down.
• The only think I didn't like is the use of XML. No only are domain specific concepts like URLs encoded in XML, but also relatively mundane things like loops and if statements. This gets ugly really quickly. I imagine, the creators of Page XML did this so they wouldn't have to build their own parser, but it really hurts the usability of the language.
• The last point really points to the need for a simple meta-language - a language for building languages. Lex/Yacc and their derivatives just don't cut it. Ruby is good for building internal DSLs, but I'd like something faster and amenable to static typing as well as something more light weight for building external DSLs.

This post is long enough and I have "real" work to do (the downside of leaving evangelism! :) ). I'll post about the afternoon sessions later.

ChucK Audio Programming Language

I saw reference to ChucK on Prompt Criticality, a Second Life Blog. I wanted to blog it for my own reference if for no other reason. ChucK is "a new audio programming language for real-time synthesis, composition, and performance" that "presents a new time-based concurrent programming model, which supports a more precise and fundamental level of expressiveness, as well as multiple, simultaneous, dynamic control rates, a precise and straightforward concurrency, and the ability to add, remove, and modify code, on-the-fly, while the program is running, without stopping or restarting".

Sounds facinating, esp. given my recent discussion of internal vs. external DSLs w/ Neal Ford. Plus it deals w/ concurrency and is dynamic.

Lang.NET 2006

Erik Meijer just posted details about Lang .NET 2006 over on Lambda the Ultimate. Looks to be the next generation of the Complier Dev Lab I attended last month. The appear to have opened up the program significantly, and are asking for abstracts for both 30 minute talks as well as 10 minute "lightning" talks. If you're interested in submitting, here's the list of topics they are most interested in:

• Dynamic languages and scripting
• AJAX and ATLAS
• Domain specific languages
• Functional languages
• Object-oriented and aspect-oriented programming
• Web-services and mobile code
• Libraries
• Language-Integrated Query (LINQ)
• Compiler frameworks
• Garbage collection
• JIT compilation
• Visual Programming
• Success and failure stories
• Non-standard language features and implementation techniques
• Tools and IDE support

Talking Dynamic Lanugages with Neal Ford

I spent a couple of hours chatting with Neal Ford from ThoughtWorks yesterday. Ted Neward had virtually introduced us a few months ago and he was in town for MTS, so he arranged a meeting. I had asked Ted to introduce me to some dynamic language folks for some research and public debate purposes, and Neal was one of the people he hooked me up with. Unfortunately, this was right before I changed roles and got real busy. Of course, dynamic languages in general and Ruby in particular plays a large role in Edge Architecture, so I’m thankful Neal took the time to drop me a line and meet with me.

Above all else, talking to Neal made me realize that I just don’t know enough about dynamic languages, which limits my ability to discuss them. To date, I’ve flirted with them, but haven’t made a real commitment. For example, I’ve played around with Instant Rails, but hadn’t actually installed Ruby yet. It was time to re-image my dev partition anyway, so I’m going to try using Ruby exclusively for a while.

Here’s a brain dump of some of what we talked about. Not sure what it all means yet, so I’ll try and refrain from making commentary.

• Hungarian notation for interfaces (i.e. ISomething) is a big code smell. This has nothing really to do with Ruby or dynamic languages, but it’s an important point that I wanted to include here. Neal’s point is that the interface defines the semantics of the type and the concrete class is an “implementation detail”. In other words, contract-first isn’t just for web services. Apparently, ThoughtWorks doesn’t use ADO.NET directly primarily because the interfaces “aren’t pervasive enough” and are difficult to mock out. Also, they’re using Rhino Mocks which I wasn’t previously aware of.
• For all the debate about static vs. dynamic languages, it seems like the value Ruby brings is in meta-programming rather than dynamic typing. Certainly, that’s one of the big differentiators for Ruby vs. other dynamic languages like Python. While Rails has pushed the popularity of Ruby thru the roof recently, Neal seems much more enamored with Ruby than Rails.
• There is an even bigger gulf between dynamic and static typing proponents than I had thought. I brought up Singularity, which uses static typing exclusively to deliver a provably dependable system. Neal disagreed with that approach, pointing out that “tests are the best way of encoding the specification of the system” rather than compile time checking. Given my lack of expertise in this space, I’m withholding comment (for now) but I’m guessing the truth is somewhere in the middle.
• However, while the dynamic vs. static typing gulf is big, meta-programming is potentially the bridge. I don’t believe meta-programming is exclusive to dynamic languages. Certainly, some of the new features in the “Orcas” versions of C# and VB bring more expressiveness to the languages while still remaining type safe.
• All this meta-programming leads to domain specific languages. Ruby has strong support what Martin Fowler called “internal DSLs”, but Neal thought over time the focus would shift to external DSLs as they are more expressive and not constrained by the semantics of an existing language. Obviously, we’re pretty heavily focused on DSLs. However, Neal did think our focus on graphical DSLs is misplaced. He called them a “hangover” from CASE/UML tools. He rightfully pointed out that “business analysis speak English”.

All in all, it was time well spent. Neal, I hope we can pick up the conversation again sometime.

The Annoying C# 3.0 "Unsupported Version" Dialog

I am sick and tired of the warning dialog box for the C# 3.0 preview. Every time you launch VS it pops up a dialog reading "This is an unsupported version of Microsoft Visual C# 3.0. As such many features may not work as expected." You know, this isn't my first time working with alpha or beta quality software. I'm also running WinFX Feb CTP, VB 9.0 Jan CTP and the Expression Interactive Designer March CTP. None of them feel the need to constantly remind me that they're unsupported.

Who suddenly thought C# developers need hand holding?

Compiler Dev Lab - Scripting

Day Two of the Compiler Dev Lab was all about scripting. Iron Python was the primary focus of the day, but they also had Phalanger (Managed PHP) and Monad folks there as well.

• I hadn't realized just how performant these dynamic languages are on the CLR when compared to their native versions. The original version of Iron Python was 1.7x faster than the standard C implementation back in the summer of '04. Now with CLR 2.0, that version is now 2x faster with out any code changes. The Phalanger folks said they are 2.5x faster than the native version of PHP (1.7x faster than PHP + the Zend Optimizer). That's pretty impressive performance.
• The IronPython folks are heavy users of the new DynamicMethod class from .NET 2.0. Otherwise known as Lightweight Code Generation, DynamicMethod allows you emit a static function but have it get garbage collected when it's no longer needed. IP almost never generates new classes, since new types can't be garbage collected. The only times they generate actual classes are when you inherit from an existing .NET class or when you generate a new delegate type.
• It's really hard to serve the dual masters of both the existing language community and the .NET community. Jim Hugunin used the example of String.Trim(). A .NET developer would expect String.Trim() to "just work". A Python developer would expect that to throw an AttributeError exception (the Python equivalent of Trim is strip). How do you handle this? In IP, it defaults to pure Python mode, but if you enter "import clr", you move into .NET hybrid mode.
• One of the typical features of dynamic languages is the ability to change the base class of an object on the fly. Jim demoed this with WPF. He created a class that inherited from one type of panel and then set the __class__ property of the object to a different panel and the display changed immediately. Freaky, but cool.
• Jim showed a demo of a WPF app that hosted Python for extensibility. One of the scripts in turn hosted Python to create an interactive console for the app. Having a scripting engine that can host itself is awesome.
• The VSIP SDK CTP (reg required) includes an sample lanugage integration project for Iron Python. So you can get both the source into IP language itself as well as the source to the integration into Visual Studio.
• I got an email yesterday from someone asking about the possibility of Visual Ruby.NET. I haven't heard anything about it, but it would be cool to see Ruby on Rails runing under CLR. John Lam is working on RubyCLR, but my understanding is that is a bridge between the CLR and the Ruby runtime, not a CLR implemenation of the Ruby runtime. (IP is a CLR implementation of the Python runtime.) I'm thinking that there are some similarities between Ruby and Python, so having the source of IronPython would be a huge help in building a Visual Ruby implementation. For example, both Ruby and Python have closures. IP has a FunctionEnvironment class which is used to lift stack variables onto the heap in a variety of scenarios, including closures. So if I was building Visual Ruby, having access to the FunctionEnvironment class would be a good start.
• I said yesterday that I need to learn more about F#. They showed a video of an internal F# presentation, but I spent most of my time cracking jokes with Sam Gentile who's in town for an SC-BAT workshop.
• I didn't pay enough attention to the Monad presentation. :(

Compiler Dev Lab - LINQ

Even though I haven't finished my ETech postings, I'm already onto another event. This week, thanks to an invite from Michael Lehman, I'm sitting in on a Compiler Lab discussing implementing other languages for CLR. The first day was about LINQ. Much of the info is rehashed from PDC or the docs up on MSDN. However, I have learned a few new things.

• One of the standard features of LINQ is Extension Methods. That enables you to declare a static method like "static void Foo(this string source)" and then use it like "stringvar.Foo()". Apparently, they are considering adding other types of extension members including properties and fields. The idea of extension fields is somewhat scary but powerful.
• LINQ uses something Anders called deferred query execution. The query isn't executed until the values are asked for (typically by calling foreach on the query). That means you can compose queries to your hearts content with no perf impact until you actually invoke the query.
• Query Comprehensions in C# and VB is a pattern implementation in a similar vein to foreach. Foreach is relatively simple shorthand for iterating through an collection by calling IEnumerator.MoveNext until it returns false. While LINQ enables arbitrary composition of queries, there is obvious gravitational pull towards the SELECT / FROM / WHERE / ORDER BY / GROUP BY approach favored by SQL. So if you build your own query operator, you can include it in a LINQ query, but C# and VB won't be able to include it in the Query Comprehension syntax. Probably not a big deal, given the breadth of standard query operators as well as the deferred query execution, but it's good to understand how the abstraction works.
• I want to know more about how DLinq is implemented. I've been refining my thinking about data since working with Ning's content store and I'm convinced of the need for a simplified datastore. SQL is designed for significantly complex database schemas, which means a significantly complex development environment.
• I'm looking much more closely at VB, given the new features in VB 9.0. Not only the LINQ stuff from C# like type inference, extension methods and anonymous types but also VB specific stuff like XML Literals and Duck Typing. Combined with VB's existing support for late binding, there are compelling features to make VB attractive over C#.
• I've been hanging out with Brian Beckman. He's a hoot.
• I think I need to take a deeper look at F#.

The Next Mainstream Programming Language

Terra Nova is not the usual place I go to get news around programming language improvements. But they linked to a great presentation from POPL 2006 by Tim Sweeney of Epic Games. Tim's talk is called The Next Mainstream Programming Language: A Game Developer's Perspective and it talks at great length the major issues facing game developers today. As Nate Combs at Terra Nova remarked, most of these issues are not specific to the game industry, but will likely be seen there first.

Most interesting (to me) was the issue of concurrency. Tim uses Gears of War for all his examples. Of course, Gears of War is an Xbox 360 exclusive. Xbox 360, as many of you probably know, has three hyper-threaded CPUs for a total capactiy of six hardware threads. Herb Sutter talked about this in his DDJ article The Free Lunch Is Over. Tim points out - rightly so - that "C++ is ill-equipped for concurrency". C#, Java and VB aren't much better. Tim conculdes that we'll need a combination of effects-free non-imperative code (which can safely be executed in parallel) and software transactional memory (to manage parallel modifications to system state).

Tim also touches on topics of performance, modularity and reliability. And he has an eye on the practical at all times. For example, he points out that even a four times performance overhead of software transactional memory is acceptable, if it allows the code to scale to many threads.

Anyway, it's a great read so check it out. Also, MS Research has a software transactional memory project you can download if you're so inclined.

Not Your Father's C++

Certainly not my father's C++. I sat thru a presenation on VC++ 2005 today. Wow, I hadn't realized all the coolness there.

First off, all the syntax is working it's way thru the standards bodies, so no more underscore underscore syntax. It makes the code somewhat easier to read, but more importantly it's following a similar standardization process to CLI and C#.

Secondly, you can now use all the native C++ features (templates, multiple inheritance, buffer overrun protection, etc) and all the CLI features (garbage collection, generics, language interop) together. Previously, you had very limited choices for mixing the two coding idioms. No longer - go ahead and mix and match. This gives you the best of both worlds. Use templates, and expose them to other .NET languages as generics.

Finally, it brings deterministic finalization to .NET. In VC++ 2005, you can declare both a destructor (used when a class goes out of scope or is explicitly deleted) and a finalizer (used when the class is garbage collected). This is similar to the whole IDisposable approach for classes that wrap unmanaged resources (file handles, network sockets, etc). Actually, it's identical to IDisposable because that's how it's implemented! And it works both ways - if you instance a managed class that implements IDisposable "on the heap" then it will automatically call dispose at the end of scope. For example:

{  //C++ Version
   FileStream fs = FileStream(path, FileMode::Create); 
   fs.Read(...);
   fs.Write(...);
}  //fs.Dispose called automatically

Even though the FileStream is implemented in C#, it behaves here like a stack type and is destructed as you would expect. In C#, you'd have to use a using statement to achieve the same effect. For this trival example, it's not that big a deal. But if you have multiple stack instances created at different times within a scope, this helps out immensely.

Not sure I would move to C++ for all my managed programming, but I'll certainly be giving VC++ 2005 another look.

Dare on C-Omega

I just read Dare's awesome article on C-Omega. If I had any time at all, I'd experiment with it to rewrite dasBlog (well, at least the back end parts of dasBlog anyway). Of course, I don't have any time at all, so it's not going to happen (well, at least I won't make it happen)

Concurrency: Next New Major Language Feature?

Several people have pointed out Herb Sutter's great article on concurrency entitled The Free Lunch Is Over. When I blogged last week about new possible features of "full-grown" OO languages I mentioned dynamic typing but I didn't think about concurrency. I think Herb is right: "programming languages...will increasingly be forced to deal well with concurrency" as applications get more CPU bound. Maybe I need to take another look at Comega (or Cw). Cw extends C# in two areas - data typing/querying and concurrency. The concurrency extension used to be called Polyphonic C#, but the name got changed when it merged with Xen/X#. (BTW, there's a new Cw release (v1.0.2) but no specifics as to changes other than no longer needed VS.NET 2003 to be installed in order to use it.)

Cw adds the idea of asynchronous methods and something called chords - sets of methods with the same method body. The chord method body in only executed when all the associated methods have been called. In the simple buffer tutorial, the buffer class has a synchronous Get method and asynchronous Put method. If you call Get before Put, it blocks until Put is called, then the method body is executed. If you call Put before Get, then the Put call returns immediately (it is async after all) but the call is queued so that when Get is called, the method body is executed immediately. FYI, the Cw docs have a variety of other tutorials of async methods and chords.

BTW, speaking of my post on full grown OO languages...My father suggested that I not jump to conclusions regarding the X-develop's support for what they term "toy languages or little domain specific languages". In fact, Hans Kratz of Omnicore (which makes X-develop) had this to say:

This comment on our website was not intended to bash DSLs at all. Instead we wanted to make clear that the plugin API in X-develop is powerful enough to allow integrating support for "full-grown" languages without placing arbitrary restrictions on language complexity.

For a language developer/integrator this is a plus regardless if he wants to integrate support for a DSL or "full-grown" programming language.

Makes sense. Maybe I was just too sensitive to the use of the word "toy" so close in proximity to "DSL". Sorry about that Hans. 

No Time To Experiment, So I'm Reading About Cw

COmega (otherwise known as Cw since most people don't have an omega key on their keyboard) is on a long list of stuff for me to look at. But instead of actually coding with it, so far I can just read Steve Maine's blog. He's got interesting posts on syncronization and streams, the two big features of Cw over C# (come to think of it, we use the “#” symbol as most people don't have an actual sharp key on their keyboard). I also learned from Steve that Cw comes with basic VS integration - you get project support, syntax highlighting and some Intellisense. Now I just need a few extra hours in the day.

Service Oriented Language

I like Jeff's suggestions for a Service Oriented Lanugage as well as RebelGeekz' idea of how to include XML Infoset information in a language without resorting to angle brackets. I wonder what a community developed lanugauge would look like? I've experimented with Coco/R which really reduces the effort it takes to build a parser. Of course, the language design and code generation are the hard parts!

Source for Coco/R is available on the Rotor Community Projects page. Other cool projects include a port of Rotor to WinCE, two different AOP projects - one for C# and the other language neutral and a set of improved collections.

Objects vs. Data

Christmas was crazy (my wife's blog has the details), so I didn't get a chance to blog about Sean & Scott's desire for unification of objects and data, Erik Meijer's two papers on the topic, or Dare's responses to both. Programming language evolution is something I've keenly interested in. When I first started this blog almost a year ago, one of my early posts was about a disruptive programming language technologies presentation from MS Research. Among the areas mentioned for improvement: were database integration and XML manipulation. Sounds like the Xen language demoed @ XML 2003 is a step in that direction.

When you read Erik's papers, you'll notice that one of the goals is to natively integrate XML into the language. He writes: "In our approach XML documents or document fragments become first class citizens." What's interesting about that is that if you take XML to mean pure-infoset-data (as opposed to angle-bracket-serialization-format) then you can argue that Data (with a capital D) is not a first class citizen of today's Object-Oriented languages. Given that a lot of OO code has been written to manipulate data, having a language that explicitly distinguished between the two could be valuable (assuming it made the programming easier and the programmer more productive).

Note, this is the opposite approach from tools like O/R mappers and XSD.exe which attempt to hide the differences from the programmer. We've seen a similar evolution in the way we think about invoking objects across the network. Tools like DCOM and .NET Remoting attempt to hide the RPC and make it appear as a local call. But as the thinking evolves, tools like Indigo is designed to make the boundaries across apps and machines explicit. Initial thoughts on data access were to make it all look like objects (i.e. O/R mapping). But as the thinking evolves, maybe we need to make the boundaries between objects and data explicit (and easy) as well.