I presented Software Factories for the first time today and I think I did a pretty good job. We had some architects were in town from a new managed SI partner and they wanted to discuss modeling. They are (were?) an IBM partner, so they’re a big WebSphere shop. They’re also XDE users, so I laid out the Software Factories concept as well as the modeling tools that are coming in VS2005. They seemed pretty impressed. Of course, they’re having what I expect is a typical experience with UML tools – they use XDE for documentation and communication only (i.e. UmlAsSketch). They don’t even try to generate code from the models anymore.

To help explain the Factories concept, I used Steve Maine’s Efficiency/Precision/Generality modeling approach (and gave him credit for it, of course. Steve, I will find a way to properly thank you for that brain.save) as well as my own ideas about VB as a Software Factory. Both worked out very well to help communicate the goals of the Factories approach, though I could refine the delivery quite a bit. I also talked about the Evolving Frameworks Pattern Language, which Jack outlines in the JOURNAL factories article this way:

• After developing a number of systems in a given problem domain, we identify a set of reusable abstractions for that domain, and then we document a set of patterns for using those abstractions.
• We then develop a runtime, such as a framework or server, to codify the abstractions and patterns. This lets us build systems in the domain by instantiating, adapting, configuring, and assembling components defined by the runtime.
• We then define a language and build tools that support the language, such as editors, compilers, and debuggers, to automate the assembly process. This helps us respond faster to changing requirements, since part of the implementation is generated, and can be easily changed.

The problem is that each of these steps is much much harder than the preceding ones. Identifying problem domain abstractions & patterns is something that most organizations are already doing, even if they don’t do it explicitly today. Codifying those abstractions in a reusable framework is much harder, primarily because it’s hard to think through all the usage variations a framework may experience. Still, many companies have the skills to develop reusable frameworks. However few companies have the language and tool development experience to make investing in custom tools cost effective. For example, even though Gregor and Bobby have defined a language for integration patterns, the only tooling available is a Visio stencil.

For Software Factories to work, we need to make it much easier to build domain specific languages and modeling environments. This is one of the big hurdles to adopting the factories approach today.

Denny Figuerres left the following comment on my last post:

[O]ne of my “wish” items would be a kind of editor that would be a merge of flow-chart and text editor so that I could view my function as text or as a kind of zooming diagram with more details as I drilldown. like when you use a map program, first you see an area and major routes, rivers, lakes etc… and as you zoom-in you see a smaller area with more detail. at some point you are down to a single line of code that is an expression of some kind.

I think this is a great idea, and is completely in line with Software Factories. Denny, what you’re talking about is working at higher levels of abstractions. If you read the Software Factories article on TheServerSide.NET, Jack writes:

How…do we work at higher levels of abstraction? We use more abstract models, and move the platform closer to the models with either frameworks or transformations, as illustrated in Figure 4.

• We can use a framework to implement higher level abstractions that appear in the models, and use the models to generate snippets of code at framework extension points. Conversely, the models help users complete the framework by visualizing framework concepts, and exposing its extension points in intuitive ways. A pattern language can be used instead of a framework, as described by Hohpe and Woolf. This requires the tool to generate the pattern implementations, in addition to the completion code. This approach is illustrated in Figure 4 (a).
• Instead of a framework or pattern language, we can generate to a lower level DSL. We can also use more than two DSLs to span a wide gap, leading to progressive transformation, where models written using the highest level DSLs are transformed into executables through a series of refinements, as shown in Figure 4 (b).

I think we’ll see a combination of these two approaches. Obviously, as an industry, we have lots of experience building frameworks and I don’t see those going away anytime soon. The second approach, however, is much more fascinating as you get that “zooming” effect that Denny describes.

One thing to note about this zooming approach of using higher level abstractions – different systems use different abstractions. The abstractions you use to build an ERP system are not the same as the ones you would use to build a telephone billing system. So the view of the top-level abstractions will be very different, even if they both end up implemented on the same platform using the same programming language.

Steve’s post on “the modeling problem” hits the nail on the head. We’re all familiar with the concept of “fast, good, cheap – pick two”. Steve breaks down modeling into “general, precise, efficient – pick two (and favor one)”. Furthermore, you can’t have a language that is both general and precise. UML takes what Steve calls the “Favor efficiency, accept generality and compromise precision” approach:

The UML metamodel is flexible enough to allow it to describe virtually any system out there. However, from a formal semantic perspective, the resultant model is gooey and formless which makes it very difficult to compile into anything useful. At best, we can get some approximation of the underlying system via codegen, but even the best UML tools only generate a fraction of the code required to fully realize the model. The lack of precision within the model itself requires operating in both the model domain and the system domain, and implies that some facility exist to synchronize the two. Thus, the imprecision of UML forces us to solve the round-tripping/decompilation problem with 100% fidelity, which is generally difficult to do.

Software Factories, on the other hand, takes what he calls the “Favor efficiency, accept precision, and compromise generality” approach:

This, I think, it the sweet spot for Microsoft’s vision of Software Factories. Here’s why: the classic problem faced by modeling languages is Turing equivalency. How do you model a language that is Turing-complete in one that’s not without sacrificing something? The answer is: you don’t. You can either make the modeling language itself Turing-complete (which sacrifices efficiency) or you can limit the scope of the problem by confining yourself to modeling only a specific subset of the things that be expressed in the underlying system domain. Within that subset, it might be possible to model things extremely precisely, but that precision can only be gained by first throwing out the idea that you’re going to be able to efficiently and precisely model everything.

When describing Software Factories, I have two analogies that I use to explain the idea. The first is the “houses in my neighborhood” example I blogged before. That does a good job describing economies of scope, but doesn’t really cover the modeling aspect of software factories. Talking about how you model cars or skyscrapers doesn’t really capture the essence of software modeling – you don’t generate the construction plans from a scale model of a skyscraper. However, it turns out that all developers have at least a passing familiarity with my second analogy: Visual Basic, the most popular DSL and modeling tool of all time (so far).

The original Visual Basic was a rudimentary software factory for building “form-based windows apps”. (Today, VB.net has been generalized to support more problem domains) Like the factory approach that Steve describes, VB was very efficient, sufficiently precise, yet not particularly general (especially in the early years). There were entire domains of problems that you couldn’t build VB apps to solve. Yet, within those targets problem domains, VB was massively productive, because it provided both a domain specific language (DSL) as well as a modeling environment for that domain.

A DSL incorporates higher-order abstractions from a specific problem domain. In the case of VB, abstractions such as Form, Control and Event were incorporated directly into the language. This allowed developer to directly manipulate the relevant abstractions of the problem domain. Abstractions extraneous to the problem domain, such as pointers and objects in this case, got excluded, simplifying the language immensely. Both of these lead directly to productivity improvements while limiting the scope of the DSL to a particular problem domain.

In his post, Steve makes the point that it’s pointless to distinguish between modeling and programming languages. VB certainly blurred that line to the point of indistinguishably. Regardless, graphical languages are typically more compelling and productive than textual ones. It’s hard to argue with the productivity that VB form designer brought to the industry. Dragging and dropping controls to position them, double clicking on them to associate event handlers, changing properties in drop down boxes – these idioms have been widely implemented to the point that essentially all  UI platforms provide a drag-and-drop based modeler. It’s such a great design that 10 years later, UI modelers are essentially unchanged.

Once you realize that VB’s DSL and modeling environment was a rudimentary software factory, you realize that Software Factories methodology is about generalizing what VB accomplished – building tools that achieve large gains in efficiency by limiting generality. Since each of these tools focuses on a limited problem domain, you need different tools for different problem domains. The problem is that while building apps with VB may be easy, but building VB itself was not. Most enterprises have the expertise to develop abstractions in their domain of expertise and to codify those abstractions in frameworks, but very few can develop tools and DSLs for manipulating those frameworks. One of the goals of Software Factories (and VSTS Architect for that matter) is to make it easier to build tools that are really good at building a narrow range of applications.

Note, it’s important to note that the term “narrow range” is relative. Darrell seems to think narrow range only means vertical market applications that don’t “solve new and interesting problems”. It’s true that the narrower the range, the more productive the tool can be. But VB shows us that you can achieve large productivity gains while solving new and interesting problems even in broad scope problem domains.

I got my twin V300′s tonight. No sweat – only issue was figuring out which one was mine and which one was Julie’s (We don’t have exactly the same plans). Of course, they look identical. We’ve set the backgrounds to different images so we can tell by opening the phone who’s it is, but it would be really nice to have a different faceplate for at least one of them. Plus we need 2 car chargers and I’d like to get a data cable to sync my contact list. Ebay has a bunch of vendors like PhoneStations that offer these accessories cheap – car charger for 99 cents, faceplate for $2.97, data cable for $14.99. S&H is a little outragous – $15 for four items – but $35 total including S&H seems way reasonable, esp. when T-Mobile charges $35 for just the data cable. Anyone know anything about places to buy V300 accessories? What about applications for the V300? The data cable doesn’t come w/ any software. I found MotoCoder, but I don’t really have time to write my own apps for this phone (esp. in J2ME).

A while ago, Tom mentioned that I had a great customer service experience with T-Mobile. Truthfully, the great customer service experience came today. T-Mobile just earned a customer for…well, maybe not for life, but they certainly have earned enough karma to keep me around a while. And at the same time, Sprint has spurned a customer for life.

I blogged back in April about getting a SmartPhone. Well, the honeymoon was short lived – it was a developer beta unit and it was not particularly robust. I dropped it about a month ago and it died. No huge deal, the MPx220 is coming soon. Of course, I need a phone in the meantime. I had signed up for T-Mobile service to power the SmartPhone so I went back to pick up a V300 which would go to my wife when the MPx220 became available. Only trouble was that since I hadn’t bought the phone when I activated my account, I was not eligible for an activation discount. I had asked about that when I bought the service, and had been assured it was no big deal. So I call up customer service, and they promise to wave the disconnect fee on the current account if I buy a new phone (with the activation discount). This was right before I went to New Zealand, so I figured I’d take care of it when I got back. In the meantime, I’m using a old borrowed phone and my existing account.

So I’ve been back a week, but hadn’t gotten to taking care of the phone yet. Then last night, someone steals my phone and my wife’s phone out of the car. So I call T-Mobile and explain the situation. They cancel the account on the spot, outright – no early termination fee or anything. Pretty cool. Of course, I did tell them I was going to buy not one but two new V300′s from Amazon (for the low price of -$100 each after rebate) but still, I thought that was pretty cool. The only bummer of this whole scenario is that I won’t be able to get the MPx220 with an activation discount when it comes up.

After talking to T-Mobile, I called Sprint. Now, I’ve been a Sprint customer for five years. You would think that would warrant a little respect and/or leeway, but apparently not. The “customer service” rep I spoke to was both rude and stupid. He was able to suspend my wife’s phone, but when I asked to cancel it he told me there would be an early termination fee. Funny thing is, the term on this “contract” (which I never signed and was unaware of) end’s on the 28th of August. You know, next Saturday. As in, six days from today. You would think that you might give a customer of five years who is dealing with a case of theft the benefit of the doubt and wave the early termination fee less than a week before the contract expires anyway. No, apparently I have to call back Saturday to terminate the contract. Fine. I set a reminder on my calendar so I don’t forget to rid myself of Sprint for good and forever.