Monday, January 5, 2009

Some Personal Thoughts on Agile Development

Recently, I have been involved in a survey of development organizations, assessing their practices and how they are changing. It is not giving away too much, I hope, to say that the results were quite surprising to me. I never expected, for one thing, that the whole idea of “agile” development would have become so pervasive in so short a time – not only in the small innovation-seeking software companies but also in large non-software firms doing massive projects. Granted, it is more popular in the US than abroad, and granted, in a few cases “agile” seems more like a fuzzy marketing term slapped onto collaborative technologies. But time and again, users are getting specific – looking at variants of SCRUM and DSDM, using tools with Agile in the product name, mentioning extreme programming practices.
It is less surprising to me that agile programming is delivering results – consistently. You see, I’m a former programmer – although I haven’t done much if anything in the last 18 years. More specifically, I’m a graduate of Computer Corp. of America, whose Model 204 User Language taught me a lot about programming that neither schools nor other development shops came within a mile of suggesting.
At first glance, M204 UL seemed a bit like a third-generation language – say, PL/1. It had English-language variants of the usual C-like commands, plus string handling that was strongly reminiscent of SNOBOL. Ah, but then there were the data-access commands. These were superbly crafted to do what they said in an intuitive fashion, and to give the programmer full rein to optimize the code using simple, orthogonal statements. What do I mean by orthogonal? I mean that each different statement type covered a large, distinct set of operations, and together they covered everything “data” you wanted to do.
Now, what does this have to do with agile? Well, if you didn’t worry about little things like performance, you could rip off a program to do just about anything in nothing flat (remember, this is before GUIs). So some M204 UL experts did just that, traveling to user sites to customize like crazy. Iterative prototyping back at CCA was a way of life, not a whole new process.
The second thing that M204 UL taught me was that close ties with the user were good, not bad (up to a point; users couldn’t tell you where the market was going, just what they needed right then). I did several revs of an email product built using M204 UL, and what drove the specs for each rev was user input, because there just wasn’t enough bandwidth to do an elaborate, controlled development process. And guess what? The result was better, and was developed quicker, than just about anything else out there. As a one-person shop, I was turning out major revs every 6 months, if needed.
And finally, M204 UL and CCA taught me a great deal about the perils of over-controlling the development process. My last project was a development tool, done in parallel with a 9-person search-tool project. I was forced to sit in “design by committee” meetings all day, every day, for 5 weeks straight. The resulting file-search-tool design was thoroughly documented, bland, and highly complicated. I wrote out some hasty, vague specs for the development tool, because I wanted freedom to elaborate as I went. The result was that the search-tool team got kudos for their design, and I got put on probation – but they still needed me to do the development tool.
Well, to make a long story short, I finished that tool – a very original, easy-to-use one that sold just fine – in about 3 months, or shortly before I left. The search tool development was done in the same over-controlled manner that its design was, and after a year the project was cancelled – they just couldn’t get it to work. And that taught me that flexibility and room for change trumped control, at least for medium-scale projects – and, as we saw from the Internet, that goes for large-scale projects done iteratively and incrementally, as well. I’m not knocking control; but it has to be control in the service of flexibility, not preventing it.
So I learned, in my gut, that the kinds of ideas that agile is talking about now – fast “spiral” iterations, lots of user feedback, loosely-coupled programmers – could work. What about getting consistent results? Well, that’s another story – for the next blog.

No comments: