One thing that kind of bothered me in my recent CSM training course was a consultant's comment that he didn't feel any need to try to "sell" Scrum to resistant management, implying that, if they didn't buy into it, it was their loss -- but winning over management is almost certainly a big part of what developers in that organization are expecting a consultant to HELP THEM WITH!

Just as management needs some level of developer "buy-in" for a project to have any chance of success, anyone hoping to transition a traditional organization to Agile needs to be able to SELL the concept, in terms a traditional business manager can understand. "Just trust us" is the arrogant Agile counterpart to management's "Just do it" -- and about as effective.

Agile is a TOUGH SELL! You're asking what may be a whole STACK of non-programming middle managers to abandon established practices, to override PMP-influenced resistance (which may finally be diminishing, now that the Project Management Institute has begun to take an interest in Agile), to buy into subversive concepts like "self-organizing teams", based on rationale they typically don't have the first-hand development experience to understand... knowing that if it all goes wrong, THEY'RE the ones who'll be held accountable.

Emphasizing Agile's ability to deliver better quality software, much faster, using fewer resources (better, faster, cheaper) is critical to convincing management to at least give Agile a TRY, but avoiding an outcome that permanently DISCREDITS Agile requires knowing where you can compromise the "purity" of your model, while holding the line on items necessary for Agile to actually produce benefits. Setting up a system of code review and mentoring, to ensure code is kept clean, maintainable, and extensible, is more important than insisting a large team of developers with poor dev. skills write sloppy unit tests to support their convoluted code. Generating some amount of unecessary documentation is preferrable to having your product fail a customer audit, or to have your project remain unfunded by corporate managers who don't buy into your "trust us" argument.

To offer reassurance and manage expectations, you'll need to provide justification for the changes that will need to take place, be able to offer credible alternatives to traditional methods of maintaining accountability, and openly acknowledge potential areas of conflict (for example, Agile organizations typically require a smaller number of more highly-skilled developers, which may eventually lead to issues of what to do with long-time employees who are under-performing in a department which is over-staffed, under the new standard).

By choosing your battles, and focusing on a few key issues like obsessive attention to code quality, replacing Big Design Up-Front with a commitment to iterative "refinement" of an initial outline spec., sqeezing in at least SOME amount of regular customer proxy feeback (usually from a busy marketing rep. who doesn't understand why you can't just go off and build the entire app. as specified), limiting the number of items under development to force developers out of their silos and encourage them to collaborate, holding regular informal reviews to try to improve the dev. process based on observed results... you are surreptitiously "infecting" the organization with traces of Agile, so you can break down resistance and eventually take over the host.

Don't make the mistake of the last organization I was in, where we managed to get buy-in on adopting specific Agile PRACTICES (continuous integration, unit testing...), but failed utterly to get commitment to the more important Agile PRINCIPLES. Management still insisted that we begin a three year project by dividing all work into "tasks", which were distributed to 15+ developers in two globally-distributed teams to "estimate", despite the fact that only three of us had significant first-hand experience with the technologies we would be using, and many developers suffered from questionable coding skills... with disasterous results.

Kanban is interesting here again, because it seems better able to overcome initial management and developer resistence, by requiring only very minor changes to ANY existing development process (including waterfall), with teams reportedly adopting more and more agile techniques naturally, over the course of the project, as productivity improvements become obvious to everyone involved.

"The (Pirate's Code) is more what you'd call 'guidelines' than actual rules."

            - Captain Barbossa, Agile pragmatist, Pirates of the Carribean, 2003

One of the more confusing aspects of Scrum for me has been the fact that "adapting" Scrum to fit your team's specific circumstances is always encouraged... but doing something like applying the "Kanban" technique to Scrum upsets many traditionalists.

"The surprising thing for me is that many smart Agile people - people I know to be intelligent insightful people - seem bugged by Kanban... I'm seeing Agile people behave as strangely about Kanban as traditional process folks behaved about Agile. They seem threatened." -- Jeff Patton

I've had a hard time getting some Scrum fundamentalists to even DISCUSS why they feel certain parts of Scrum should be "off limits" to change -- a strange idea, considering that Scrum, as it is typically applied today, is already very different from the way it was originally described. This seems inconsistent with Agile principles that encourage change based on continued analysis and experimentation, especially since the majority of people applying Kanban seem to be experienced Scrum practitioners, who report great success with this Scrum + Kanban approach in solving real-world issues that their teams struggled with under "straight" Scrum.

There is a subculture of agilists who seem to promote an "all or nothing", "enlightened vs. heathen" attitude, reflected in statements along the lines of "if you're not doing X, Y, and Z (or if you try to add Q), you're doing it wrong and you're not Agile". Many senior Agile dev. community members have explicitly spoken out against this, as counter-productive and just plain wrong. Kent Beck included an apology in the preface to the second edition of his book, Extreme Programming Explained, specifically to try to address this for the XP community, and published works by other industry thought leaders including Mary Poppendieck, Alan Shalloway etc. have made similar points. IBM has pushed for ALL its developers to adopt SOME form of agile, while explicitly REJECTING the idea that you need to follow any specific set of practices or methodology.

The Dancing Agile Elephant: IBM Software Group's Transition to Lean and Agile Development

Having worked both in highly-productive start up environments, and some amazingly dysfunctional corporate beauracracies, I'm an absolute supporter of developing software in a way that's consistent with Agile PRINCIPLES, and it is frustrating to think, after finally convincing your corporate management to take a serious look at what Agile has to offer, they'll encounter a discussion space filled with a mixture of hippie-dippy idealism and bickering that's likely just to make them roll their eyes and dismiss the topic out-of-hand.

"Badges? We don't need no stinkin' badges!" - Blazing Saddles, 1974

I became a "Microsoft Certified Professional" (MCP) for .Net back in '03, got my MCP for WPF last summer, and just picked up a Certified ScrumMaster (CSM) badge, all of which means... SQUAT!

Any experienced developer can tell you an MCP certificate (or a CS degree) says absolutely NOTHING about whether someone can actually write production quality code or help a company ship products, and there's been some criticism over the way the term "Certified ScrumMaster" is sometimes promoted and abused. For the record, a Microsoft certification essentially proves you can memorize API trivia out of a study guide, and a CSM certification just says you've had two days of classroom exposure to the basic concepts of Scrum.

Of course, there's a LITTLE more to it than that. A Microsoft certification guarantees baseline familiarity with the recommended way to handle common Windows dev. tasks, and a CSM badge guarantees exposure to fundamental Agile concepts, along with a general idea of how to apply them, and obviously indicates a commitment to helping to refine a company's development process. When interviewing new hire candidates at my last three companies, I always asked about certification as evidence of some level of "devotion to craft", beyond simply working on "assigned features", but I was always more impressed if the candidate was familiar with notable developer community authors, or attended local dev. events.

From the job candidate perspective, certification is most useful in helping get past HR and other non-programming middle manager filters. From management's perspective, the risk in being overly impressed by a Microsoft certification is alot less than the risk of turning an inexperienced CSM loose on your enterprise organization's development process for your mission critical software. CSM training and resources often have a kind of naive idealism to them ("Let's play a game to illustrate collaboration!"), and I think a CSM who hasn't personally experienced harsh corporate realities is in danger of having management and technical staff quickly flip the "Bozo Bit" on them. For example, there's a HUGE difference in promoting "self-organizing teams" in an environment of top-notch, experienced, highly-motivated developers, versus in a bloated, complacent, corporate development environment.

I've been keeping up with Agile practices and approaches since the late '90's, and I'm currently most impressed by approaches that combine Scrum and Kanban, as described in Henrik Kniberg's newly-published "Kanban and Scrum, Making The Most of Both". The book is succinct and the clearest explanation I've come accross, of what all this "Kanban" stuff is that everyone keeps talking about, and how it relates to Scrum

Update (2/13): Scrum co-creator Ken Schwaber recently parted ways with the Scrum Alliance, and started Scrum.org to offer more rigorous developer training.

"Princeton could USE a man like Joel."

     - Hope for all us unconventional interview candidates, from "Risky Business" 1983

I've been living off my incredible shrinking stock portfolio, hanging out in local coffee shops and restaurants with my laptop, working with WPF and other development technologies...  for too long now, and I'm itching to APPLY these new technologies and practices, to something challenging and interesting, working with people who are "smart, and get things done".

In addition to pointing out that I'll be starting my job search in the depths of a nation-wide employment freeze, friends have predicted that hiring managers will reject me because of my recent "sabbatical", but I think any company that would consider intense work with next generation development technologies on my own dime to be a BAD thing is probably one I wouldn't want to work at anyway 

I'm obviously delusional enough to think I can still afford to be picky, and there are some situations I'm really hoping to AVOID this time around...

• The New Number 6:  "This 'best practices' stuff all sounds very interesting, but right now what we really need is for someone to take over John's code.  We're down to five developers now that John got fed up and quit, no one undertands his stuff, and marketing has commited us to delivering a big stack of new features in eight months."
• How Hard Could It Be?:  "We can't spare any of our hardware engineers to spend a weekend knocking out this 'simple' software, so we want to hire you.  We've already done the hard part of defining the database schema, and we just need you to add a few things and get together with marketing to slap on some kind of pretty UI."
• Brain Silos: "Joe is designing data access, Tim is handling remote comms, Fred is responsible for security... you don't need to concern yourself with those issues, and they don't need to be a part of discussions involving your area."
• Plug-and-Play Developers:  "We're going to need to pull you and Joe off the project for eight weeks to jump on a new marketing request.  Remedial Roger's not doing anything right now, so he can fill in until you get back, and then maybe we'll have you split your time 60/40 between projects X and Y..."
• Sacred Tablets:  "I'm not sure what you mean by 'Agile', but we follow the process described in this 5 inch thick binder, written ten years ago"
• Invertebrates:  "Yes, we've all been saying for years how we should be doing X, Y, and Z, but what can we do?"

I have this fantasy where I work in an environment where we avoid doing things we already know are dumb before we even start, and where we actually FIX things we discover we got wrong along the way.  I'm not holding out hope for an environment as enlightened as Fog Creek, but a company should fall within some REASONABLE RANGE on these issues.

Some organizations believe they "can't afford" to spend time refactoring their development process, especially in the current economic environment, when its probably the adoption of a more streamlined iterative process, featuring cross-functional teams, relying on a smaller number of more capable individuals, that is likely to have the most dramatic long-term positive impact on a company's software development costs, schedules, and quality.

I've come accross several good posts recently, reminding developers not to get sucked into a "Red Queen's Race", trying to keep up with every new development tool and technology, and every popular development author's latest book.

On the other hand, your job as a developer includes the responsibility to promote tools and practices that enable your company to create more reliable, more aesthetically pleasing versions of its software, delivered much more quickly -- and quite possibly requiring fewer (but better trained) developers.  Somehow, you have to sort through all the hype, to identify the subset of tools and practices that are actually destined to become industry standards, and you need to understand these technologies well enough (and early enough) to design your CURRENT code and architecture in a way that will make eventual migration to the new standard as painless as possible.

You'd think, with several months free of any of those pesky JOB responsibilities, that I'd have no problem catching up on my backlog of unexplored technologies and unread books, even with the distraction of daytime TV shows like "MythBusters" and "Dirty Jobs", and regular trips to the beach, but even after dropping some of the more "speculative" technologies, like F#, Acropolis, and the ADO.Net Entity Framework, from my study list, I still feel like I'd need to take the rest of 2008 off, just to cover the essentials... and maybe 2009 too!

One of the first things I wanted to do this summer, was to set up a realistic simulation of an acceptable enterprise development environment, using virtual machines, in order to practice with, and test, Agile tools and processes.  I finally settled on using VirtualPC 2007 to create VS2005 and VS2008 Pro Beta2 workstations, with associated continuous integration VPC servers running a whole slew of open source analysis tools.  I can check for issues developing as a Least Priviledged User on my Vista laptop host system, compared with developing with full admin. priviledges on Windows XP running inside a virtual machine.  To create a fresh work or test environment, I can simply copy a "base" VPC image, without having to reinstall the OS.  It may even be possible for the server to automatically publish the compiled output to a fresh VPC image for testing. If you want to set up something similar, you'll save yourself A LOT of time and misery if you refer to the links below.  One hint: Use the NAT networking option to share the host system's internet access and virus scan software, for tasks like downloading OS updates, and switch to a Loopback Adapter when you want to communicate between VPC images, or between your host and a VPC image. You'll also want alot of disk space (each VPC dev. workstation ends up around 9GB.), and alot of RAM (My system has 2GB).

    VPC Development and Debugging White Paper
    Creating Smaller Virtual Machines

For more information and guidance on CI server-based development, I recommend the recently-published Continuous Integration", from the Martin Fowler series. Setting up your server to automatically regenerate the database, complete with a clean set of demo data, in order to validate schema changes, is just one example of possibilities I'd never considered.

I found a couple of new books, covering enterprise Agile and Agile project management, to be enjoyable, quick reads, suitable for managers who may just be starting to seriously consider Agile practices, but the books didn't contain anything that would be new to developers who have been keeping up with Agile community discussions over the years.

    "Scaling Software Agility", from the Agile Software Development series
    "Manage It!", from the Pragmatic Programmers series

I've been working through the LINQ examples in the recently published "Introducing Microsoft LINQ", and I definitely recommend that book as a quick way to get up to speed on the three flavors of LINQ (to Objects, to XML, and to Databases), and the new C# 3.0 language features LINQ is built on.

Its a strange kind of vacation...

Halfway through a four year stint in what was then a "peacetime" army (how quaint does THAT concept seem these days?), I was attached to a Special Forces team for the duration of a three month Arabic language course at the JFK Special Warfare Training Center, at Fort Bragg, North Carolina.  I noticed my new classmates didn't seem to share my unit's obsession with spit shined boots and starched uniforms and buzz cuts, and I was stunned on our first day, when a low-ranking team member cut off an attempt, by an overzelous regular army officer classmate, to assign us some unrelated busy work, saying "No, lieutenant. That's not why we're here" -- leaving the young officer red faced and stuttering.

Over the next few weeks, in answer to my many questions, members of the team described HALO freefalls, sniper, SERE (survival, escape, resistance, and evasion), and Ranger schools, the stress of trying to keep a goat alive through a series of burns and bullet wounds inflicted as a part of medic training, techniques for constructing demolitions using common household materials, the experience of being "drowned" in a swimming pool by military scuba instructors, and much more.  One afternoon, the team's medic suggested I join them at the local hospital after class, where they would practice giving each other injections.  I expected to start off with some carefully-supervised sessions using oranges or something, but they just handed out vials and syringes, and the team's weapons man guided me through the steps, on his own arm.  When I had trouble piercing the center of the vial cap because my hand was shaking, he just arched an eyebrow.  When I failed to jab the needle deeply enough into his shoulder, and had to gradually PUUUUUUSH it in the rest of the way before depressing the plunger, he simply drew a deep breath slowly in through his teeth.  Shrugging off my embarrassed apologies, he gave an evil grin and said, "My turn."  Of course, I didn't feel a thing, and my own attempt on a second victim went much more smoothly, but I still turned down an invitation to come back the next week when they were scheduled to practice inserting IVs.  The no-nonsense attitude and competence displayed by every member of the team was such a contrast to my experience in the regular army, that I spoke to a recruiter about transferring -- until I found out I'd have to extend my enlistment an additional three years.

The make up of a special forces team reflects many of the same values promoted by Agile methodologies, consisting of a small cross-functional team of expert individuals, who share a commitment to ruthless pragmatism.  An S.F. team consists of a team leader, with one expert and an assistant to cover each of four specialized areas: weapons, communications, demolitions, and battlefield medicine.  Team members continuously cross-train one another in their specialties, constantly attend courses to acquire new skills, and receive language, culture, and jungle, mountain, or dessert training appropriate to whatever region is their group's focus.  They can't afford to suffer fools, or foolish processes, lightly.

In contrast to the pragmatic "adapt whatever proves useful" philosophy embraced by most Agile advocates today, the first generation of Agile proponents tried to impose their own brand of process fundamentalism, and only gradually mellowed, partly through the influence of thoughtfull critiques, such as in the book, "Extreme Programming Refactored: The Case Against XP".  Most published Agile references now emphasize that Agile practices are not a "one size fits all" proposition, and it's the shared underlying Agile PRINCIPLES, rather than any specific methodology, which are important for organizations to cultivate.

"Critics of the first edition have complained that it tries to force them to program in a certain way... I'm embarrassed to say that was my intention... in this edition, I have tried to rephrase my message in a positive, inclusive way"  -- Kent Beck, in the preface to his second edition of "Extreme Programming Explained".

Unfortunately, some development groups remain unfamiliar with even the most the basic concepts of Agile development, with management still determined to try to nail down a "complete" design and "accurate" schedule up front, dividing responsibility for work into separate silos, preferring innocuous mission statements and shiny quality commitment plaques to the potential confrontation inherent in rigorous code reviews and post-mortems...  After ten years of intense discussion in the software developer community, with support for Agile practices increasingly baked directly into the developer tool set, that's hard to excuse.

As irrational as some management decisions affecting your development group may sometimes appear to be, at least they aren't likely to get you killed. When my Arabic language course ended, I returned to my own unit, just in time for our commander to order us to get our newly-redesigned camoflauge uniforms starched and pressed.  The uniforms came packaged with warnings from the manufacturer, clearly stating that starching and pressing would destroy infrared dispersing properties embedded into the fabric, making them a bright target for anyone with a night vision scope.  I suspect that more than a few individuals in a Special Forces unit would have stood up to remind the commander of why they were there...