« Bimbo Blowout| Main | NSI Caught Front-Running Domains »

It's not the Code, Stupid

| | Comments (2)

Programming is not about programming.

I came to this realization after reviewing my XP and Agile books as part of an engagement for a large client. I must confess that each time I start on my XP books I have a hard time because I start laughing when I compare the books to reality. I think I finally can explain why.

It's not that I don't buy into Agile and XP -- heck, I was doing a lot of that stuff long before those books were ever published. It's just basic performance-based project management. I guess I always felt that in the theory and applied theory department, some of the XP and agile software authors either didn't know what they were doing or just making it up as they went along. I especially found funny their skewed view of "heavyweight" systems like RUP, CMM, CMMI, etc. The authors told horror stories about these other systems and then pointed out how much better their tactics were. After working with people who helped write the CMM and RUP, I found most of these guys don't want onerous paperwork systems -- these paperwork monstrosities are created by the practitioners, not the inventors. Then there was the whole cult of personality in the Agile/XP space that I found non-productive. To top it off, the XP books in particular felt more like political statements than serious tools for figuring out what software construction was all about. They were the kind of book to sell to frustrated programmers laboring under a huge paperwork beast and saying "We've got it all wrong!" Power to the people, man!

Understanding how Agile/XP ran off the rails will help you to understand why software development is not about developing software.

There are two mistakes that people make when learning how to manage software projects: 1) Mistaking the deliverable for the conversation, and 2) Mistaking a list of suggested instructions for how to deal with risk (tricks) for the real work of identifying process risk and creating responses.

We see the first one in requirements a lot. "When are the Use Cases done?", people ask. But we also get it in other places. "Are you done with design?" is a good one. Another one is, "Has testing been completed?"

People confuse artifacts with the really important stuff. It's important to know that people have worked at testing the program. It's important to know something has been done with design. It's important to know what the program is supposed to do: that's why we have conversations about requirements, design, or testing.

But these conversations are never completed. You are never through with requirements, for instance. Even when the program is delivered and you decide you're not doing any more, people will still add things to a wish list somewhere. What's important is the ongoing conversation, not the printed result of that conversation. This conversation can be split into different topic areas, like "requirements", "test", or "design" If there are more than a few of us, somebody might want to write something down to keep track of the conversation -- you don't want to revisit the same ground twice and in anything bigger than a few hours, you need to keep track somehow, By the same token, we can use all kinds of "tricks" to have the conversation -- story cards, activity diagrams, role playing, negotiation, interviewing, etc.

So if you're using use cases, the printed stuff you have for your use cases is just an ongoing written report of where the requirements conversation stands. Likewise, your design document is a written record of what you thought you would do for design. Maybe you've stopped having high-level design discussions for now. In that case, the design document will probably not change a lot.

The second thing that people do is mistake a pre-existing list of tricks to deal with risks with a recipe for running a project. Why are the "heavyweight" systems full of so much BS? Because the people implementing them don't want to tolerate any risk at all. What's the premise of all those agile process books? To talk about ways to address the minimum amount of risk necessary to make software happen for a customer in a small setting. It's the reverse of "How hard can it be?" It's more like "How easy can it be?" Sadly for us, we teach agile without really telling folks the reason. What we're really saying is "there are only a few major risks that you need to be aware of no matter what in software development. Here are some tricks for dealing with those risks." We leave the risk part out, and the reader is left only with the tricks.

So of course the reader confuses the tricks and the paperwork with the real work. It's easy for us to confuse the written record of a discussion with the results of the discussion itself. It's only natural -- when you finish some kind of document, you get the feeling that you've created something that didn't exist before. When you do story points or CRC cards, you get the feeling that they are not just a trick to enhance a conversation -- you're really doing something important. The the document or the trick becomes the deliverable, instead of the advancement of the conversation, which is really where the value lies. We take a list of implied risks and responses -- which are usually conversation tricks needed to address those risks -- and the we forget about the original risks. We're left with just the form of the response without any of the substance of the thinking that goes into making the response. Everybody wants the recipe and nobody wants the responsibility.

Here's the mind-bending part: the statement about the deliverable getting confused with the conversation is true about the completed code as well. The completed code is just the formal result of the conversation the players had about the problem domain. Nothing more, nothing less. The format, because it is executable by a machine, has value to the people who are paying you money (hopefully.) But that doesn't mean that the code is the only thing of value, or even that it has the most value.

Have I forgotten to take the little blue pill again? Am I off my meds? Am I going to break into a song-and-dance version of "I've gotta be me"? Am I seriously suggesting that the main purpose of software development might not to be to develop software?

First, let's level-set. I'm not talking about you and 4 of your buds writing the next Doom in your basement. It's hard to see it at that level. I'm talking about situations that involve more than a few teams.

Let me give you several examples of how the delivery of executable code is not the primary business value to the organization and let you decide for yourself.

  • Trying the Technology - Ever work on a project where a large organization was implementing technology X and you were the first team? You really think that your software was as important as the knowledge of how that technology was going to work? I mean, even if they put you on the flagship product of the organization (which happens all the time with new technology, unfortunately) your value to the organization is primarily trying the new technology. Somebody made that choice when they gave you the flag. Let's put it this way: if you fail, people will probably say, "they were a great team, but technology X just couldn't do it for them" That should tell you where the real value lies.
  • Aligning the Organization - I really wish we could live in a situation where we have one product owner, some developers who are all generalists, and a big room to play in. A foosball game would be cool too. But once you get so big, you start having people who are responsible for various slices of an organization. You've got the "huge database" guys, who are too valuable to dedicate to any one project and who keep the keys of the kingdom close by. You've got the Enterprise Architecture guys, the production deployment guys, the customer-facing bug tracking guys, etc. Most of these groups can and should exist because any one project has the ability to crash all of the others. You botch a deployment and the damage you cause is much greater than the value you are supposed to provide. In practical terms, that means you don't have your nice co-located generalist team anymore.

    Sometimes these other groups don't play nicely with each other. In fact, most of the time there is a difference in how each group sees the organization. An old friend of mine once sold machinery into factories. He found that each department in a factory usually had an organization chart on the wall -- and on each chart, the department he was talking to was presented as being the center of the organization! So you're going to need to help people work productively with each other. That may not be listed in your team charter, but you can bet it's a valuable service. Many times the real reasons for projects in big organizations boils down to trying to get the groups all heading the same way. It's usually not presented that way -- mostly because people who present us with problems rarely know what the heck they are talking about. As proof, how many "enterprise" products does an average organization have that each could perform the roles of most of the others? So it's not the technology -- organizations buy solutions to their problems over and over again. It's the alignment that is missing.

  • Failure is an Option - 80% of software projects fail or are judged poorly by their sponsors. So what's wrong with that? As long as we realize that in most all cases it is not the technology but the people who cannot get along, doesn't failure show us that we are not addressing our people problems? If those projects failed because of poor software teams that's one thing, but do we really think that a good software team without a heavy-hitting sponsor can make 17 departments that don't get along want to play nicely? Not likely. Once team ability and work structure is taken care of, failure is what software teams should be doing, and as quickly as possible.
  • Understanding the Business Problem - Let's be honest with ourselves: nobody thinks at a small level of detail. Everybody naturally thinks in huge concepts and generalizations: that's the only way we can deal with the world. So when your boss says something like "I'd like the workflow system to be hooked into marketing so that qualified contacts can enter the web-sales pipeline" does he know jack squat about any of those systems? Probably at a high level, but nowhere near the level you're going to need to finish the job. And I'm not talking technology, either. I'm talking the definitions, processes, and world-views that each of those departments bring to their work. Marketing doesn't want to share leads -- research shows that one-on-one sales is only effective after a definitive interest is shown. Web sales is not interested in doing cold-calling. It's been decided by the lawyers that workflow systems can only handle work-related product and not client data. The list goes on and on. No matter what your software project, you're going to be learning the business at a lower level than the guy writing the check, you can count on it. Many times gathering and distributing that knowledge has more value to the organization than the actual project itself. Sounds crazy, but it's true.
  • Understanding the Customer - Guess what? You can pretend you have a single product owner, but that's mostly self-deceptive BS. Take a startup. You have 4 guys and a bunch of machines. So where's the customer? Perhaps you have a stand-in, or perhaps you have an initial client. But is any of that the real customer? The hundreds or thousands of folks you are going to provide value for? Heck no. It's just that you've decided to cheat and let somebody stand in for those folks. Most times there is no single customer and there is no way to find out what the customers as an aggregate want -- somebody has to guess. Maybe it's an educated guess, maybe not.

    Sure -- assigning a single product owner or stakeholder is a great way to get paid. But it's not locating and providing value to the customer. And that greater mission is what any organization is about. That means that's what you're about, too. So you're always going to be writing code based on a bunch of assumptions, studies, guesses, overheard conversations, whatever. Once the code is out there, then hopefully you start getting feedback and the development conversation continues.

And that's where we get off the bus. Programs are just executable versions of the results of conversations we've had with everybody that's important. It's the conversations and the results of those conversations that are important, not the code. And it's not just one customer or product owner -- we have to have conversations with a whole slew of people. Because the computer can execute code much faster than humans can, there is cost-savings associated with making an artifact that is a program. But we could do the same process without computers and still provide worthwhile value to the customer. It's always the conversations that are important, not the artifacts. If you don't understand why, I've failed in this article.

Software doesn't exist in some kind of vacuum apart from the rest of the organization. There really isn't any logically separate place called "I.T." Information Technology is the brains for the entire organization, We're the guys who create and augment the organizational brain and software construction is the process of the brain re-organizing itself. Every piece of attempted software represents a tenative philosophy, effort at compromise, or quest for knowledge. You've got a brain too. It might be nice if everything your brain did was directly related to making money or helping another person, but that's not the way your brain works either. Why do we expect IT -- a brain representing dozens if not thousands of humans -- to work any differently?

Writing software just about making computer programs? Not a chance.


Nice article.

I like to think it's the human dimension of developing software that makes it worth doing. It can make even the most 'boring' sounding project worth doing.

If all we really did was follow a process and translate results into code, it wouldn't be much of a life.

Great comment, Robin.

You are right that it is the human element that makes it worthwhile. I think too many times when we move from programmer to lead to PM or architect, we start thinking of other programmers as machines, and try to create a "program" for these little machines to follow.

That wouldn't be much of a life, even if it did work. And it doesn't.

Leave a comment

About this Entry

This page contains a single entry by Daniel published on December 25, 2007 2:45 PM.

Bimbo Blowout was the previous entry in this blog.

NSI Caught Front-Running Domains is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Powered by Movable Type 4.23-en
Daniel Markham