“When you can measure what you are speaking about and can express it in numbers, you know something about it. But when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind.” – Lord Kelvin
In the four decades I’ve spent in the software business, I’ve learned that there are three fundamental abilities you need to do a quality job of managing software engineering:
the ability to understand complex situations so you can plan a project and then observe and act so as to keep the project going according to plan, or modify the plan
the ability to observe what’s happening and to understand the significance of your observations in terms of effective adaptive actions
the ability to act appropriately in difficult interpersonal situations, even though you may be confused, or angry, or so afraid you want to run away and hide.
All three abilities are essential for quality software management, but I didn’t want to write a large, imposing book. Therefore, like any quality manager of software, I decomposed the project into three smaller projects, each one addressing one of these three fundamental abilities. Volume 1, Systems Thinking deals with the first ability-the ability to understand complex situations. Volume 3, Congruent Action deals with the third ability-the ability to act appropriately even in the presence of strong feelings. (Since this preface was written, I decided to add a fourth volume, on how to make the changes suggested by the previous three volumes-Volume 4: Anticipating Change.) In this second volume-First-Order Measurement-I invite your attention to the ability to observe what’s happening and to understand the significance of your observations.
I spent much time anguishing over the correct title for this volume before settling on First-Order Measurement. For one thing, when a book is published with the word “measurement” in the title, the author seems obliged to quote Lord Kelvin. I have kept to that tradition, but now feel compelled to analyze what Lord Kelvin meant, and did not mean, by his oft-quoted statement.
When you can measure what you are speaking about, Kelvin said, you know something. If you are measuring just for the fun of knowing things, then it doesn’t much matter what you measure. But if you are trying to accomplish something, then arbitrary measurements are perhaps not what you need to know. Engineering is about accomplishing things, so engineering measurements are not arbitrary.
Lord Kelvin was a physicist. Although I wanted to work with computers since I was a boy, I was trained as a physicist because there was no computer training in my school days. Today I consider myself a software engineer, and though I still love physics, I distinguish between knowing in physics and knowing in engineering:
Physics knowing is understanding the universal laws of nature.
Engineering knowing is knowing how to do something.
I use the term “third-order” measurement for the kind of measurement that supports the physicist’s search for universal laws. An example of third-order measurement would be an experiment that tested the First Law of Thermodynamics that says, in effect, that it’s impossible to build a perpetual motion machine.
Engineers are not trying to know universal laws, but special laws relating to building specific things. Therefore, they are interested in what I call “first-order” and “second-order” measurement. An example of such measurement would be measurements to build and tune a high-efficiency machine.
The term “first-order measurement” is used in engineering to describe a measurement just adequate to the task of getting things built. First-order measurements are used in “back of the envelope” (or, in England, “back of the cigarette box”) calculations. They support the “rule of thumb” and the rough sketch, as well as the “quick and dirty” or “seat of the pants” estimate. These are the measurements that allow us to build a machine in the first place.
The term “second-order measurement” refers to the kind of refined measurement used to optimize systems-to make them cheaper or faster. These are the measurements that are used to tune a machine to its most efficient performance. If you examine the other software engineering books with “measurement” or “metrics” in their title, you will find that they primarily deal with second-order measurement, and in some cases, even third-order measurement.
Although second- and third-order measurements are essential to the development of the software engineering profession, they do not really address the day-to-day problems of the typical software engineering manager. Perhaps the following parable will clarify why I make this claim:
Mr. and Mrs. Tweedle had 15-year-old twins, Dum and Dee, who were just learning to drive a car. Dee had driven the family car ten times, and she had a perfect safety record. Dum had also driven the car ten times, but he had been involved in three wrecks. Mrs. Tweedle told Mr. Tweedle that he had to have a talk with the boy, before someone was killed.
Mr. Tweedle opened the conversation by reviewing the three accidents, then asked Dum, “What do you have to say for yourself?”
“Well, three accidents out of ten trips isn’t such a bad percentage.”
“I might agree with you,” said Mr. Tweedle, “but your sister, Dee, has also made ten trips without so much as a stone chip on the windshield.”
“That’s true,” said Dum, “but I get much better gas mileage than she does. And I don’t get mud on the tires.”
“Oh,” said Mr. Tweedle. “I hadn’t thought of those things. Well, just try to drive more carefully from now on, and keep up the good work on mileage and cleanliness. I’m going to have to talk to your sister about her driving habits.”
Even to those of us who have been bamboozled by our own teenagers, Mr. Tweedle sounds like an idiotic parent. But before you judge him too harshly, remember that the story is only a parable. And what is it a parable for? You may recognize the three out of ten figure from software engineering. It’s the fraction of large software projects that, among many of my clients, don’t ever get finished. Capers Jones, Tom DeMarco, and Tom Gilb have all confirmed to me that this 30% figure is consistent with their experience.
Suppose you were the manager of software development in an organization that had done ten projects, three of which had failed. The software engineering managers from those failed projects can show you precise measurements proving that they produced more lines of code per dollar and fewer failures per line of code than the other seven projects. Would you go to the other managers and talk to them about their managing habits? Of course not.
As John von Neumann said, “There’s no sense being precise about something when you don’t even know what you’re talking about.” But that’s exactly what many managers are doing when they involve their organization in measurement programs based exclusively on second-order measurement. Bill Silver, one of the gurus of software measurement, said,
“It’s sad, but true. Most software measurement programs fail.” (1)
Silver’s observation is confirmed by Capers Jones, whose experience is that 80% of software measurement programs are out of existence within two years. It’s also confirmed by my own clients, who fit his description of the first and foremost reason for failure:
“Few companies have a quality culture that provides a positive environment for measurement.”
Let me put it more bluntly. A company that wrecks three out of ten major software projects is not ready for second-order measurement. Even worse, if such a company attempts to install a measurement program based primarily on second-order measurement, they will do more harm than good, and probably wind up with a million-dollar measurement mess. This is not “a positive environment for measurement.”
The data on software quality cultures indicates that at the present time, only a small percentage of organizations have organizations that would support a second-order measurement program. (2) Quality Software Management is designed to help the managers in those organizations improve the quality of their management, so that in turn they may improve their organizations.
The purpose of this volume-First-Order Measurement-is to help those organizations reach a point where they can meaningfully consider the use of second-order, or even third-order, measurement. If you work in an organization that churns out software products on time, within budget, that please your customers and continue to please them over their useful life-and you do this at least 99% of the time-then you won’t need to read First-Order Measurement. In fact, if your organization meets these criteria, I would be delighted to refund whatever you paid for this book in exchange for lessons in quality management.
But if your organization doesn’t meet these criteria, then I hope to show you how to create “a positive environment for measurement,” to avoid the costly mistakes that have led to failure of so many measurement programs, and how to measure things simply and efficiently-things that will help your organization consistently produce the quality software you want.
1. Silver, Bill, “The RPM3 Software Measurement Paradigm,” Software Quality World 1991:3,2, 11-15.
2. Humphrey, Watts, D. H. Kitson, and T. C. Kasse, “The State of Software Engineering Practice: A Preliminary Report,” Tech. Report CMU/SEI-89-TR-1, Software Engineering Institute: Carnegie Mellon Univ., 1989.