The following five examples were originally written in response to questions that Don Willerton posed in SEM96 and SEM97 e-mail dialogues. His questions were
“Why do projects fail?”
“How do you train project managers?”
In thinking about this I add the implication, “so that project fail less often,” to the second question.
These five personal case experiences are presented to support my answer that the two most common causes for project failure are
1) ineffective user involvement, and
2) ineffective management understanding of the project subject matter domain.
I deliberately refer to the user not the customer. The customer is who pays for the project. The user is who uses or must live with what the project produces. They are sometimes the same.
By the word “management” in the above causes I mean project management and management above the project manager as a group. Thus, I maintain, as a project manager it is not enough if I understand the subject matter domain, my management must also understand. By understand the subject matter domain I mean a clear understanding of what the resulting product will do and how it will be used. I mean to imply that management must understand well enough to actually use the resulting system. They do not need to be skillful or efficient users, but they must be capable of actually using it. Management above the project management level does not necessarily have to understand technically how the system will be built. The project manager does need this understanding.
I state user involvement first because this can compensate for a lot of management lack of understanding. However, I have never meet a manager who had such understanding who did not also insist on the user involvement. So one might reasonably state them in reversed order.
I do not mean to imply that one must always ask users what they want. There are occasions when the user cannot be expected to know what they want until you show them what they can have.
I believe that the patterns of failure that Don identified in his e-mail are symptoms of these two causes.
I refer to these as personal because I experienced them first hand. I do not refer to these as case studies because I have not documented them in sufficient detailed to call them studies.
CASE EXPERIENCE ONE:
This was a project for the US Air Force (USAF) performed through a series of contracts totaling just over $15M. We were building a hardware and software system to test, diagnose, and support the repair of military jets without disassembly. The previous approach was for a technician to follow a diagnostic tree to identify a potential failed component, remove it, bench test it, until they found the failed component. It was not working well. Aircraft average availability time was dropping.
I had just been hired by the contractor for an entirely different job. The previous project manager had just accepted another position with the contractor. The first contract for a feasibility study was coming to a successful completion. I was asked if I would take over as project manager until a more suitable permanent individual could be found. I said, “yes.” I was asked what I knew about an A-10. I said, “What’s an A-10?” For those of you, who like me, didn’t know, it’s a jet fighter.
The team we assembled was technically competent but there were a few difficulties on the project. Two individuals who were key in the success of the first contract felt they should have my job. They had some justification for this. One of the managers I hired went through several weeks of severe depression as the first anniversary of his fiancee’s murder approached. Another manager was significantly distracted by the trial of two individuals who were eventually convicted of killing his son-in-law two years earlier. My administrative assistant’s 13-year-old mentally retarded daughter was raped. The wife of one of the key technical persons was a slow cycling manic-depressive with paranoid delusions. In addition to these individual problems we also had significant personnel conflicts for which, in one case, I eventually had to involve corporate HR and the company local security officer to help resolve. The Air Force project manager had a reputation of being a wild man. It was well deserved. He wore it as a badge of pride and honor. On the other hand he was no bureaucrat and he was technically very competent. He knew the problems and aircraft maintenance extremely well. He and I were both INTJs (personality types in the Myers-Briggs Personality Indicator). We understood each other very well.
I was also given the assignment to bring the software team to a Software Engineering Institute (SEI) level 2 and implement all project specific SEI level 3 practices. No one at the contractor’s site had previously attempted any SEI implementation.
The basis for the project sizing estimates was questionable. In fact, I later discovered the estimate had significant errors.
The project was completed on time and with in budget. All goals were met. Early field trail implementations showed an increase in aircraft availability and decreased maintenance cost. How did this happen in spite of all the problems?
The users, the mechanics on the line, who kept the jets flying, were involved. No one had asked them what they needed but nothing was produced without their involvement and feedback. Every person on the team was aware of their reaction to what we produced as we tested pieces and preliminary designs. Every team member was committed to help the mechanic do a better job. Yes, I was a fantastic project manager, a got a nice bonus, solved a lot problems, learned about A-10s, and as a reward I was given harder and more interesting projects. But, it was the effective involvement of the user that held the team together and kept us on course to produce the right product that worked. It was not I. The most credit I might give to the Air Force project manager and to myself, was that we promoted and insisted on that user involvement. Either of us could have destroyed that involvement.
I have an additional supporting argument for this thesis. One of my passions is the building of models of human system behavior. I convinced the Air Force project manager to fund building a computer based model of our project. When I started building the model I was convinced that the personnel problems we were experiencing were the greatest threat to the project and the need for additional very high quality technical personnel the second greatest problem. The model I built said I was wrong.
No matter what I did to either fix or aggravate the people problems the project succeeded. I could make it a bit late and a bit over budget, but not really fail. I could, however, make it fail (i.e. totally collapse) if the level of technical expertise dropped below a certain critical value, but that was rather obvious. It took me several days before I concluded that the bug I was looking for in the software or the algorithms was not present.
I eventually discovered that it was the user feedback loop that was making the project model stable. As long as this feedback loop was operating and the required minimal technical expertise was present the project was successful. When I removed this feedback loop the project failed. Even if I removed all personnel conflicts and maximized the technical expertise, the project still had a higher probability of failure than success.
If I had read Quality Software Management (QSM) prior to this, I might have anticipated this result. It is a very clear example of what, I believe, Jerry is referring to when he describes the feedback and control mechanism in a project. Many aspects of the project goal need the refined and detailed adjustments of the user. Without a quality definition of the target it is difficult to hit it. [I would have read QSM beforehand, but Jerry had not yet written it. Reading it after this experience gave that section a lot of meaning. It also has given me a better understanding of similar patterns and has been a useful tool in explaining the point to others.]
CASE EXPERIENCE TWO
The European operations of one of the world’s largest oil companies was looking for a contractor to address their Year 2000 problem. I was given the assignment to convince them to give us a pilot effort at their Germany location. What I did not know at the time was that previous meetings and proposals had placed the estimated cost at over $120M and that the prospect’s IS management was unconvinced they wanted us to do the job. I went to Germany blind to the history. When I got there I was asked, “How much would it cost?” Three things went through my mind;
1) Jerry’s Orange Juice Test, [in The Secrets of Consulting]
2) there is something about this “music” that I do not understand, and
3) I didn’t know how much it will cost.
So I said, “I don’t know how much it will cost and here is how I propose to find out.” The key issue that I was later told convinced them to let me do the preliminary work was the proposed involvement of the technical staff who knew the applications. In this case the people who would be the users (recipients) of the work we proposed to do. At the time I thought this was odd, because it was such an obvious thing to do. It’s real significance did not become apparent until much later.
The customer’s management asked if I could start immediately. I, of course, agreed. I did have some difficulty explaining to corporate management that I had started the work and that there was no written proposal and no contract. There was a P.O. that said do a Y2000 Pilot not to exceed $120K. However, the P.O. did not explain what was meant by a Y2000 pilot.
I left Germany one week later with source code for four applications. I returned to Germany eight weeks later with a detailed plan for the Year 2000 work in Germany and a tentative plan for the rest of Europe. It was a bit of a struggle getting the staff I needed for those eight weeks to do the analysis, but it was done. And with the help of the German technical staff we developed the plan with an estimate of $23M. The BP Europe IS management was a tough customer. They examined the plan carefully. They check the analysis and the estimating procedure. They found several mistakes and they made several suggestion for improvement. All were clearly valid. We got the contract.
All I did was take the time to understand the specific domain of their Y2000 problem (the customer’s IS management already understood), and I involved the user (the future recipients of what we were to produce). Incidentally, my management did not understand the problem domain. This caused some difficulties, but the customer management took a very strong position. They told my corporate management that if they wanted the contract it would be done the way we had worked it out and that I would be the project manager. Since they were a multi-$100M customer all internal issues with my management vanished.
I got a lot of accolades, but no bonus. My management did want the contract to be $120M, not $23M. Oh well. I did get my real reward. I was given some harder and more interesting projects.
Due to the burden of heavy travel to Europe I withdrew from the project but not before finding a replacement. The project was going well and the estimates were holding when I left. I do not know the current status.
CASE EXPERIENCE THREE
Because of the success with the prior Year 2000 situation, I was asked to work on a similar project at a major Health Maintenance Organization. It was thought to be about $150M. It was a failure, at least from my point of view. At first it appeared to follow the same pattern as above. But, it was soon apparent that their corporate IS management did not have the same level of understanding of either of the software they ran or of the Year 2000 problem.
Every attempt to involve the technical staff and technical management, i.e. the user of our proposed Year 2000 remediation was a failure with one exception. I was able to get a phone survey completed of a group of mid-level IS project managers who were responsible for the day to day operation of an estimated 5% of the applications’ source code. If one accepted what they said at face value and extrapolated, the estimate would come out at less than $10M.
My corporate management’s eyes were fixed on getting a $100M plus contract and they believed they could convince the customer management to buy it. We did get a pilot contract but I removed myself from the project since I could not support the approach. I left the contractor shortly thereafter, but the last time I checked the pilot project was failing and hopes for the big contract were low.
CASE EXPERIENCE FOUR
I was for a short time responsible for a project to help the IS department of a multibillion dollar State Government agency improve its software process capability. The department management was never really involved with nor took the time to understand the issue behind software process improvement and never accepted the type of effort it would take. None of the technical staff who would be the users of the resulting processes were seriously involved.
My attempts to change this situation were strongly resisted. The project was further compromised by people problems and organization power struggles both at the State and within management of the company for which I worked. The project went on for some time. We and the State Agency changed some staff. The project was eventually declared successfully completed, but that was a political statement not a technical one.
Many symptoms of project failure were prevalent, but, I believe they were symptoms of the underlying causes of lack of management understanding of the problem and ineffective user involvement.
CASE EXPERIENCE FIVE
A few years ago the many federal offices within the welfare system allocated over $1B dollars for the development of new automation systems within each state. A formal statement of requirements was written by the federal agency. One state eventually spent $250M trying to implement a system with a well-known and very large contractor. The project was canceled during implementation after 3 years of development.
During the first year several patterns of failure were apparent. They were also apparent in two other projects, both well in excess of $100M within the same state agency. Some state managers were fired and all three projects were transferred to a different state organization. At this point all of the failure patterns began disappearing.
I knew the application area, the state project management, the state technical staff, and the contractor project management. The people were competent and the design goals of the project were being meet and tasks completed on time. They were doing a professional job the way we are supposed to do it per project management classes. But during implementation it was discovered they were doing the wrong job.
That is not quite accurate. The federal agency, who was ultimately the customer, had a well thought-out set of requirements that meet their needs (wants). Unfortunately, the user, the administrative personnel, who were responsible for implementing the federal program in local communities did not have their needs or wants satisfied. The system significantly reduced their ability to perform their job.
I know something of the details only in this specific state, but not any other states. My friends in this state government told me that similar projects in the other 5 largest states were also well behind schedule and were believed to be failing. A total of over $1B was wasted because, in my opinion, no one in the management chain really understood what the staff in the field actually did to implement the federal rules and regulations. And no one asked them.
This has been rather lengthy. I could go on with several more examples. I shall not. My thesis is simple. For successful projects we need to have
1) A clear understanding of user wants
2) Management that understands the problem domain
3) A clear understanding of customer wants
4) A moderate level of project management practices that provide a measurement of goal achievement status
5) An honest commitment to achieve the goals
I have stated them from most likely absent to least likely absent (at least in my experience).
If these are present you will succeed. If scope changes, it will be because the goal is better understood and clarified. The appropriate adjustments will be made. If people fail to perform to the expected level of effort, it will be corrected. If staff and/or project management skills are not up to the job, corrections will be made. If sizing estimates prove to be ill founded, adjustments will be made, assuming continued adequate value for goal achievement.
I would add that there is one type of failure mode that I will not label a failure. To the extent that the goal is difficult or not fully known, the project is at risk of attempting to do more than may be possible or more than is value justified. O do not, I cannot, think of this as a failure. To NOT attempt to do more than we have ever done before is a greater failure and tragedy.
If you have read all the way down to this point, thank you for you patience. I would be most interested in your thoughts.