Diaries of a Project Manager – Defects, Defects, Defects
Defects, defects, defects. The bane of every project managers existence.
Now, if you are an Agile aficionado, please stop reading now. You don’t have defects, you have learning potential! It is really important to note this fundamental difference between the two major methodologies. Agile works on the principle that every error is an opportunity to learn. Stakeholders, clients, fellow team members welcome and embrace learning and there is no shame in it. Trial and error is encouraged. For this reason, an Agile method is best applied when you are endeavoring to deliver innovation or something you have little experience with. But beware, this is a culture that needs to be harbored within the organisation and your company must be ripe for this culture to take root.
But our waterfall friends, they don’t fare so well. I have experienced a project that went live only to discover nearly 1000 defects. By month three, this number rose 3 fold. A year later funding ran out and the project was closed… And still today, teams members working with the delivered product will refer back to that project, making reference to the various defects that remain open today. Technical debt that may never be addressed, leaving the product limited in it’s use.
Some of you may be wondering if there is any truth in my story and how in hell’s name that happened! But to be fair, this is not the only time I’ve seen this happen. Many years later, with another organisation, an old, weathered system was re-structured. This fundamental change to the database and application was promised to breathe new life into an aged process and improve the overall stability of the system. Sadly, those hyped benefits were over-shadowed by 100 defects in SIT and close to 300 in UAT (which equated to over $300k in additional cost).
My feedback, in hindsight (which is always a wonderful thing).
- In both cases, the organisation delivering the product used a waterfall methodology to deliver a product to a user group they had no experience with.
- In both cases, the majority of the defects were categorized as ‘lack of clear requirements’ or ‘change of requirements’. Which in most cases really means non-documented requirements or processes.
- In both cases ‘requirements gathering’ was a very high-level management document, which had been produced to acquire funding. There was no detailed objectives, scope, business process or testing regiment articulated. In both cases, the customer had provided little to no process understanding.
- And in both cases the delivery organisation put in a very competitive bid to win the work and then ran-ran-ran at the target with little planning. Hence, no one stopped to say, hey, we really should include a sum for robust planning, maybe even a small POC to test whether we know enough. Have I ever heard a leadership team stopped to ask why the BRD was a little light on and if it really holds a good level of information on the objectives, scope, business processes, requirements and testing…without which, we cant really know what needs to be delivered.
And all those people that were required to ‘sign-off’ on requirements? Where are they when it all goes pear shaped? Did they actually read the documentation and provide appropriate critique before they signed it? Sadly, not always.
I like to call it corporate burn. Quick, quick, quick, I have a great idea. Quick, quick, quick, give me a price. Quick, quick, quick, give me the money. Quick, quick, quick, lets get started.
The moral of the story for me and a note to the project sponsors of the world: spend more time being specific about what the product needs to do and what the testing outcomes need to be. Exactly what it needs to do and exactly what you will test, which should align 100% to your revised business process. If you can’t map out every process point for your major deliverables, you’ve not done enough. And then, only then, should you start considering products, delivery partners, method…and cost!
Over many years, I have developed a rule of thumb: the larger the cost, the more robust the initiation and planning stages need to be. Either do your homework upfront or seek agreement to learn as you go. But doing neither will come back to haunt you, no doubt.