Taylor Vs Deming and Software Development - Part 2

Posted by Kevin Brady on Thu 28th September 2006 at 04:13 AM, Filed in Software Dev Methodologies

It is clear that over the past 50 years Deming’s ideas have become the dominant management doctrine among the western industrialised nations and have often been lauded for the transformation of Japanese car manufacturing in the 1970’s. Adoption of these ideas by Toyota and other Japanese car manufacturers generated such stunning productivity that it firmly entrenched these companies as the dominant players in the modern, global car manufacturing market.

image

Heart land of Deming & Taylor – The production line.

Despite this transformation in manufacturing, Deming’s doctrines have not had the same impact on the civil engineering, construction and to some extent the IT industries. The reason for this can be traced back to Deming & Taylor’s focus on improving production line efficiency. In contrast civil engineering and construction derive their revenue almost exclusively from the sale of unique project ventures which are rarely repeated. A key difference.

Similarly, IT Project Managers are also largely responsible for the set-up and management of one off enterprises, to deliver a single custom-made product which satisfies the requirements of a specific business case. To achieve this result, a Project Manager is responsible for the design of a “Bottom-up” project plan & the assembly of the necessary resource to deliver such a project in accordance with the designated time, quality, cost and scope coordinates. Furthermore, the one-off nature of IT projects means that a strong element of upfront product design is required.

In my view, one of the biggest reasons for large IT Project Failures is the lack of collective responsibility by line managers (who are largely the sponsors behind most IT projects) for a project or programme delivering the software equivalent of a Skyscraper rather than a Dog Kennel.

This lack of recognition can generate clashes between project and line management philosophies severely impacting the likelihood of project success.

All too often line manager sponsors see standard project management practices as totally alien and culturally unacceptable at the point where the project interfaces with the line function. These thoughts frequently cause the project to be run along modern line management principles. Sponsors often believe that values, team work, positive thinking, and motivational activities, can, if applied correctly, overcome all issues and problems and make rapid high quality delivery a reality. The engineering detail in the form of product designs /plans are left in the hands of the newly empowered and motivated project teams often working towards designated delivery dates. Vital processes such as risk /issue management become one off shopping lists, and things like change control, detailed project plans, quality control and targeted delivery based on bottom-up planning become sidelined activities.

To get a more detailed idea of this important and often overlooked reason for frequent IT project /programme failure I suggest that you take a look at one of my favourite blogs Carpafactum and pick up an inexpensive copy of Race through the Forest – a Project Management Fable by Timothy Johnson.

The Dot.com era spurned numerous companies unable to deliver technically feasible, cost effective ecommerce solutions because their projects were run in accordance with line management principles i.e Boo.com. The merchant bankers and former line managers from retail backgrounds believed that motivation and the employment of clever young people was all that was needed to make the magic work.

How wrong they were and how little things have changed today!

One memorable example of this kind of project /programme failure occurred in the London Insurance Market a few years ago when a huge programme setup to deliver an electronic insurance trading system failed spectacularly. The sponsor spent most of his time taking staff on days out to discuss and agree organisational values and sort staff into “pods” (small teams) without project plans, resource plans or any kind of change control or quality management. The sponsor naïvely believed that teamwork, positive waves (no talk of reality), trust, values, long hours focused on fuzzy task breakdowns and ever changing requirements would overcome everything.

It was no surprise to find that after two years these youthful and motivated project teams failed to overcome the towering and ever growing levels of complexity, technical infeasibility and wasted effort i.e “Thrashing” (tasks and activities completed only after discovering that they were no longer necessary due to unclear and ever changing requirements). In the end, and only after running millions over budget, the backers cut the money and shut the programme down.

However, a strange and surprising aspect of this story was that the programme had been attempted twice before, failing each time for similar reasons !

I don’t want all of this to sound like I am anti Deming where IT projects are concerned, because it is clear that happy and motivated staff who feel empowered, are very important factors in making IT projects a success. However, as IT projects start to increase in size and scope, their complexity tends to increase exponentially along with the level of potential software bugs. Without taking a more engineering and scientific based approach to the running of such projects, one usually finds that exceptional productivity gains will not alone guarantee success – Please read the book CRASH for many case studies covering this and other reasons for IT Project Failures.

If this realisation was not enough, we now know that Deming inspired thinking is being injected back into the way we run IT projects from a different direction. Yes, I am referring to Agile development methods and associated mind set /culture. Setting to one side the Agile approved list of iterative development methods for discussion in a separate post, it is clear that the Agile mind set is based around the Agile manifesto (please see my post Agile Fees Feeding Frenzy). My views on this manifesto are like my views on socialism. “Nice idea but doomed to failure” because it makes incorrect judgments about human nature (please read my post Agile fails to get to grips with human nature) and the narrow self interested behaviours of corporations, consultants and individuals alike.

Agile /Scrum and associated development methods win where motivation /energy empowerment are more important determinates of success, and when IT project complexity and size are low (new media projects). In larger scale projects where the complexity dramatically escalates, Agile approaches offer no alternative to solid proven project management and software engineering principles.

The moral of all this is to never get lured into running large IT projects in the first place. They are investments strictly not for “wives or widows”.

This message is corroborated by Bill Joy one of the biggest names in the US IT industry. In 1990 at the Churchill Club in Silicon Valley Bill Joy a tousle-haired programming genius and Silicon Valley legend known sometimes as “the other Bill”, predicted that Microsoft would ride high for several years and then everything would change. Joy was disgusted by computer code complexity and was convinced that something small and simple would generate the next great change in the computer industry. He stated that the next great breakthrough would not come from smart people working at Microsoft or Sun but a few unknowns working in a small company. These amazing predictions came true in the form of Google and Netscape.

Therefore, it has always been clear to me that the key to successful Project Management is not so much Taylor vs Deming, but having the talent to inject enough of Deming’s ideas into your project /programme to make sure you have a motivated /dynamic project team, whilst at the same time (relative to size), containing these freedoms within boundaries dictated by solid /proven engineering and project management principles.

Self assembling project teams (Scrum) and redirecting valuable Technical Architects into team management (Scrum Masters), is not going to be the recipe which brings the world consistent and repeatable IT project success.

Click here to see Part 1

This entry has been viewed 5604 times.

READER COMMENTS:

Kevin - thanks for the reference to Race - you are very right in your observations between quality management and project management principles.

Posted by Timothy Johnson  on Fri 29th September 2006 at 12:26 AM | #

Thanks Kevin. Enlightening summary.

Posted by Chris Tremlett  on Fri 29th September 2006 at 07:20 AM | #

Hi Kevin,

I’ve read your Deming versus Taylor posting, and to be honest I was disapointed. Apart from reproducing what is publically available material (wikipedia), your only orginal statement,namely that Taylor and Deming ideas aplly only to manufacturing is factually wrong.

The Toyota Production System has applied Deming ideas to New Product Development for decades. New Product Development is the Design and Development of new Products (Cars in the case of Toyota) and is a one off exercise just like Software Development.

I have some suggested reading for you: Lean Software Development, Mary Poppendiek. I also suggest that you take a look at the work of Jim Highsmith. In particular: Agile Project Management.

Your main criticism of Agile as I understand it is that it lacks Management discipline, responsibility and control. I belief this to be a mis-understanding on your part. The grand farther of “Agile” processes is Scrum and it’s central theme is Management process control. The difference in the approach suggested by Scrum is the use of Emperical feedback, rather then relying soley on an open loop deterministic approache (other wise known as a big upfront guess/estimate that then becomes a prediction that the Management hold the team to for the duration of the project, leading to hours of overtime, stress, corner cutting, poor quality etc).

Emperical Process Control is a scientific and tested aproach to complex system control well known to anyone with an Electronics or thermodynamics background. I suggest that you read up on the subject (Finding out how your central heating system regulates the tempersture of your home would be a good start :^).

I like your insurance example, but this has nothing to do with Deming or Scrum, (or Taylor actually), it is merely another example of poor management. An illustration of the effects of a lack of management control and oversight as you point out.

A discussion over emperical versus deterministic system control (Closed Loop versus Open loop )is an interesting one in itself, and IMO it is scientific fact, that closed loop control is more effective when controlling complex non-deterministic processes (in contrast to a big upfront project plan that after the first week everyone knows is a ‘big lie’, but carries on following anyway). Software Development IMO is a complex, non-deterministic process which cannot be accurately predicted open-loop.

Kevin, I really do believe that you are mis-guided. If you are interested in knowledge and understanding then I suggest that you take a good look at this video to gain a real insight into what lays behind Scrum:


The Roots of Scrum

On the other hand, if your sole interest is to promote your Consultancy, then your mis-information is no better or worst then the “Agile” consultanties that you have chosen to rebuke.

The problem with our industry is that anyone can set themselves up as an expert and in the end it is up to the Buyer to Beware!

Paul.

Posted by Paul Beckford  on Mon 2nd October 2006 at 12:47 PM | #

Hi Kevin,

I’ve just read some of your earlier posts on Agile, and boy you really have got an AXE to grind!!!

Is this the same guy that took offence when I suggested that his postion could be slightly bigoted? :^)

I thought lets find out some more about this guy, so I looked at your biography link. Pretty impressive, but an Insurer with a Degree in Economics - what is he going to know about New Product/Software Development?

BTW, nice photo - I love the soft focus and the “all knowing” pose.

So you acknowlege that historically we have suffered from 70% Software Project Failure. So what has the great Kevin got to offer the world as pearls o wisdom?

Let’s see…, do more of the same.

Let me give you a definition of insanity: If solution X fails you over 70% of the time then the remedy is more Solution X.

Kevin, what are you doing differently from what has failed the Software Industry miserably over the last 30 years?

Paul.

Posted by Paul Beckford  on Mon 2nd October 2006 at 01:41 PM | #

Paul,

Mr Brady can argue his own corner but I do wonder whether you actually bother to read his posts. You come up with some frankly bizarre misconceptions.

As someone who enjoys seeing reasoned debate, I also find it very disappointing that agile evangelists time and time again resort to petty criticism and attempted character assasinations (a classic sign of a weak counter argument).

Even as someone who likes aspects of Agile, it does make me wonder why you become so hot under the collar. If Agile is that good, you really wouldn’t have to defend it at all. The actions at the business end would do the talking for you!

Posted by Tom Hayden  on Mon 2nd October 2006 at 03:09 PM | #

Tom,

Character asasination are you kidding!!

Have you read the stuff this guy has wrote? He has assasinated a whole gendere of itellectual work on the subject of Software Development, most of which he clearly hasn’t read.

It wouldn’t be so bad if he didn’t have an interest and was an impartial observer, but he is not. As far as I can tell Mr Brady has a clear agenda and a clear AXE to grind.

He seems come from the “PRINCE 2” school of software project management (although I’m not sure what he actually is trained in because he spends more time slagging off the approach of others without specifically saying what he would do himself) and he clearly sees Agile as a threat to his business.

Unlike Mr Brady I have provided clear references to the material and ideas I raise son anyone (like yourself) can go a read the books or view the video presentations I have provide links to.

Mr Brady is yet to provide any supporting information (published research/ respected intellectual books/articles etc) other than just his own personal opinion.

If there is s specific point that I make that concerns you then I am happy to explain further. Otherwise check my sources for yourself and view the video link I supplied and make up your own mind.

Paul.

Posted by Paul Beckford  on Mon 2nd October 2006 at 03:54 PM | #

Umm.. grin

You guys really worry me. The tone of your language is aggressive, childish and militant in nature. It’s doing Agile a disservice quite frankly.

Don’t you get it? The fact that you resort to this sort of head banging makes people wonder why? If it’s that good, you really wouldn’t be wasting your time here defending it. All good philosophies, theories, products, services etc are adopted through word of mouth and osmosis. They accept and embrace regular criticism in all it’s forms, growing stronger for it.

This preaching from the ivory tower is quite frankly bizarre and disturbing.


Mr Brady as far as I can tell is mostly careful to qualify his observations, barring doing this in every other sentence, I don’t see really what else he can do to appear objective. So that tells me something else is going on here.

Vested interests Paul? Buyer beware isn’t one way traffic buddy!

I suggest you take a look at yourself and ask why you randomly throw in ridiculous words like “dangerous”. That’s laughable dude! Once again it only serves to lower the debate to the lowest common denominator i.e purile mud slinging.

I have seen that video many times before. Interesting and useful in parts, flaky and woolly in others would be my objective observation.

Posted by Tom  on Mon 2nd October 2006 at 04:42 PM | #

Hi Tom,

“AGILE, as mentioned in previous articles “AGILE Enough is Enough”, “AGILE Will Burn Your Arse”, is pure “Farty Floops” (a phrase coined by a good friend of mine ) as a fit for purpose approach to delivering large scale IT projects /programmes of work.

In the beginning, the scalability of AGILE methods was even questioned by some of the AGILE founding fathers and leading authors in this field.

Then the fees gravy train came and changed everything!!!!!!!!!!!!!!

The stratospheric popularity of the AGILE has kicked these concerns     potential weaknesses quietly under the carpet, where I fear they will stay until more money is lost on failed large scale IT projects   programmes of work.”

And you believe these are the words of a man who you describe as:

“Mr Brady as far as I can tell is mostly careful to qualify his observations, barring doing this in every other sentence, I don’t see really what else he can do to appear objective. So that tells me something else is going on here.”

Are you for real?

“Vested interests Paul? Buyer beware isn’t one way traffic buddy!”

True. the difference is that I am not selling. If I was I would stick to what I had to offer. Negative campiagning seldom works - it just turns the punters off.

“I have seen that video many times before. Interesting and useful in parts, flaky and woolly in others would be my objective observation.”

This is a blatant lie. It was taped very recently. So that leads me to my next question. What is your interest in this all Mr Hayden?

Paul.

Posted by Paul Beckford  on Mon 2nd October 2006 at 05:07 PM | #

Hi Paul,

Selective editing of articles can enable you to draw any conclusion you want. Here’s a challenge which will test your objectivity - go pull out some text from Mr Brady’s comments and posts that qualifies and balances his observations. Do you have the balls, or are you just a bigot. We shall see ehh! grin 

You’re really on a roll with the character assassinations their buddy grin et al “blatant lie”. Seems consistent with the rhetoric of a fanatic if you ask me. If you don’t believe that I have seen the video many times that is up to you - it only goes to prove that your ability to debate is pretty weak.   

My interest here is whatever you perceive it to be Paul. If you read my comments carefully you’ll appreciate that I favour Agile in some respects. What I dislike intensely is the ivory tower mentality / aggressive nature of it’s most vehement protagonists, of which you are surely one. And one with with a vested interest at that. Think about it.

I look forward to seeing you continue the debate on your own. Some of us have lives dude grin

Posted by Tom Hayden  on Mon 2nd October 2006 at 06:02 PM | #

CONSULTANCY FUD TURNS ON AGILE

hi All,

I am no lover of Software Consultancies, Agile or ortherwise.

All Large Software Consultancies are in it for the money and exploit their clients and IT professionals mercilessly

As an indepent consultant, I know this from experience. Much of my work is gained cleaning up the mess left by big consultancies after their clients finally gain the courage to kick them out.

This is why it is common knowledge that software projects fail over 70% of the time.

PRINCE as a methodology has been around since 1989 so has had plenty of opportunity to prove itself, yet here we are 30 years later with 70% project failure.

So the rememdy for failure is… more of the same!!

Interestingly only one of five articles on this site is on PRINCE, the rest is negative campaigning. so my question is:

If PRINCE is the remedy, why hasn’t it worked over the last 30 years

The simple truth is that no methodology is a silver bullet, it really comes down to the skills of the practioners. having said this however heavy weight mamangement process like PRINCE are not highly respected by Software Developers who are in a good position to know what works.

Infact the main contribution I can think of for this type of project management approach to the world of Software Development is Dilbert :^)

Joking aside. There is nothing going on here then good old fashioned FUD. Which makes me think two things:

1. Why did it take me so long to work it out? I guess it was just my honest desire to want to explain Agile to those who may genuinely mis-understand it.

2. Agile must be really taking off, if it is attracting mainstream FUD like this.

Paul.

Posted by Paul Beckford  on Mon 2nd October 2006 at 06:03 PM | #

Hi Tom,

You’re really on a roll with the character assassinations their buddy grin et al “blatant lie”. Seems consistent with the rhetoric of a fanatic if you ask me. If you don’t believe that I have seen the video many times that is up to you - it only goes to prove that your ability to debate is pretty weak.

On the top ine of the video it shows the date it was taken, 20th September 2006.

So to watch it many times you must be:

a) REALLY interested in Scrum and wanted to be sure you got everything.

b) You have got a very short memory and needed to watch it several times so as not to forget.

c) You are lying and haven’t watched it at all.

This isn’t a debate it’s a joke.

Paul.

Posted by Paul Beckford  on Mon 2nd October 2006 at 06:21 PM | #

grin

Just as I thought - no balls!

Posted by Tom Hayden  on Mon 2nd October 2006 at 06:25 PM | #

Hi Tom,

grin

Just as I thought - no balls!

A cheap shot. I am meant to find reason where there is only FUD - sure.

OK - Here is a challenge for you. Find intellectual works (publications, articles, papers etc)  that coroborate Mr Bradys’ point of view and I’ll eat my hat.

Paul

Posted by Paul Beckford  on Mon 2nd October 2006 at 07:37 PM | #

Hi Paul,

Thanks for comments.

I have had time to fully study you, your comments and other bits and pieces. I have made some humble observations as follows–

(1) What’s the motivation for your behaviour, which is almost graffiti of a kind when so little of it is focused on the post your supposed to be commenting on.  There’s only three things which would provoke me into getting involved in such a verbal torrent of abuse.

- Someone steeling my property or my money
- Someone steeling my woman
- Someone threatening myself or my family with intimidation or violence.

Taking this into account your behaviour is indeed perplexing and very strange and because you don’t know me your behaviour can’t be (2) or (3) smile

It’s transparent!!!!!!!!!

I must be a threat to your revenue stream. It’s the only thing that fits. 

(2) You’re worried about revenue because you’re an Agile consultant –

I decided to do a Google search on your name Paul Beckford. Guess what I found:-

Pages and Pages and Pages of Agile promotion, defending and mentoring posts and comments all over the net. If your working for someone your obviously self assembling your employers project into your huge internet presence. One quick look at this huge volume of STUFF and you start to ask yourself a couple of questions:-

- My consulting work is a 65 hour week and most of my articles /posts are written as this one is at night or early morning. You can’t be earning fees filling the net with all this Agile evangelism unless you don’t need sleep?

- Are you in fact a paid commenter and defender of the faith from one of the big Agile consultancies?

- Perhaps Agile is not delivering the fees you hoped and this is a way of letting off steam with the intellectual high ground and riches in sight but just out of reach. I can hear you say damn those critics!

If I wrote a post saying the Pope is evil I would get one comment, but attempt to write a critique of Agile and my blog starts to pile up with laughably vitriolic comments such as yours. All very very strange / weird and very tiresome on my part.

Who frankly would want to employ a consultant that is so aggressive about anything other than his own point of view.

I am just interested in constructive criticism of development and project management methods so they can be seen warts and all and therefore applied where they can do the most good. I have made many points on my blog about the new media industry and egg.com and wonderful companies like these that keep their projects small and where, Agile can work wonders. These are points obviously lost on you because small project work is not where consultancy revenue is earned.

I am only critical and a sceptic as are many on the net these days concerning Agile’s scalability and the limitations of many of the Agile approved iterative development methods. I have views and I am entitled to post my observations and I don’t care if they are fee earning or not “let right be done” for the good of the industry and all those poor developers who email me frightened to speak out about Agile.

Posted by Kevin  on Mon 2nd October 2006 at 11:46 PM | #

Kevin,

You are posting to your blog just like we agile folks post to ours.
It’s because we all believe strongly in something.

I have been watching your blog for a few weeks now and have been astonished at what I have read. The accusations, finger-pointing, name-calling, insults… I feel like I’m reading something written by kindergartners. The maternal instinct kicks in and makes me want to put all of you in a corner. “Balls” for Pete’s sake. Have we been reduced to who has testicles and who doesn’t? This is actually becoming quite humorous.

If we all agree that there are times to use traditional approaches, as well as times to use agile approaches, what’s with all the mud-slinging? Shouldn’t good project management practices underly both methods?

FYI, I just left London to teach Scrum to a group that has been using PRINCE2 practices for years. They felt that they could take the best of both worlds and devise a really good solution. I truly believe there’s a way for agile and traditional environments to coexist.

Best wishes, Mr. Kevin. I hope to cross paths with you one day and share an ale. Until then, I won’t be visiting Clarety Consulting’s website. This blog is making all of us look like a bunch of asses (and some of us DON’T have balls, thank God).

We should all take a position to learn more about the successes and failures and take the golden nuggets from each approach.

Good day.

Posted by Withheld  on Tue 10th October 2006 at 03:54 AM | #

Hey Withheld,

Couldn’t agree more that some comments here have lowered the intellectual tone.

I hope you realise my quip was to only highlight that Mr Beckford had failed to read Mr Brady’s other posts and comments which clearly qualify and balance his observations - some of which actually concede that Agile works in certain circumstances. Of course many of the evangelists only seem to see polarisation of Anti Agile and Agile. This is where I firmly part company with the bigoted Agile people who only wish to bloat their own consultancy / lecture circuit coffers. I work on the basis of using what has been successful for me dictated by circumstance. Not on the basis of blindly or slavishly following some sort of cult. 

As an objective observer I personally felt that the abuse of Mr Brady and his site had massively over stepped the mark and that those from the ivory tower were only seeing what they wanted to see. Hence Mr Beckford’s selective editing of articles and my challenge for him to pull out Mr Brady’s balancing observations and statements.

He failed or refused to do so which only confirmed his true bigoted intentions all too clearly.

All the best. From a pragmatic Agile enthusiast.

Posted by Tom  on Tue 10th October 2006 at 09:39 AM | #

Hi

I am running a very large IT project on a shoestring budget (for a charity)... any ideas on how to manage this v cheaply… inspite of your sniping guys i think you know what u r talking about….smile

Bella xxx

Posted by Bella xxx  on Mon 16th April 2007 at 11:24 PM | #

Hi Bella,

Send me an email to .(JavaScript must be enabled to view this email address) and I will give you a number to call. Really want to help any charity do the right thing. Anything to help.

It is very important to right size the project and its processes to the size of the deliverables required.

Posted by Kevin Brady  on Thu 10th May 2007 at 05:00 PM | #

POST A COMMENT:

Please feel free to submit relevant comments to this entry but note: inappropriate or purely promotional comments may be removed as will be personal abuse and defamatory remarks. Reasoned debate and substantiated critique on the topic in hand is encouraged and welcomed. Email addresses are never displayed, but they are required to confirm your comments.

Name:

Email address is required but will not appear publicly:

Add your comments below:

Remember my personal information for next time

Submit the word you see below:

12 + 1 =