AGILE Enough is Enough !

Posted by Kevin Brady on Mon 26th June 2006 at 12:00 AM, Filed in Software Dev MethodologiesKey Articles

During the week Computing Magazine, one of the best freebee computer magazines I know, dropped through the front door. All was going well until I reached the analysis section and Fig 1 :-

Fig 1 Government versus non-government adoption of AGILE

image

It states that AGILE another “No Pain High Gain” methodology is on the ascendance, and appears to be taking the intellectual high ground.

However, it looks to me like a suffocating fish trying to trash about in a useless attempt to find the next potential life saver. This trendily named method is nothing new, as I have come across it at various points throughout my IT career. However, in those days it was referred to as the SOP (Seat of the Pants) or JFDI (Just F***King Do It) development method. These names don’t sound so good do they? Such approaches to system development only work in a limited range of circumstance for systems of a particular type /level of complexity. In my view AGILE is nothing more than a posh word for hacking together software.

The interesting fact to bear in mind when considering an AGILE implementation is the fact that most IT project surveys state that, IT projects which are less than 6 people and 6 months in duration make up the large majority (over 90%) of the 10% to 30% of successful IT projects delivered. Websites are an excellent example of how small scale IT projects can be delivered successfully. Also, development of some of the worlds most famous software applications fit this category i.e MS DOS, PKZIP and believe it or not NETSCAPE.

The reason why small projects turn-up, irrespective of method used, is because:-

- The requirements and technical design can be visualised in the mind of one person.

- Requirement changes at any point in the design /build phases are quick and easy to make, and tend to have small impact on the project drop date /costings.

- They tend to be applications which are visual based (easy to prototype WYSIWYG), negating the need for detailed requirements

- Contain few lines of code and thus less “buggy”.

- Usability & Maintainability issues tend to look after themselves, especially when many of these apps are regarded as disposable with short life spans

- Low requirement for reusability in terms of any future migration path.

You don’t need a book, or employ a management consultant to tell you about this rebranded (AGILE) collection of resurrected paradigms. Just do what EGG.com has successfully done for years – employ young talented system developers, give them high level feature based requirements, shut them in a room and time box them on an “or else basis”. Then, providing you give them on demand, super gulp drinks, pizza and Mc Dee ready meals, and allocate space for sleeping bags then MAGIC happens. Steve Jobs from Apple built his empire on working practices like this.

The problem with AGILE /SOP or JFDI is that the world has moved on since the Alto and MS DOS. Today’s enterprise level applications, and even desktop software like VISTA, are hugely complicated with 10s of 1000s of lines of code and dependence on other layered proprietary software products, which often have unknown latent system defects. It is these types of applications which are the source of most of the 70% + annual IT project failures (CHAOS Standish Group).

It is clear from fig 1, and from my experience of working along side one of the leading AGILE software houses, that AGILE is being mistakenly considered /pushed as a viable development method for large scale software engineering projects. The strange thing about this is that some of the founding fathers of AGILE, such as Martin Fowler, have in the past recognised the limitations of AGILE –

Here are some of is his words of wisdom :-

“If you want to build a doghouse, you can just get some wood together and get a rough shape.  However, if you want to build a skyscraper, you can’t work that way – it’ll just collapse before you even get half way up.”

The thing I would like to know is whether people like Martin are willing to make the same statements today, with the fees juggernaut running at full pelt?
Today’s PLC’s /Governemnt Departments aren’t in the business of building Dog Kennels, and the large scale systems they do want need to have the following characteristics:-

- Functionality capable of giving a clear competitive advantage - This nearly always involves complexity

- Ability to constantly upgrade Functionality overtime - Release programme of enhancements to meet changing business needs

- Maintainability

- Reliability – Low number of software errors combined with high levels of availability.

- High standards of usability.

- Clear future migration path

- Predictable and reliable development costs

To name but a few!

It is clear that AGILE is not the “Holy Grail” of System Development Methods and, as with everything, it does have its place, even if it seems to be reinventing the wheel.
I suggest you keep looking for the Holy Grail because it is out there, but always “be suspicious of any methods that just look too dam good to be true (No Pain High Gain), because the sad truth is they usually are !”

This entry has been viewed 11358 times.

READER COMMENTS:

Hi Kevin, I’m sure I don’t have as much experience as you but my own experience is all I can go by.

My understanding, from reading and experiencing what I’d describe as an ‘agile’ philosophy is that the following elements are valuable contributions to any project:

- early, regular and tangible visibility of true project progress in a way that business owners can understand (it could be working software, but it won’t be a skin-deep prototype as this sets very high expectations)
- high level of flexibility when it comes to changing priority or even presence of items
- focus on high quality communication—colocation is one very effective way

I’m both a ScrumMaster and a PRINCE2 Practitioner and I’ve worked in a wide variety of environments. If you inject the above concepts then you’ll almost always be better off than not, IMHO.

Posted by Julian  on Tue 18th July 2006 at 10:24 AM | #

Your preconception that it is the same as SOP is just plain wrong.  I have done more planning on Agile projects than on any other projects in my 15+ years of programming.  It is just more focussed on achieving results, rather than producing spec documents. 

The things you state as Agile’s likely weaknesses (evolving functionality overtime, Maintainability, Reliability, etc.) are in fact its strengths.  Take evolving functionality for example… 

The reality is that it is a almost always a mistake to make architectural decisions based on predictions of the way that software will need to evolve.  Sure, you can guess some of it, but you *will* be caught off-guard.  Agile lets you be prepared for the future, while efficiently producing what is demanded today.

Agile is not always a good fit, but don’t needlessly dismiss it, or the tools that it is associated with.  To mention just a couple - both Short Iterations and Test-Driven Development can be used even in non-agile projects.

Regarding “no-pain high-gain”, anyone that has done Agile development will tell you it is very difficult, even painful at times.  It is difficult because software is hard, and complex.

Posted by Steve Campbell  on Mon 7th August 2006 at 07:06 PM | #

I’d have to agree that, other than Scrums and Sprints, I’ve not seen anything in “Agile Methodology” that could possibly qualify as a true “methodology”. 

I’ve read much of what Scott Ambler has to say and I’ve come to the conclusion that Agile is really just matching the process to the needs of the project.  In fact, he has posted as much, that sometimes RUP is necessary for a project, sometimes EUP, sometimes AUP.  To quote from a seminar blurb for SDWest 2006:

“The RUP is an evolutionary software development process which is serial in the large, iterative in the small, delivering incremental releases over time. Many organizations have successfully streamlined the RUP to make it very agile in nature. Agile RUP is possible to achieve when you ruthlessly cut out the bureaucracy and choose to focus on high-value activities.”

For those projects that need to be absolutely bullet proof (NASA Shuttle software, for instance), there will be more need for high process.  For “Hello World”, very low process.  The trick is to find the right balance.  I think that quite often project managers may feel the need for higher process as a way of trying to make sure that a project does not fail; yet this is not always the path to success.  This is the take home message I choose to get from Agile folks. 

Too often, Agile evangelists seem to wax poetic at the sheer wonderfulness of agile and the sheer stupidity and ignorance of anyone else.  Too bad, seems like we already have enough polarization in the world without it having to affect (infect?) software engineering.

Paul

Posted by Paul K. Courtney  on Mon 14th August 2006 at 02:18 PM | #

The project I’m currently working on runs for +8 years and has now seven developers working on roughly a million lines of Java code. Naturally, requirements are constantly evolving.

I can assure you that the introduction of Agile practices, mostly inspired by XP and Scrum, has helped us a great deal to improve the reliability, maintainability and extensibility of the system. They also don’t resemble “hacking” at all, but in fact require a quite disciplined style of working.

Posted by Ilja Preuß  on Tue 15th August 2006 at 07:45 PM | #

Hi Kevin, I am curious. What PM techniques does Clarety use? Maybe instead of pointing fingers you can share some insight on your successful techniques. Let’s make this blog a beneficial resource. We can all blast each other all day long, but I think there’s probably a way to be helpful instead.

Posted by ScrumMaster  on Mon 18th September 2006 at 08:07 PM | #

Hi Scrum Master,

We’re working through a number of techniques here, drawn from a number of philosophies and methodologies which have worked for me and many others.

I will never claim these are the panacea of all IT ills, unlike the militant Agile evangelsits who attempt to deface my blog on a regular basis.

It will very much be an exercise in passing on my 20 years worth of expertise and observations in an attmept to see the wood for the trees. You may even be surprised to see small elements of Agile thrown in for good measure - as I am not afraid to use anything that improves working conditions and productivity!

A reasoned critique from you and anybody else on the suggestions I put forward is welcomed. However, comments that descend to the weaker forms of debate such as defamatory and personal abuse will be removed without hesitation.

Thanks again ScrumMaster you are a welcome and valued contributer to this blog.

Posted by Kevin  on Wed 4th October 2006 at 11:54 AM | #

It seems odd that you complain about “militant agile evangelists” who “deface” your blog when you write in a deliberately provocative manner.  In other words, your comments are full of “fighting words” designed to stir emotions yet you are annoyed when you succeed?

Any critique of agile must begin with the Agile Manifesto where the values of this “frenzy” have been agreed to and stated for all to see.  Following that are the 13 Agile Principles.  Please explain the fatal flaws you find there.

Posted by Dan Pierce  on Wed 4th October 2006 at 02:12 PM | #

grin

Interesting.

So your saying observation, experiences, and accumulated knowledge count for nothing. Only the “Agile Manifesto where the values of this frenzy have been agreed” count in terms of evaluation.

Well here’s a home truth. That aint’t the real world dude!

Posted by Tom Hayden  on Wed 4th October 2006 at 02:43 PM | #

Thanks Dan,

I’m afraid that when people lose the ability of reasoned and calm debate and resort to:

1.  Comment spamming.
2.  Extremely threatening emails.
3.  Attempted defacing of a web site. (which includes incitement and rallying of others on other forums and sites to do the same.)
4.  Defamatory and personally offensive comments to me and others. (of which I have acted on in terms of parties from both perspectives.)
Provocative or not - what else am I left do but label a <u>minority</u> of the Agilistas “militant agile evangelists.”

Others like Brad Murphy of Valtech are able to disagree with me in a reasoned and calm manner without resorting to the   militant tactics of the bigoted Agile minority. Please see Brad’s comment here Agile Scrum fails to get to grips with human psychology

Posted by Kevin  on Wed 4th October 2006 at 04:05 PM | #

Kevin,

I commend you on your series of articles.  I posted an article , Agile Methods and Other Fairy Tales for which I have been personally attacked.  I have been doing some personal speaking engagements too.

http://www.SoftwareMetrics.Com/Agile

David Longstreet
Software Economist
http://www.SoftwareMetrics.Com

Posted by David Longstreet  on Thu 7th February 2008 at 04:25 PM | #

I know nothing; I come from the world of the mainframe. However, listening to the argument, it seems that people are still expecting methods and procedures or some revolutionary technique to stand in for competence, intelligence, adaptability, the ability and willingness to communicate, patience, a sense of humor, cooperation, business sense, fair play, integrity, energy, dedication, a sense of proportion, good intentions, and several other virtues that, when combined with any half-reasonable organization and honest management will often produce good results.

Posted by David S. Scott  on Tue 3rd June 2008 at 06:10 PM | #

Agile processes/disciplines are certainly not Cowboy! From my 20 years of experience as a software engineer working on embedded systems and business systems I have not experienced such a high level of quality using any other form of software development lifecycle and I would be interested if you can provide me with examples where these levels of quality are or have been achieved without the disciplines of unit testing, pairing, continuous integration etc.

Not doing these things would be cowboy in my opinion and the majority of waterfall projects I have experienced did not practice any of these disciplines and thats why we now suffer and have to live with “legacy systems” which we struggle to change and improve as its so risky to make that one line change. I would go so far as to say if you do not create and use XP practices you are essentially on your way to building a legacy system.

remember Agile is about KISS and continually improving by inspection and adaption…embrace change don’t try and control it you know who the winner will be.

Posted by Kevin Johnston  on Tue 8th July 2008 at 11:00 PM | #

Only just had a moment to read David Longstreet’s post mentioned earlier - “Agile Methods and Other Fairy Tales”. This is a valuable extension to the debate and I suggest readers download the following PDF http://www.softwaremetrics.com/Agile/Agile Method and Other Fairy Tales.pdf

Posted by Kevin Brady  on Sun 5th October 2008 at 09:14 PM | #

Interesting. Just came across this blog by accident by feel that I have somethibg to offer.

My background is in IT Services and Project Management and I have been doing this for 30 years in various guises and roles. I have done numerous management and methodology training courses including the usual suspects (and some not some well known). Like all methodologies you should only take what is relevant to your project/role but be aware that there is always more than one way to ‘skin that cat’.

I have just had a great experience with ‘Agile’ as it is being implemented in our R&D dept (over 100 people) to enable them to deliver ‘better and more reliable code’. In infrastructure we need to support the needs of the R&D team and assumed it would be beneficial to copy their practices. What a mistake!

As already stated Agile is purely stating the bleeding obvious and revolves around ‘task’ management. It covers over the obvious gaps within other SDLC methodologies which have been evangelised for years.

My project managers feel that ‘Agile’ offers benefits to project deliverables by ‘ensuring that everybody knows what they are doing’. This is nothing more than a copout of both the PM and the task manager. If Agile provides the excuse to tell people that they are not performing as required then fine. But, I do not see it providing anything over and above normal good project and task management practices.

Mind you us ‘older and wiser’ people need to keep an open mind, and accept that these new methodologies are now needed to support/replace the lack of good management skills.

Posted by Ian Horwood  on Sun 14th December 2008 at 01:48 PM | #

Thanks for the comment Ian. I totally agree with your comments /sentiments, and despite our long experience as professionals it is vital that we keep an open mind to new methodologies until things mature as they have in the engineering and construction sectors. Looks like a more practical and pragmatic view of Agile and its limitations ans uses is gaining traction in many areas on the internet. Look forward to your comments in future.

Posted by Kevin Brady  on Tue 6th January 2009 at 12:44 AM | #

Great posts.
I work in a huge telecom company, working with software development.
We are starting to introduce agile in our part of company. As I was looking for any learnings from introduction of agile I stumbled across this site and posts. Especially http://www.claretyconsulting.com/it/comments/agile-scrum-fails-to-get-to-grips-with-human-psychology/2006-08-17/
Must admit that the same comments exist in our organization. We have finished two prototypes, one that lasted about 20 weeks and one of about 8 weeks.
Every time I was surprised how one group of people talked about agile as they are in agile cult, while second group of people had huge number of criticism.
Incredible.
I agree that agile/scrum made great boost for our organization, and that it is not a silver bullet, and I agree that if organization was more open it would give better results.
But what I wonder the most is if big parts of organization don’t want or just cant adapt to agile/scrum is it really a good way to go? Isn’t it doomed from beginning, ending in merely adapting old ways of working to new methodology - agile/scrum (or it just takes time for this change management to fit into place)?

Posted by HrvojeHadzic  on Fri 2nd October 2009 at 09:16 PM | #

POST A COMMENT:

Please feel free to submit relevant comments to this entry but note: inappropriate or purely promotional comments may be removed as will be personal abuse and defamatory remarks. Reasoned debate and substantiated critique on the topic in hand is encouraged and welcomed. Email addresses are never displayed, but they are required to confirm your comments.

Name:

Email address is required but will not appear publicly:

Add your comments below:

Remember my personal information for next time

Submit the word you see below:

20 + 2 =