AGILE /SCRUM Fails to get to grips with Human Psychology.

Posted by Kevin Brady on Thu 17th August 2006 at 03:10 PM, Filed in Software Dev MethodologiesKey Articles

I promise in the next few weeks to concentrate on other subjects other than AGILE and its deficiencies, but my readership statistics show that AGILE is a popular subject. I also feel I have to keep up with my protagonists comments, which are appearing not just on my blog but all over the net.

image

At the moment, the net is buzzing with AGILE evangelist websites and Blogs making statements like “the AGILE manifesto is the equivalent to Newton 4th Law of Motion”. When someone takes a fanatical belief in anything without proper empirical evidence you have got to start thinking!!. Please see my post Storm in a Tea Cup. I really think these evangelists do not expect us to engage our brains. They think if they keep repeating the common look-up list of humorous slogans about AGILE’s invincibility, we will all be conditioned into turning a blind eye to the detail and asking for verifiable and independent evidence concerning this approach’s claimed scalability and prowess over other methods /approaches. At the moment the whole IT methods industry from Waterfall to RUP to AGILE and SCRUM is in need of consolidation based on some sound independent research. As we speak, large corporations take huge “flip flop” financial gambles adopting this method over another, largely based on the word of lightweight fee-earning consultants.

Most development /project management methods, which have come and gone over the last 20 years, are all variations of a theme. AGILE /SCRUM is nothing different and is a cobbling together of mostly small scale IT development methods which were drifting into obscurity until AGILE came along and gave them a new lease of life. AGILE has for some of these methods become the equivalent of being sent to your local A&E.  For example, prior to AGILE DSDM had been removed from all of IBM’s front office websites.

I do think AGILE and SCRUM has its place.  It is just that I believe this place is running projects under 5 people and delivery durations of less than 6 months. At that level, employing the right people with the right skills and motivation frequently guarantees that MAGIC will occur. The new media industry proves this everyday, irrespective of approach /method used. Good people always have the potential to overcome deficiencies with methods /approaches and poor management. I have seen it time and time again.

Wonderful to be involved in such projects.

The problem I have with AGILE /SCRUM is that, like many methods, and indeed political doctrines such as communism (Das Kapital – Karl Marx), they look great on paper but fail to work in reality because they forget the human factor. Any Paradigm, which has human interaction at its heart, will fail if human psychology is not understood and taken into account. The key aspects of human nature which IT development /project management methods have to take into account are no different to those at the heart of most modern economic theories:-

- People will always put their own interests ahead of the interests of the group.

- People are self-interested

- Commercial production decisions are based on rational expectations.

- Karl Popper’s “First law of collective action”. You can never get more than 5 people to agree on anything.

AGILE /SCRUM has its own culture based on the best and worst of human nature, which often precedes its introduction into an organisation and often starts to imbed itself even before the implementation consultants have the caps off their marker pens. AGILE /SCRUM culture cuts across all of the points mentioned above and can from experience in the field turn often already sickly IT departments into meltdown.

Notes from my diary on the views of people working within 3 self-proclaimed AGILE /SCRUM organisations, describe the culture they experienced in the following terms:-

The Project Managers /SCRUM MASTERs turned themselves into Project Administrators. In each organisation this was characterised by a total lack of project plans, risk /issue management, quality management, change control, budgetary and status reporting. When questioned, all these managers felt they were responsible for the failure of their projects but had, under AGILE, no authority to direct or change anything. The team was responsible for delivery and they were responsible for taking the heat if it failed! They also felt they had to provide softly softly, and frankly patronising, Uncle Buck support for their project teams.

I was surprised to find in many cases that the Techncal staff conducted meetings with clients, leaving the Project Manager /SCRUM MASTER as a mute bystander, and totally ignoring risks, costs and commercial considerations. In all 3 organisations the managers, despite failure and chaos everywhere, were leaving for home on time and were mute concerning any questions concerning poor project delivery performance, instead choosing to hide behind the AGILE crucifix, with statements like “the team is responsible for this - go ask them they made the decisions. “Go talk to them! I am just following AGILE /SCRUM!”

Great we have a ship with no captain on deck being steered by the ships crew with any sinking / loss of life being the responsibility of the slumbering captain. No wonder they were in meltdown. CRAZY!

The project teams had in almost all cases been taken over by strong personalities leading to mini dictatorships. These mini Hitler’s (senior technicians) kept all the most interesting development work for themselves and had often taken possession or laid claim to successfully delivered system deliverables leaving the rest of the mess to the other team members. These ‘dictators’ often seemed to accumulate more work than they could cope with and refused to hand it to others. This meant these projects slowly ground to a halt as all too frequently these tasks sat along the critical path.

Knowledge Monopolies. The ‘dictators’ detailed in (b) also, in the absence of documentation, had absolute control of the code should the developed system ever be launched. They would be the only people in the organisation with the knowledge to maintain and enhance the code going forward, thus maintaining their job security, and guaranteeing pay rises above the market rate. I really noted some serious arm bending on pay by these guys - squeezing in one case the budgets to such extent that previously turned down outsourcing deals were back on the burner.

Resource Management had vanished. With no proper project plans, except lists of team SPRINT deliverables at the most rudimentary level, no one knew how long a project would take and when resource would be needed, or could be released, between projects. The dictators appeared to want as many people in their teams as possible and were reluctant to let them go in order to maintain their status.

Having had a taste of freedom the dictators were a hateful and aggressive bunch when asked about their managers /SCRUM MASTERS. They felt they were in charge and knew best, and had been liberated from the incompetent direction of their old management. They were without exception, never going to be directed by anyone except perhaps the client!!. Many of these dictators seemed to start and leave work as they pleased as an ultimate expression of their importance. No one is an island or a law unto himself /herself!

Most of the talented young development staff were leaving because the dictators were hoarding work and only handing out uninteresting tasks to the new boys on the block.  This created a bunch of disaffected and unhappy people who had no idea what their future prospects were, or how to get promotion in such a CHAOTIC situation with so much posturing and territorial teeth showing between team members (like Gorilla packs in the forest) when they wanted to learn something or expand their remit.

Each of these organisations had differing development approaches and tools from project to project. One team would be using Java J2EE and another using .Net C# and another struts, all within the same IT department, so that maintainability and resource management was impossible. Each AGILE scrum appeared to make up their own minds as to which constructs /tools etc to use in order to deliver the project. On close investigation, the decisions seemed to be based on what might enhance the capital value of their CVs as against what might enhance the capital value of their employers. Not having interchangeable staff between projects and maintenance and support crews is a real constraint on an IT department’s productivity and ability to control costs. Cutting edge technology means the replacement developers /tech designers were also going to be expensive.

Clients feed up with never-ending, continuous involvement in IT projects. The clients could not afford to work so frequently with the IT project teams as envisaged under AGILE. In all cases client input was sporadic and inconsistent due to the duration of their required assistance right through the project life cycle. The clients could not afford the additional overhead cost. This is common with other development methods but with AGILE /SCRUM their involvement is more intensive, and for longer durations.

I am not knocking AGILE /SCRUM for the sake of it, more sharing my observations of organisations claiming to be AGILE shops struggling to make AGILE /SCRUM work in the REAL WORLD. If any method fails to survive front line implementation (large IT projects /programmes), you have to start asking questions.

All I can say is that when I have seen AGILE /SCRUM fail it is sad and painful for the professionals working in these places. If AGILE /SCRUM is to survive and have any chance of going forward, it has to be reworked so that it encourages the most valuable and productive aspects of human nature, whilst constraining the expression of some of its worst and most destructive aspects. This is the essence of people management.

This entry has been viewed 45293 times.

READER COMMENTS:

Interesting and informative observations. I’m beginning to see where some of your opinions about agile development came from. The organizations you describe sound fairly dysfunctional. Too bad the words “agile” and “scrum” were associated with them. Doesn’t sound like they really understood what to do.

I’ve mentioned before that agile methods have worked well at our company. The final two bullet points in your post pertain to our organization, however.

Initially, we used agile methods on small-scale projects to develop webapps. As you describe, each team chose its own tools. Some of us began to question how sustainable that would be, and predicted the approach would begin to break down in the production support area. Sure enough, that’s what happened. Since then, we’ve reined in the range of technology choices open to the agile delivery group so that we have a reasonable set of standards while still offering sufficient options to support the disparate needs of our projects. This is a factor any organization considering agile methods must take into consideration.

Some of our internal customers have balked at the amount of their time agile methods require. Some have adapted themselves to it, because they appreciate the relatively high business value they get from the approach. Others have been enthusiastic initially, but faded away from iteration to iteration because of other demands on their time. I think the key here is that an organization has to approach agile development with eyes open, and part of the reality of it is that it will demand more time on the part of customers. If that’s not feasible for a particular organization, they should consider alternatives such as lean development, or perhaps just tightening up the traditional process controls using a process improvement method such as Six Sigma.

At the risk of sounding like one of those zealots making excuses, the other bullet points in your list frankly sound as if the people mis-applied agile practices. In particular, the project managers who behaved as you describe really didn’t “get it.” Agile methods apply most directly during code development. They don’t substitute for all the other activities necessary for a project to be delivered successfully, such as planning and resource management and so forth. To be successful with agile, you have to integrate it with the rest of the activities in the organization. If you just sort of drop it on the floor and walk away, I have to wonder how anyone could expect anything useful to happen.

I’d like to make a final comment about your observation that agile methods only work on very small teams. With traditional methods, we scale up just by adding more people to the project team. Many people assume that’s the way to scale agile teams, too, but it doesn’t work. To scale an agile project, you decompose the work into sub-projects and coordinate them. Each of the sub-project is ideally staffed with no more than about 9 people. Obviously this adds management overhead to the process, but the net of it is still more cost-effective than traditional methods in most cases.

Posted by Dave Nicolette  on Fri 18th August 2006 at 03:29 PM | #

In my experience this is very much how well meaning agile projects degenerate. Great post.

Posted by ChrisT  on Sat 19th August 2006 at 11:36 AM | #

Thanks David for your comments. Enjoyed reading your real world /Coalface experiences concerning AGILE /SCRUM.

I totally agree with AGILE being used for web related projects, which have a large visual aspect. DSDM, RAD etc work well when a system can be prototyped based on WYSIWYG (What you see is what you get). Having developed many commercial n-tier websites, I can verify your comments from personal experience. 

One interesting aspect of your comment is the fact that you seem to have admitted that AGILE /SCRUM in its raw form (self-management, self-assembly) is perhaps not enterprise ready and in your organisation it has had to be adaptated i.e control over development standards. At what point does one make changes to a proprietary method before it can no longer be called AGILE /SCRUM /RUP and evolves into something else. Furthermore, such changes /adjustments are just piecemeal refutations of the AGILE /SCRUM paradigms, which is paraded by many as irrefutable. 

I am pleased to hear that AGILE /SCRUM is scalable in your view and your organisation is pleased with their delivery success. At the end of the day IT disasters can be really stressful depending on the type of organisation you work for.

The question I am interested in asking is, are these so-called successes political or real project successes (please see my spin management post - Anti Spin Management Check List     
If the later then success must be judged in the following terms:-

- Did the business people set at the beginning what order the project /programmes drivers should be in. i.e Quality, time, cost, scope. Did these large-scale projects conform to these requirements? If so what evidence.

- Did these projects test well in accordance with defined and agreed acceptance criteria?

- Were these large scale IT systems tested properly - usability - regression and non functional testing such as load /stress testing against fit for purpose test scripts based on agreed and detailed user requirements.
 
- Are these large-scale systems maintainable and capable of further development (technical documentation which AGILE says does not have to be detailed) by independent resource other than those who built the system in the first place? Without this, your key and most capable development resource are tied to legacy systems, which kills the flexibility that AGILE claims to provide!

Unless success is judged against the above, it is entirely subjective as to whether a project is successful or not. In one organisation, I consulted in two years ago, the sole judge of a successful project /delivery was “whether the business people liked the project /programme managers assigned to them and the extent to which they were willing to work with them again on a future project”. Suffice it to say I never found in this organisation a single homegrown system, which could be classed as a success. If success in your company is judged in these terms then I think the jury is still out on AGILE /SCRUM and it scalability.

I recently saw a huge market wide finacial system built with 6 man AGILE teams with managers acting as facilitators as described in SCRUM, limited feature based requirements, iterative development methods and SPRINT project /programme planning.

RESULT -

- Project closed, business case closed and £30 million loss quietly absorbed and forgotten.

- Testing was impossible with such limited up front requirements - further developed on an iterative basis. The users could not tell the difference between a bug and a missed requirement. Real dissatisfaction!!!

- Acceptance criteria impossible to define because the requirements were never defined until too late in the programme. The system would only run to 3 concurrent users. Not suitable for a system to be used by 1000s of people daily but who’s to grumble AGILE says this is OK to start building databases and systems without detail like this. Just keep moving forward is all that matters in this cloud of confusion.

- Ran out of budget without early warning because no one could see more than 28 days ahead or new the full scope with unmitigated and uncontrolled changes in scope (OK according to AGILE).

- Some of the worst usability I have ever seen because the user flows could not be tested up front because the requirements were at feature level (OK with AGILE) and a pure HTML wireframe system could not therefore be constructed to make this testing possible.
- A horrid final system. Because iterative development methods were used with each part of the system independently designed without a broad view of the full requirements, the system produced was reminiscent of a car with different sized wheels, an engine, which would not connect to the axial, and doors, which would not fit to the body. At integration the engine would not even start.

This is just a taster of the magnitude of the kind of disaster, which can be caused when you adopt and follow key AGILE principals within a self-managed and self assemble IT development environment.

Some very young and inexperienced staff cut their teeth on this programme and I have often wondered how many of them are still in IT. Really sad for all those people involved.

Posted by Kevin Brady  on Sat 19th August 2006 at 08:57 PM | #

>At what point does one make changes to a proprietary method before…

A brief digression, if I may, to set up my answer.

In my view, there is a distinction among three levels of abstraction in how we do our work: Approach, process, and methodology. I realize this isn’t a “standard” formulation.

An “approach” establishes a general mindset about how problems are to be addressed. This is where the buzzword s like “agile,” “adaptive,” “predictive,” and “waterfall” fit into the picture.

A “process” is a management framework for running projects. This is where Scrum and Agile UP fit into the picture, and in traditional organizations it’s where SDLC and so forth fit in. Scrum supports the agile approach because it is a very lightweight process that doesn’t dictate many details, and is highly adaptable to different types of projects. It was not an invention of the agile software development movement, but predates that movement by some 8 years. It has been used with success on other types of projects besides software development.

A “methodology” is a detailed set of practices that people follow to carry out some particular type of work. The most popoular agile methodologies in use today are XP, Crystal Clear, and DSDM.

Looking at it in this way, it makes no sense to refer to “agile” as a “proprietary method.” It’s not any rigid “method” and it isn’t owned by anyone, so it isn’t “proprietary.” If you want to ask about practical adaptations to integrate agile practices into an organization, you should be asking at the level of methodology: XP, Crystal Clear, etc.

Having reformulated your question, the answer in my opinion is that you should adapt the methodology to whatever extent is necessary to add value. If you have to adapt it to such a great extent that it’s no longer “agile,” then it would be wise to reconsider what you’re doing and why you’re doing it, rather than just to plow ahead chanting the word “agile.”

As many have mentioned during this extended discussion, “agile” emphasizes people over process. That means the people actually have to think. They can’t just follow a documented methodology by rote. The general tenor of your questions and comments suggests you’re looking for a methodology can can be “followed.” That’s not what this is about.

>Unless success is judged against the above…

You’re quite right, and I can confidently state that the projects we’ve undertaken at our company using the agile approach satisfy those criteria for success. My personal tendency is to look at the hard financial value delivered by a project, rather than subjective statements of “soft” value.

>I recently saw a huge…

Yes, and based on your earlier brief description, it’s clear they did not think about what they were doing or its effect on the enterprise. To dismiss that mess as “agile” is rather glib. It would be more productive to follow your suggestion to analyze what actually went wrong so that we can learn from it.

Posted by Dave Nicolette  on Tue 22nd August 2006 at 01:42 PM | #

Hi Dave

Thanks for the comment. Thanks for helping to turn this post into a helpful and stimulating discussion.

I personally never try to discuss subjects I know nothing about without first having read the key books on the subject and having gone out in the field and seen its application. I have done both (just finished a great book - AGILE Project Management with SCRUM by Ken Schwaber).

I have heard the same comments you mention in other places on the NET i.e That SCRUM and AGILE is a state of mind and not a method. Really, think this is starting to sound like communism and the move away from Das Kapital during the first communist revolution. Look where that got Russia!

No method no order and no order equals what I have seen in the field CHAOS, counterproductive attitudes, down right laziness and unhappiness among the less outspoken project team members. 

States of mind my view do not build projects. Great staff, plans, risk /issue management, vision, sponsorship, change control, quality control, budgeting, good technical documentation, sophisticated professional testing and first class design are what make things happen.

Self-managment and self-assembly by releasing people from the dreaded project manager (even if some of them are CRAP) is not the way forward. Furthermore, projects which lack fixed price contracts with outside software development vendors because AGILE promotes trust in one of the most cut throat industries in the world was 25% of the reason why the projects I mentioned in my post ran massively over budget.

The teams worked collaboratively with vendors only to find themselves ripped off. Don’t people ever ever learn ! Software development deals with external vendors are like making love with a porcupine “do it carefully” and contracts help save a lot of pain in this respect. 

I have always practiced participative management and have never asked a professional person to do so something which I would not do myself or without getting their bottom up view /estimates of the tasks ahead of them. However, all the Jelly, which represents a project and its tasks, has to be held together with discipline and constraints coming from the top down. These constraints and discipline are not states of mind they are the basis of a method designed to reach a goal.

Could a sky scrapper be built with self assembled teams, with self determination and site manager and clerks of the works acting as facilitators and not directing in accordance with the architectural plans ? Believe it or not some AGILE evangelists are saying such things are possible under Agile through state of mind, increased motivation, trust letting people chose the right way for themselves collaboratively – sounds like collective farming in Russia. Ow by the way 10 million people starved.

The AGILE books I have read do talk of AGILE as a real method by listing development methods such as DSDM and Feature Release as now AGILE methods (used these myself ). Mr Schwaber spent 150 pages in the book I mentioned earlier not talking to much about states of mind but how to apply SCRUM as a clear if not entirely detailed, enterprise ready method. Not kidding !

Once again thanks for the intelligent comments really helpful and well written.

Posted by Kevin Brady  on Wed 23rd August 2006 at 08:57 AM | #

I’m glad you have found the discussion worthwhile. I think it’s important for us all to consider ways to improve the state of the art in our profession as objectively as we can.

I tried to draw a distinction among different levels of abstraction in which the “state of mind” idea has its place, but does not define the entire notion of agile development. I regret I failed to convey the point clearly this time.

Of course there are well-defined methodologies that are consistent with and that complement an agile state of mind. This is not a startling revelation. No one claims a state of mind alone will deliver any results. To argue against that position is to fall into the classic strawman logical fallacy. Certain agile practices are more effective in the context of certain organizational cultural attributes. Having the right state of mind can enhance the effectiveness of those practices.

But there are also practices that can be applied without any deep cultural changes. One practice that can improve results dramatically is iterative development with incremental delivery, for instance. That is strictly a procedural change, and doesn’t require cultural change. OTOH that practice will deliver better value in the context of a culture of transparency, since it would be easier to see what adjustments to make at the end of each iteration if you had accurate and honest information to work with. That’s how the state of mind can enhance the value of everyday practices. Obviously, too, you can strive for transparency without ever even thinking about “agile.” For reasons like that, I’m more interested in what people do and the consequences of their choices than I am in labels and buzzwords.

I don’t expect the agile approach to be for everyone. IMO if an organization can properly understand and apply it, they stand to benefit significantly as compared with the traditional, linear waterfall approach. If they can’t quite “get there,” then they would do well to apply whichever agile/lean practices they can handle, to achieve at least some of the potential value.

If they can’t get off the proverbial Square One in understanding agile, or if there are political realities in the organization that inhibit positive cultural change, then perhaps the best course for them is to do their best to apply traditional thinking and traditional approaches as effectively as they can. That doesn’t mean those approaches are ideal. They aren’t. But we can’t expect people to apply what they don’t even grasp.

To read a book is a great start at understanding any new subject. The book you mention is full of anecdotes about things that worked and things that didn’t work on real projects, and I think that’s a good type of book to start with. I’ve recommended that very book to others who were interested in learning about agile development.

But do keep things in perspective. Reading a book about martial arts wouldn’t help you in a practical situation, although it would probably be an interesting read. To benefit from the information, you would have to practice physical techniques every day while trying to adopt the appropriate mindset, all under the tutelage of a qualified instructor. When you had learned the basics of a technique, it would then be a matter of repetition until the movements became second nature to you. If you failed to achieve some goal - say, breaking a board - would it mean that martial arts “don’t work?”

So it is, too, with other skills in life, whether gardening, music, cooking, project management, or software development. Do you hear people proclaiming that roses, trombones, or lasagne “don’t work?” There are good and bad ways to grow roses, play the trombone, and prepare lasagne. There are good and bad ways to manage projects and develop software, too.

When you see failures such as those you have described, it might be useful to understand who did what and how their choices affected outcomes, and how they might do things differently next time, rather than to become hung up on words. 

In any case, as long as we’re all trying to maximize the value we deliver to our customers, then we’re contributing to the advancement of the profession in some way.

Best of luck to you.

Posted by Dave Nicolette  on Wed 23rd August 2006 at 01:53 PM | #

David,

Thanks for your comments.

I do believe such open discussions are beneficial to the project management profession (god it needs it) and I think everyone can learn from such dialogue.

Look forward to talking gain.

Posted by Kevin Brady  on Wed 23rd August 2006 at 05:23 PM | #

Wonderful debate guys. Have been observing with interest.

Posted by Simon Henderson  on Wed 23rd August 2006 at 06:45 PM | #

Hi

Just thought I would let you know that I will continue the debate once I return from Rome during the first week in September.

Posted by Kevin_Brady  on Thu 24th August 2006 at 02:10 PM | #

I’ve been successfully using Agile Development with SCRUM for over 2 years. I’ve turned around a development shop that released software every 18+ months to a 6 month release cycle.

SCRUM is not a magic bullet. It doesn’t fix dysfunctional toxic work environments. If you don’t have Software requirements in the form or clear use cases(user scenarios), configuration management, daily builds, database change management or unit testing your projects will be late and over budget.

To use Agile Development methodologies require a strong SCRUM Master with people skills to match. Putting a chief Architect with poor people skills is a recipe for disaster.

To learn more on the realities of Agile development with SCRUM, check out my blog: http://morethanagile.blogspot.com/

Posted by Derek Gilmore  on Sat 2nd September 2006 at 10:56 PM | #

Kevin,


I read your concerns about Agile methods with interest.  I’m a senior executive at Valtech (a HUGE Agile Advocate) and a former founder/CEO of an OO development and software process training company.  I “get” the human side of things… starting a company, raising private equity capital, winning client contracts and keeping stakeholders “happy” is intuitive and instinctive to me, so consider my comments in this context. 

My business unit is into year three of building out a global, scalable, outsourcing / offshoring development capability with Agile/Scrum at the core of our engagement / development process.  When I have more time, I will post our specific experiences, so for now let me make just a few general remarks.

First, Agile/Scrum can and does work and in many instances, produces breakthrough results.  When I do a follow-up post I will include objective metrics we’ve been collecting the last few years to validate our own investments.  Suffice it to say we have NOT been investing millions of dollars based on an emotional belief in Agile.

Second, one of the organizational benefits of Agile/Scrum is how transparent and revealing it is a about the “character” of a team.  Specifically, Agile/Scrum will actually highlight all that is good and bad with how an organization collaborates.  So, I would argue that the example of sickly IT groups “melting down” is not an unexpected outcome when dysfunctional IT groups latch onto Agile as a silver bullet.  We are very upfront here at Valtech with prospective clients that we will NOT enter into a long term client/supplier relationship without a pilot that allows us to mutually access the “health” of the culture within the client organization.  Toxic cultures that attempt to transition to collaboration methods based on trust and open communication will almost always fail without outside intervention and recognition of this reality… Kind of like an alcoholic trying to “cure” themselves.

Lastly, I will respectfully suggest that you’re assertion that Agile is the “cause” of the melt-down is not entirely fair.  I actually agree with your point about the human factors influence, however I would argue that in the specific examples you share, that Agile is simply revealing and accelerating the very conditions that already exists.  For me the more appropriate criticism / warning for those adopting Agile/Scrum is the core risk related to organizational “health”.

Those who promote Agile/Scrum (Valtech included) have an obligation to recognize and promote the cultural risks that MUST be addressed, otherwise the outcomes you’ve highlighted are quite possible, if not probable.  To suggest however that this is “proof” of why Agile wont work isn’t terribly constructive or helpful.

YMMV.

Brad Murphy
SVP/Global Client Partner
Gen. Mgr. - Global Agile Outsourcing / Transformation Group

Posted by Brad Murphy  on Mon 11th September 2006 at 01:16 PM | #

The author of this article is totally incompetent and unqualified to speak on this subject. 

The fact that he capitalizes “SCRUM” and “AGILE”, in addition to that he talks about “AGILE” as if it is a methodology in and of itself is telling.

Disappointing and the reason I tend to avoid reading blogs.

Posted by CB  on Mon 11th September 2006 at 02:10 PM | #

CB you strike me as someone unable or unwilling to accept any constructive criticism of something that does have bona fide faults. 

Whilst I favour Agile, it is important, ney crucial that we all look to understand where and how it does fail, instead of trying to knock everybody flat who happens to disagree. That tends to indicate there is something more at stake than just a viewpoint.

Posted by TomHayden  on Mon 11th September 2006 at 02:32 PM | #

Hi,

Pointing out the cultural implications of adopting Agile practices is like blaming the police for all the ills of society.

In my experience when applied correctly Agile tends to shine a light on deeply entrenched organisational problesm, and like you say that can lead to melt down. But was Mikal Gorbachov wrong to try and reform the Soviet Union?

Agile doesn’t offer answers to these issues, and it is definately not then root cause. What Agile thinking provides is tools nothing more. Human nature has been around a long while, and if you can package how best to inspire and lead people, then go ahead you’ll make millions.

BTW: Which “Agile” books have you been reading? Agile is a marketing term. The values behind the slogans go a lot deeper and have a proven track record outside the software field. Take a look at Lean Development by Mary Poppendiek. She looks to the Japanese automotive experience at companies like Toyota where people are trusted to self organise, massively increasing productivity and quality without the need for over-bearing and expensive central control.

The crux is Taylorism versus Deming, and you are right - many organisations in the west struggle with the cultural change needed. But hey don’t blame the messenger :^)

Paul.

Posted by Paul Beckford  on Mon 11th September 2006 at 03:03 PM | #

@TomHayden

I have no vested interest in agile.  But at what point did we stop requiring that people actually know what they’re talking about before they make broad sweeping claims like “AGILE /SCRUM Fails to get to grips with Human Psychology”?  I at least expect proper spelling of the terms.  Come on man!

I don’t care if he agrees with me.  I do care if he’s experienced or at least educated enough to either agree or disagree.

Posted by CB  on Mon 11th September 2006 at 03:34 PM | #

Hi Kevin,

Just a few thoughts on your post.  First, you write
“Great staff, plans, risk /issue management, vision, sponsorship, change control, quality control, budgeting, good technical documentation, sophisticated professional testing and first class design are what make things happen.”

I 100% agree with this statement.  My question is, where does it say that those things are NOT in an agile environment? 

The main point being that a project can fail/succeed with any time of methodology.  If we are hitting 70% project failure rate with waterfall, who says that we won’t hit that rate with agile?  I don’t remember ever reading any statement which says “Agile will guarentee success”.  I also don’t remember it saying that
a) you don’t need documentation
b) you don’t need planning
c) you don’t need a budget

Since you have read Schwaber’s book on SCRUM can I recommend another to read?  Mike Cohn has a book called Agile Estimating and Planning.  This book specifically covers budgeting, planning, release management etc.  It is an excellent read.

Also, just for fun, go through your post and replace every instance of AGILE/SCRUM with the word WATERFALL.  Notice that your post still makes sense and then becomes a pro-agile position.  The point being that software is still written by people and as such no matter which methodology you use, you will find the same problems.

Posted by Aaron Korver  on Mon 11th September 2006 at 03:43 PM | #

CB - as an Agile enthusiast, I think we are singing from the same hymn sheet - but quibbles over spelling are completely superfluous and pedantic to a proper deconstruct of Agile. 

I want to see Agile develop, evolve and improve. Unfortunately you seem hell bent on frenzied head burying. Is it vested interests? I don’t know, but when someone venomously attacks another individuals valid perspective and expertise, you have to ask questions as to why?

Throwing childish diatribe offers nothing to the debate CB. Engagement and elaboration does. So I suggest you set out clearly the grounds for your perspective as others have done.

Posted by TomHayden  on Mon 11th September 2006 at 04:44 PM | #

Get a life everybody.

Agile sucks.

Posted by Tash Novak  on Mon 11th September 2006 at 05:31 PM | #

Most of the things that are described in the diary section of this post are things that could be ascribed to any project.  Agile has no monopoly on dysfunction.  If your team is made of up ego maniacs and leaders with no leadership ability, then you will have major problems no matter what methodology is used.  The so called scrum master that’s referenced clearly had no idea what he was doing and no control.  He should have been fired and replaced right away.

Don’t blame Agile for the incompetence of people who claim to be agilists.  But lets be honest, your don’t really care about that.  You open by complaining about how there’s no proof that agile methodologies are effective and then you offer no proof of the contrary other than some diary entries of a failed project.

Posted by Sam  on Mon 11th September 2006 at 06:34 PM | #

@TomHayden

“quibbles over spelling”—my god man.  Your expectations are minimal indeed. 

Thanks for contributing to the lowering of the bar.

Aim low.  You’ll hit something.  Probably the ground.

Posted by CB  on Mon 11th September 2006 at 06:49 PM | #

CB - It’s a shame, you seem to be ducking the debate. grin

I wonder why?

Posted by TomHayden  on Mon 11th September 2006 at 07:30 PM | #

I’m not going to debate about this article.  It’s like debating a fictional story.  It’s a waste of time.

But you seem interested, so I’ll bite.

Let’s debate.  I’ll start with a context and a followup question to get us started.

So far you’ve only attacked my opinion without offering up your own.  My opinion is clear in that I think this article is complete BS without any merit and was written by someone completely incompetent.  My feelings are warranted easily by the points made above to the point where the author cannot even SPELL that which he is talking about.  More importantly he completely misses the point of agile and apparently knows nothing other than that it will get him blog hits (which he admits to in the first paragraph of his post).

My question to you: Why are you defending him?

Posted by CB  on Mon 11th September 2006 at 09:49 PM | #

Hi Folks,

Great to see that this post is really started a helpful and very interesting debate. However, some of the comments in my view are a little heated and emotional in some cases and in one or two comments I can see many of these people really need to get out a little more.

However I would agree that I am passionate about Project /Programme Management but I do have a family and have outside interests ranging from shooting and fishing to studying ancient history.

However, for those young developers sticking comments onto my blog concerning spelling and hints that I am some kind of Cretin, I want to make it clear that typos of this kind appear in many articles and books I read these days and never lessons the read if the content is great.

However, for those who believe they have the intellectual high ground on the Agile argument on any other for that matter because of a typo better read the list below which catalogues some of the worlds most well known bad spellers :-

Albert Einstein, Winston Churchill, Cher, George Burns, henry winkler, Whopi Goldberg,  Walter Elias Disney, Alexander Graham Bell, Leonardo DeVinci, Thomas Alva Edison, General Patton, Woodrow Wilson,Danny Glover, Fred Astaire, George Bush, Michael Hesletine, Henry Ford, Hans Christian Anderson, Harvey Cushing, Pablo Picasso, Stonewall Jackson, Enrico Caruso, Auguste Rodin, Michael Faraday, Richard Branson, Mackenzie Thorpe, Jamie Oliver.

Enough I think  Lets now get back to the real debate.

I want to make it clear that the one thing I have discovered from reading this great list of comments is that the views seem to heavily depend on what your role is in the IT Industry.

Hear is one possible take on this –

- IT Trainer - Agile is a great fee earner please do not kill the goose. Are you mad this is a revolution, I want in. I do not care if I haven’t got real independent and indepth empirically based evidence that Agile and Scrum gives the most benefits over other approaches /methods to organisations who adopt this new religion. At the end of the day why go against the grain when so many people say its right and OK. Ride the wave and be rich.

- Developers and Technical People - Creative Freedom through code. Liberation to cut code and architectural design as a means to end in itself. At last I can concentrate on using the most advanced tools and building at the company’s expense the best CV for the future. If I am lucky I might grab a part of the code and call it my own and become one of the unsackables. My low documented systems be a thing of the future. I feel at last I am back at university and follow things that interest me. 

- Project Managers - We are a discredited bunch anyway and while I still hold the baby I at least can leave all the detailed project planning to the SCRUM MASTER (Tech Architect) and development team. I keep quite in front of clients and let the team handle the client which does give me some political leverage should things go wrong.  I no longer able to provide long-term project /stage - time /cost /quality projections beyond the current Sprint because the system is built iteratively without any need for change control. UREKA I never have to tell a customer NO because change is OK and anyway I can’t estimate change costs for my clients because I can’t map out the full task breakdown and do What-if changes to the plan. The reduced “NO’s” to the client means perhaps I might beat the annual turnover rate for PM’s of 30%. Hears hoping 

- Management Consultants – All I have to say hear is read my POSTS -

Page 2 Management Consultants Friends or Foe
Page 1 Agile Fees Feeding Frenzy.

Then there is myself. Well I am a self-interested individual like everyone else on this planet and my typical role is putting project management and development maturity into medium sized software houses and troubleshooting and turning round failing IT programmes and service departments. My interests are in the adoption of methods and approaches that give real benefits to the clients who pay for my time. If Agile really made things better with respect to the running and delivery of large IT projects /programmes of work I would use it.

I am not interested in private “firms within a firm” interest groups (the people listed above) I am interested solely in getting the quality, cost, time and scope determinates required by the client delivered every time or I am out of work. Hard, simple and true.

It has always been key to this role to make sure the departments I leave behind are able to repeat this for themselves. I engage in mentoring management and making sure staff moral is maximised through a bottom –up and participative management style. Please see my post

Page 1 Time Boxing IT Projects Parts 1 /2 /3

Finally, I can tell from some of the developer comments, that they fundamentally are in flat land and while I admit I cannot think JAVA code in my sleep, equally it is clear that many of these guys have not suffered enough hair loss running IT project /Programmes of work and seeing what works and getting beaten up for what plainly does not.

I am not able or willing via this comment to educate or train such people. Try it and see how reality can make Agile and some other approaches I have come across recently become a concrete waistcoat if your job depends on delivering LARGE IT projects successfully. Agile worsens the risk /issue profile of any large IT software development. For example it ties large amounts of resource (including users) to an IT delivery right through the project life cycle due to it iterative development approach. Not good for the client but great for the job security of the staff.

Even some of the Agile God Fathers have mentioned the scalability issues associated with Agile. I have mentioned these in some of my previous posts. 

This does not mean that Agile cannot deliver large IT projects. It just means that even a blind man given enough ammuntion can hit a bull at 50 metres. He might even get three in a row its all about odds.

Only on Friday last week I visited an Agile based software house in Farringdon where the CEO was in a desperate state. He had a successful company (13 million turnover) and wonderful software products in a niche market, which made me, feel – if only I could be so lucky.

Then he told me with the HR person present that the 12 developers (including 2 Tech Architect Scrum Masters) who had built these products were killing his business and holding him to ransom. Low documentation meant they controlled the maintenance and enhancement work and they were abusive, threatening to him and the younger members of staff who wanted to try and get a look in and develop their own skills and abilities.

These free spirits as I like to call them, were not turning up at work, were disappearing after lunch, were unwilling to go abroad on client trips. Finally they were killing the business with wage demands well above market rate on an or else basis. Freedom meant control which was focused against the business shareholders and not in an attempt to help the company grow. How many times have I seen this recently !!

Every time I put these real stories forward, people come up with statements like it cannot be a true Agile organisation, it was dysfunctional before it tried Agile etc. Agile is not a method it’s a state of mind and clearly they have not made the connection.

The real world cuts straight through all this folks:-

Most IT departments and software houses are in poor health or we would not have 70% project failure rates each year. Some have just a cold and some have HIV.

If Agile culture and freedoms turn typical organisations into a further downward spiral it is plain to me that Agile is not a good medicine and we should stop using it.

If the objective of Agile is not to deliver real benefits to the organisations who adopt Agile then please don’t comment on this site. 

Its 1.30 am and I have just got to go to bed. Once again thanks to all the contributions you have made, some of which have been very interesting and have on occasions made me relook at things.

I will be attending an Agile discussion group next month and I will update you all on the results of this meeting as soon as I am able.

Big Thanks !!

Posted by Kevin Brady  on Tue 12th September 2006 at 12:39 AM | #

>> However, for those who believe they have the intellectual high ground on the Agile argument on any other for that matter because of a typo better read the list below which catalogues some of the worlds most well known bad spellers <<

That the weakest argument I’ve heard yet:  “They do it so why can’t I?” —who mentioned childish?

Sir, it’s not just the spelling. The spelling is a side effect of your apparently complete lack of knowledge about Agile methodologies—to the point that you think it’s an acronym or a trademark.  The same goes for Scrum. If you had misspelled something other than THE VERY TOPIC OF YOUR ARTICLE, I wouldn’t have noticed or mentioned it.

I’m a certified ScrumMaster, and I dislike it.  I can tell you exactly why Scrum sucks, because I know what I’m talking about.

You simply don’t know what you’re talking about.  Let me help you with your article:

  * BAD Project Managers or ScrumMasters (two completely different roles) turned themselves into Project Administrators…

  * DYSFUNCTIONAL project teams had in almost all cases been taken over by strong personalities leading to mini dictatorships…

  * Knowledge Monopolies are bad and DON’T EXIST by nature on agile projects (see “Bus Factor”)...

  * Resource Management had vanished which was clearly an issue with the organization, NOT the delivery methodology…

  * Having had a taste of freedom the dictators were a hateful and aggressive bunch when asked about their managers, SO WE FIRED THEM…

  * Most of the talented young development staff were leaving because SOMEONE FAILED TO DO SOMETHING ABOUT THE DICTATORS.

  * Each of these organisations had differing development approaches and tools from project to project because they failed to take 5 MINUTES to talk about STANDARDS and failed to take A FEW HOURS every couple months to do a CODE REVIEW…

  * MISTREATED clients were fed up with never-ending, continuous involvement in IT projects.

I wouldn’t want to be part of an incompetent team either.

So you worked with these organizations.  Were you part of the solution or the problem?

What did you do to help?

Posted by CB  on Tue 12th September 2006 at 03:29 AM | #

CB I thought so. Vested interest grin

Seems to me, like you need to take a chill pill!

It’s a shame, because your the sort of person that gives Agile a bad name and the ammunition Brady needs to continue his anti Agile deconstruct.

Posted by TomHayden  on Tue 12th September 2006 at 08:04 AM | #

Hi Kevin,

Farringdon London? I’m in London too. Where and when is this Agile meeting you mention? I would love to come along and debate with you.

Paul.

Posted by Paul Beckford  on Tue 12th September 2006 at 09:17 AM | #

Hi CB,

You obviously do not get it.

Knowledge monopolies in mission critical systems mean the people are unsackable. They know it and you know that they know this. The board of directors in such places are held to ransom. As I said I only saw one last week.

The clients were questioned by me and new nothing about how truly serious the problems were, because “spin” had taken care of that. However, they were unwilling to contribute staff for full life cycle work. They just thought it was crazy when alternatives offered shorter levels of resource tie up. They felt the developers were interested in the idea of Agile and not their specific needs /constraints. Remember the customer is always right ! 

As a Scrum Master your view of the project world is most likely running a six person team or less. Not a qualification for running at a stratgic level an entire IT department. The difference of putting a small crew together and delivering an entire order management system for say Asada is like chalk and cheese. Please read carefully what I write. I have no issue with Agile at the small project level. Worked for the best in the New Media Industry and know only to well how well talented people put together in small groups working on small projects with a high visual aspect to the deliverable can really make thing happen. My issue is with large projects !!!!!!!!!!!!!

Knowledge monopolies do exist in many organisations but Agile makes it worse. It gives scope for those in the Scrum to self organise for their own ends as well as the ends of the Scrum especially in an environment of low documentation. Its perfectly natural and I can’t blame them. We all have to get the best out of life and indispensability is a worthy objective if the culture and development approaches allow.

Fragmented and evolving use case models lead to massive resource tie up - frighteningly inefficient. If you don’t understand the typical resource profile of an Agile project DSDM etc then you just a green newbie fresh out of university without a clue how to make your employer profitable. Its obvious your in the developer scenario I mentioned in my earlier comment.

Ow by the way, last year had to work for one of the UK’s largest oil companies. They have also had enough of Agile and one of the key founding fathers. Still trying to get them out I understand due to knowledge monopolies and maintainability issues.

Posted by Kevin_Brady  on Tue 12th September 2006 at 12:09 PM | #

Concerning my discussion group this is being organised by a US University to help develop its aspiring MBAs in Project Management. If I could allow others to participate I would it would be kind of fun.

However, if I could I would need to think carefully about who could participate. I am interested in discussion and dialogue but not insulting sabre rattling from those who are nutty evangelists or some of the more extreme types of people detailed in an earlier comment.

This Agile thing is a real worry. The emotion and rudeness detailed in the comments section of this blog and the witch-hunt attitude of Agile believers is very similar to the attitudes I describe in the post and only goes to support what I am saying smile

Posted by Kevin_Brady  on Tue 12th September 2006 at 12:17 PM | #

Why do you think my rudeness is worse than your incompetence? 

Your elite attitude, thinking you’re right about something you can’t even spell, is testimate to your thousand pound ego.  You are no better than the dictators you describe.

Regardless…I’ve likely seen agile succeed as much as you’ve seen it fail, in companies at least as large as you’re discussing.

Enjoy your study group.  I’m sure it will be unbiased. 

I’ve wasted enough time here.

Posted by CB  on Tue 12th September 2006 at 04:42 PM | #

CB - your sounding a bit like a troll dude.

Posted by Dave  on Tue 12th September 2006 at 06:13 PM | #

My apologies for sounding like a troll and getting too personal.  But let’s clarify something:

  * Kevin is not just some guy who wrote a blog post about his feelings about Agile and Scrum.  He is a self proclaimed specialist in the area of IT Project and Programme management.

On this particular subject I feel he’s misrepresenting himself as an expert who can speak on the subject of Agile, or more specifically Scrum, with some degree of accuracy. 

I’ve called him on it.  No I wasn’t nice about it.  But really, why should I be?  This is damaging and ridiculous. 

I hope I’m doing him a favor.  What do you think he’s telling his clients?  What would the consequences be of making this mistake in front of them—rather than just ticking off a troll (i.e. me)?

In any case, to be more honest he really just needed to change the tone of his article to:  “Although I’m not an expert on the subject, it is my opinion that….and I could use help understanding why it seems to me that Agile and Scrum fail in these areas…” etc.

And yes, it would have helped if he could at least spell the very topics he was discussing before sharing his grand dissent.

Regardless.  I’m sorry for sounding like a troll.

Posted by CB  on Wed 13th September 2006 at 05:17 AM | #

Agile approaches embody a certain set of principles (i.e. Agile Manifesto). Either you ‘get’ these principles, or you do not. Many companies have found agile approaches to be very beneficial in creating momentum, increasing morale, delivering a higher quality product and a product that’s more useful to the customer (because of the customer’s involvement). If traditional practices work for you, Mr. Author, then by all means keep using them. There happen to exist successful implementations of agile approaches in many organizations around the world, and these companies sing the benefits of competitive advantage, innovation, etc. I suppose time will tell which is the better approach, and I place my bets on agile. I have to agree with one poster to this blog: I don’t believe you really ‘get’ the agile principles. It’s apparent by your one-dimensional observations. Or perhaps you observed a really badly implemented example. You should seek out any of the successful agile orgs (google it - you’ll find plenty) and offer to observe or visit a product demo. You’ll be surprised. You’ll also notice that these shops are eager to share stories. You’ll also notice that they have their fair share of war stories. It isn’t easy. I am saddened to hear that such a seasoned “IT expert” would allow such a narrow set of observations/experience to dictate his overall opinion. Explore, observe other companies, learn and don’t be so narrow-minded. The successes ARE out there.

Posted by ScrumMaster  on Thu 14th September 2006 at 07:00 AM | #

“Explore and observe” works both ways ScrumMaster.

Agilist’s, unfortunately seem to prefer one way traffic and simply denounce a critical account as a bad implementation.

Does this mean Agile is impermeable to criticism, because this seems to be the stock defence of every failed Agile project - of which there are increasing numbers.

If the implementation is wrong so often, then someone has to start asking the question whether your average joe is ever capable of delivering it. Are they ever actually capable of adhering so slavishly to the manifesto principles. Because that is an implication of this line of defence. 

Does this all amount to a case of burying heads in sand by the Agile community? I hope not, for their sake.

Posted by ChrisTremlett  on Thu 14th September 2006 at 08:24 AM | #

Yes, ‘explore and observe’ works both ways, which is why I was suggesting it in the first place.

I have seen both great implementations and horrid disasters when walking into agile adoption. It is difficult. I have seen a handful of “average Joes” make it happen successfully. More often, organizations need help. I guess this is where the consultants that you hate so much come into the picture. Like it or not, some of us are actually value-add and have been successful.

What I’d like for you to consider is just how many companies - all over the world - are seeking alternate ways to develop new products, because waterfall just ain’t cuttin’ the mustard. I’ve visited at least 25 companies over the last 5 months and that’s just little ol’ me. So maybe these companies explore Scrum, or XP, or the No_Shit Approach – whatever you want to call it, they’re looking for a better way. That is a fact. And it is also a fact that adaptation is critical to survival. As this global market picks up more and more momentum, we’d better do what we can to adapt and better yet, stay ahead of the curve. That’s another blog for another night, my friend.

Not all can be helped (or want to be helped). Maybe something else is more appropriate for these folks. Or maybe these companies will decide to stay stagnant, never change, and die off. And I’m fine with that. Not every business will make it, and the strong and adaptable will survive. Think Darwinism, mate.

You know, it sounds like you have a really bad taste from agile. Maybe it was a person who did this to you, or maybe it’s a result of your own inability to change. Many of us “agilists” are actually very approachable, open-minded people who are not afraid of criticism. I came from a ‘traditional’ world and I still hold on to many of the great skills I learned as a traditional project manager. I never fail to consider criticism. I debate the subject at least three times a week (and am here gladly debating it again on my own time). My head never goes down in the sand, and furthermore, I come from an honest place of wanting to help. Truth be told, I will always do this; the experiences I have had have truly changed my life. Maybe our paths will cross one day.

So here’s a challenge to you agile haters: Why don’t you invent something better? Be innovative!! Come up with what you think will work! I’d love to see what good can come out of this as a constructive discussion, instead of a finger-pointing blame game.

How can you add value? What’s your answer? I suppose time will tell, overall, which is the better way. But it’s a good, healthy debate in the meantime. =)

Posted by ScrumMaster  on Thu 14th September 2006 at 09:56 PM | #

I never said I hate Agile and incidentally, neither has Kevin in his articles. But, I can’t be bothered to justify that, as most will see what they want to see.

The polarisation of your language i.e “agile haters” etc etc is exactly where the problem lies. It’s not just you Scrum Master, it’s endemic throughout the web for Agile protagonists to use venom against anybody questioning Agile practices without properly taking on board that these problems may just be happening in the real world and that it may actually be because Agile is the problem. I recently noted someone admit that they were actually scared to admit their reservations regarding Agile. That is simply not right and is a small indication that all is not well.

Bland statements like “poor implementation” , “poor motivation” etc are repeated all over the net as the standard response to constructive criticism. Any good methodology, philosophy, hypothesis, etc etc should be subjected to constant and rigorous scrutiny. Those who really believe in Agile will engage, debate and take on board criticism - using it to improve what’s there. Like your good self scrum master.

Other’s who might be considered to be the militant element of the Agile movement are holding this evolution back with their ridiculous evangelical zeal. Just read some of the brainless comments on here and you’ll get a flavour of that.

Osmosis of successful implementation and word of mouth, will always work better than head banging - just google some of the debates going on and you’ll see how aggressive some Agile proponents become. It makes you wonder whether these people have lives!! 

I applaud your open mind scrum master and I hope you, I, and everybody else of a sensible nature continue to scrutinise and ask questions - from both perspectives.

Posted by ChrisTremlett  on Fri 15th September 2006 at 09:33 AM | #

I want to make the following clear!

- I have never stated at any point in my blog that I am an Agile expert. I am not sure that some of the people who have made comments are either. Lets look at some extracts from the huge number of commments made on the back of this post:- 

*“Agile is a state of mind rather than a method or approach”
*“Its not a rigid method”
*“Its not a methodology which can be followed”
*“No one claims state of mind alone will deliver any results”
*“Having the right state of mind can enhance the effectiveness of those practices”
*“Agile is a marketing term”
*“If we are hitting 70% project failure rate with waterfall, who says that we won’t hit that rate with agile?

None of this speaks to me in terms of offering real solutions to real problems and delivering tangible business benefits to organisation thinking of adopting Agile.

- I am not an Agile hater. I just make observations and I have reported on those and I will continue to do so.

- I have never gone into an organisation which claims to be Agile and rubbished what has been a huge investment for some of my clients. That would be unprofessional.

- I have never said Agile /XP /SCRUM do not work. I personally believe from my own first hand experience of seeing things go wrong, that some common threads seem to emerge out of each of the Agile project /programme failures I have observed. Many of these threads are particularly dramatic and excentuated by Agile /XP /Scrum methods /approaches /believes /states of mind ? Personally I have stated that I believe that my evidence shows that Agile is a significant risk when applied to large projects and programmes of work. IT does not mean that successes are impossible just the odds are lower. This is why I believe Agile is best applied to small projects for consistent results.

Even Mark Fowler says :-
“If you want to build a doghouse, you can just get some wood together and get a rough shape.  However, if you want to build a skyscraper, you can’t work that way – it’ll just collapse before you even get half way up.”

Alan S.Koch says :-
“All Agile methods seem best suited to small project teams, that is, teams that do not exceed 10-15 individuals. Although some methods do not explicitly place a limit on team size, their preference for face-to-face communication over written documentation places a practical limit on size”

Even the experts have their doubts about the scaleability of Agile. 

- I will be writing a book in the near future if I have a flat point in my consulting work detailing the Clarety Methodology which has served me so well over the years. Your find its not unique it just combines the best aspects of the many methods doing the rounds but it adheres to the fundamentals of how projects should be run. It does involve participative management and “bottom-up” staff involvement in projects. Industry wants and needs at the moment reliable delivery not 70% failure rates and that’s what I put all my time and effort into these days.

Worryingly I have a number of emails from people claiming that they are developers and architects working for one of the most well known Agile software houses who feel intimidated and frightened to speak out. That can’t be right. This whole thing is like a cult and nothing to do with software engineering. Its not much different funny enough to the observations I made in this post.

The final and most important point I want to make is that many of the comments posted on the back of this article only seem to corroborate some of the key OBSERVATIONS mentioned in my article concerning culture and attitude of staff involved in Agile projects. Many of you have done the Agile cause a huge disservice in filling my blog with vitriolic and personal comments. It just goes to show what this Agile state of mind is about and the collusive and pack like nature of the Agile cult:)

Furthermore, if my observations are so uncommon which they are NOT - looking at my order book at the moment. Why do I have 35 comments on this post if what I am saying is so unreal?  Surely of if Agile is the equivalent of “Newtons 4th law of motion” as one evangerlist blog called Agile - oops he made a mistake its a marketing terms smile why are people so concerned about bashing this post if its all so irrefutable.

As I have said in this post, I intended to make this post my last one on Agile for a while as this blog has more to say on other more important topics than Agile and states of mind etc. However, I find myself in the same position as the God Father Part 3 when he says"as soon as I am out I find myself being pulled back in”. The emails I have received from supporters this week have changed my perspective on Agile for the worse and thus I feel compeled to write another post towards the end of next week.

Its all very strange!!

Posted by Kevin Brady  on Sun 17th September 2006 at 12:07 AM | #

Hi Kevin,

I’ve read your response, and I can’t see anything in them beyon a re-stating of your IMO bigotted postion.

Agile IS NOT a Silver Bullet - and AFAIK no one ever claimed that it was.

The deeper issue is management culture - You have still to repond to my point about Taylorism versus Deming. Why?

Did it go over your head, or does looking at the root causes for organisational failure not fit in with your over simplistic take on the world?

Paul.

Posted by Paul Beckford  on Mon 18th September 2006 at 09:39 AM | #

I think this blog is great. We all have strong positions because of our successes and (painful) failures. We should continue to question the best ways to run projects. Kevin, I am not in a cult, just passionate about what I do. Sure, some of “us” are radical, and I believe this is in response to some ugly situations and project disasters that we’ve experienced. I welcome these forums to debate the issues, the pros and cons, what works, what doesn’t… and you’re right, your blog shouldn’t become a place for personal vetting. Agile is so counter-intuitive to the way that businesses currently operate; there’s no surprise that it’s difficult to grasp and extremely challenging to implement ‘correctly’. And when I make blanket statements such as ‘poor implementation’, there can be any number of reasons that this is the case: no support from leadership, no buy-in from middle management, no interest from the teams, etc., etc. Any combination of these reasons can exist. And may I point out that this will happen with any organizational change? Not just agile. I’ve seen wide-scale product and other process implementations (six sigma, pmi) turn for the worst because of poor organizational change management. People are not fond of change because it is unnatural, seems silly and is difficult.

Oh well, enough ranting - I’m sure you know all of this. One thing’s for sure: you definitely have me hooked on this blog! =)

Posted by ScrumMaster  on Mon 18th September 2006 at 01:31 PM | #

p.s. Paul, your question about Taylorism vs. Deming couldn’t be more appropriate. Taylorism infused our business culture with scientific management principles - that management in infinite wisdom could time the actions of subordinates to optimal efficiency.

Deming influenced many organizations, among them Toyota, with his plan-do-check-act cycle. (We all know the Toyota story.) He was innovative in his approach to business on so many levels. Just read his resume and accolades on the Deming site.

I love Deming’s point that speaks to creating an environment of self-improvement and removing barriers that rob people of joy in their work. On a very basic level, this is what I try to do every day with my agile teams. Find ways for them to be self-organizing, creative, happy, motivated. This isn’t easy in cultures who are more of the “Taylor” style.

Gotta go. Thanks, Paul, for reminding us of the bigger picture.

Posted by ScrumMaster  on Mon 18th September 2006 at 01:46 PM | #

Hi Kevin,

The legacy of Taylorism still lives with us. The most common emotion I experience in the workspace is fear. This is usually a consequence of an endemic lack of trust.

Change is not always for the good, so no wonder that people often fear change. For example change solely motivated by increasing earning for shareholders or small elite is not “good change” IMO.

The Japanese see change very differently:

http://en.wikipedia.org/wiki/Kaizen

For Kaizen (“Good Change”) to occur people need to be free from fear. For open and honest communication people need to be free from fear, for mutual respect…  You get the idea.

A country where the corporate culture has lead to things like the Enron scandal will struggle with the concept of Kaizen.

For Agile to work these types of issues need to be addressed. I have led successful agile teams that have succeeded, but only after spending considerable effort building up trust within the team and with stakeholders. Given this backdrop I’m not surprised that many agile projects fail - but I wouldn’t jump to the conclusion that the failure is down to “Agile”.

Paul

Posted by Paul Beckford  on Mon 18th September 2006 at 03:48 PM | #

Hi Folks,

I want to apologise to everyone who has contributed to this discussion that I can’t answer each person’s queries individually. I have a living to earn along with my partners which consists of turning around sick IT departments /projects and programmes of work.

I notice that the aggressive /vitriolic behaviour mentioned in my last comment continues. I find Paul Beckford remarks which describe me as a bigot as offensive and unprofessional.

As I mentioned in my last post such behaviour only goes to demonstrate that perhaps some of the attitudes I pointed out in my original post are in fact quite common among Agile defenders /believers. Your doing your cause a disservice by such behaviour.

You do not know me and I do not know you. This blog is a place for open debate and where I write what I think about the IT industry. Nothing in this blog indicates an unwillingness to discuss concepts or ideas. All comments are retained and none are removed or edited, so I can hardly be a bigot. In fact if anyone is a bigot its got to be people like yourself and some of your rude friends who get abusive when they fail to get consensus belief in Agile.

It might surprise you that Agile sceptism is a growing issue on the internet. Stay tunned for a post on this next week.

Please read my previous comment and you will see that the issue I have with Agile is it’s not a scalable delivery option able to deliver consistently realisable benefits to organisations. I have seen Agile work in the new media industry frequently but only with respect to small visual based projects.

This opinion is based on 20 years of experience some of it derived from working closely with one of the founding fathers of Agile. As I have stated all over this blog, I pick development methods, and try to modify organisational cultures in order to make IT project delivery success a repeatable proposition along with reduced IT costs. As a by product of this it might surprise you that making project staff happy and professionally fulfilled is part of this solution. If you look at my Time Boxing post

Time Boxing Part 1


you will see that I run project and programme teams with a participative management style with ownership and bottom-up planning of projects /programmes. It is infact a cornerstone of my approach to the running IT projects.

Paul you also make the point that Agile is not a Silver Bullet. All I can say is that we live in a stakeholder society and organisations /companies are not going to change IT development working practices if the risks taken and the costs associated with the change don’t give shareholders an adequate return on capital employed. This return has got to be over and above that provided by existing methods /approaches or its going to be a non-starter.

As Churchill said “The capitalist system is a bloody awful system but it’s the best we have”.

Changing society comes a poor second to profits. “Sad but true!”

If your day job is a consultant you wouldn’t be able to sell Agile if you can’t demonstrate definable business (stakeholder returns) and efficiency benefits.

I am conscious that your comments concerning Taylorism Vs Deming sail dangerously close to the opening up of a political debate which I am not willing to participate in.

This blog is not a forum for the discussion of political ideas unless they are DIRECTLY related to the delivery of IT related projects.

Being a qualified economist (don’t think this is over my head Paul smile) I am very familiar with the Taylorism Vs Deming argument. I personally try to bend towards the Deming side of things up to the point where the lack of controls /constraints actually equals CHAOS. For example having worked in the new media industry, fun at work is a real motivator.

The degree, with which I move along the continuum running from extreme Taylorism towards an extreme Deming approach, depends on whether you are applying it to a line management or project management situation and if the later small projects vs large projects.  I will be explaining this further in a post during the next week.

Posted by Kevin_Brady  on Sat 23rd September 2006 at 12:41 AM | #

Hi Kevin,

You seem to take offence very easily. I did not say that you was a bigot, I said that I felt that your position was bigoted. Blaming the ills of corporate IT on the recent use of agile ideas by some, IMO is a narrow (bigoted) point of view.

The patient has been sick for a very long time. Any analysis must look at the root cause of the illness IMO, and not focus on a convenient scope goat.

You seem to want to make this personal. It is not, it is just about ideas. You seem to want to make it political. It is not. Deming believed in Capitalism just as much as Winston Churchill. What I am talking about is how to effectively apply capital, and the values and the principles that are needed to do so.

Historically many of the most successful capitalists have held enlightened values and principles (think of the Quaker entrepreneurs of the 18th and 19th centuries). Capitalism and enlightened corporate values are not at odds. I would argue the opposite.

I look forward to your detailed discussion on how the ideas of Deming can lead to Chaos. Incidentally, talking about chaos you chose not to address my point about corporate governance specifically.

Interestingly HP is recently in the news with regards to the same issue. Adding their name alongside Enron, WorldCom and others when it comes to corporate scandal.

Like I said the patient has been ill for a very long time, and we need to look at the root causes.

Paul.

Posted by Paul Beckford  on Sat 23rd September 2006 at 08:27 AM | #

Hi Paul,

For what it is worth I agree -
“Change is not always for the good, so no wonder that people often fear change.”

I don’t agree that change - “Solely motivated by increasing earning for shareholders or small elite is not “good change” IMO.”

This is naive. At the end of the day the world Economy is now a stakeholder society where money and investment chases organisations which produce things which people need at prices and quality at least comparable or better than the next best alternative.

Forget this and no one has a job. The west is in my view forgetting this important point as i write this post. It is clear that while we are concentrating on shorter working weeks and heavy employment legislation, European markets are being taken away from us by places like China where profits are everything and they don’t care much about green issues, or peoples working conditions. We live in a competitive world which means all companies have to concentrate on the bottom line. If the bottom line was not so important I would be out of business myself.

Only recently I travelled to France to talk to some students and I was told that in France there was so few jobs for young people that its best to stay on in education as long as you can. That’s how serious things have become.

I agree with this up to a point - For Kaizen (“Good Change”) to occur people need to be free from fear. For open and honest communication people need to be free from fear, for mutual respect…

I believe that this can be achieved whilst pursuing the wishes of stakeholders. However, some companies just don’t get it right and subsequently create work for people like me.

Only recently I told a CEO of a Software House in Surrey whilst supping some Ale in the local pub that “his development staff owned his company not just the stakeholders”.  I pointed out that the products he had and their future development in the absence of detailed system documentation depended on them. To satisfy his stakeholders (banks, venture capitalists) he needed to treat them properly and possible make them stakeholders as well with share option schemes etc. The later statement was not well received and I was not offered another pint. I predict that within 12 months he will be calling me in to sort out some serious cultural issues effecting delivery of the next generation of software products which have made them so much money in the past.

The pursuance of stakeholder returns is easier to achieve with happy motivated staff than without. I will have much to say abut this in future posts.

Posted by kevin Brady  on Sat 23rd September 2006 at 01:59 PM | #

Hi Kevin,
Reading your last post, I see that there is much on which we can agree. The success of the Japanese since World War II is not down to chasing “the bottom line”, but down to the common drive to see Japan succeed, meeting the needs and the dreams of everyone involved, not just a small elite.

Before Consulting I was a Manager for about six years, so I’m trained in Japanese inspired Management appraches like “Total Quality Management” and “Six Sigma”. Yet throughout my experience, I’ve witnessed that such ideas find it very difficult to find traction in both the US and UK.

As we move into the information age, de-skilling and de-powering workers by making them interchangeable and dispensable in the way pioneered by Henry Ford in the 1920’s is no longer a feasible approach.

Modern high tech economies are powered by creativity, innovation and rapid change. To achieve this, the workers need to be empowered and enabled to utilise their talents to the full.

Places like China and Korea are currently mopping up low skilled work exported from Japan and the West, by providing cheap labour. Unless we intend to radically reduce our standard of living, a low skilled Taylorist approach will not work in developed economies.

Software is inherently complex, and requires a great degree of skill and ingenuity to do well. De-skilling by turning it into a production line (waterfall) process and applying centralised control (big upfront MS Project Plan) in a Taylorist fashion is just not the best fit for Software Development.

To produce good software you require small empowered and motivated teams working autonomously yet in concert. The focus needs to be on results and the teams should be judged on what they produce. Process should be left to the teams themselves, who willingly take responsibility for continuingly improving there own performance.

If you where to spell this out to the average trade unionist or Manager in the West you would get raised eyebrows, yet in Japan at companies like Toyota and Nissan, such an approach is the reality and totally accepted as the norm.

Agile practices without Agile values and principles will not work. I have noticed that cultural change is easier to attain bottom up. With Agile teams performing and delivering for themselves. In time it is my belief that these values will begin to permeate up organisations into the boardroom as well.

I believe in attacking Agile that you are miss-guided. I would agree with you that “Agile” as currently promoted is not much more then a Marketing mantra. But in 16 years as a Software Engineer the Agile movement is the first to address the underlying causes of project failure, which IMO is mostly due to the Management culture prevalent in most Software Organisations. Companies that make the cultural transition will succeed, along with start-ups where the Directors “get it”. There are many out there that “do not get it”, and as experienced practitioners we owe it to them to help them see through the smoke screen of marketing and hype into the core values and principles that lay behind the Agile movement.

IMO the marketing of Agile is/was necessary as we live in a society were people need to be sold to. But successful marketing is not sufficient, what is needed also is a deep understanding followed by cultural change. Without the later Agile is likely to pass just like so many other Management fads over the last few decades, such as TQM and Six Sigma.

Paul

Posted by Paul Beckford  on Sat 23rd September 2006 at 05:11 PM | #

Hi Paul,

It’s late so I will make it quick. Thanks for the dialogue. I would agree that we are closer than at first you might think. I have always known this but posts are a frustrating slow drip drip way of exposing the nuances of my approach to successful delivery, which I am desperately trying to catalogue in an e book, which will be sold on the back of this blog when ready.

However, it is worth stating that we have some fundamental differences concerning Agile. Some of those will be detailed in my two-part Taylor Vs Deming post coming in the next week and others going forward as this blog gradually starts to explore some of the development methods adopted by Agile.

Furthermore, Agile is not just a marketing term. Just a short visit to the bookshop or to some of the numerous Agile consulting websites and it is clear that Agile incorporates some well-known methods such as XP, Scrum, FDD, DSDM, RUP and has real substance.

All of these are small project approaches (with the exception of FDD, which does scale well if used correctly as explained earlier on in this dialogue). Even some of the Agile God Fathers have their doubts (please see earlier comments) about the Agile’s scalability.

I really favour FDD, which I have adapted into an enterprise ready approach, which has repeatedly delivered real results and as such helped develop my consultancy into a successful business.

Have a good weekend Paul

Posted by Kevin Brady  on Sun 24th September 2006 at 01:34 AM | #

Hi Kevin,

This is the problem with Labels. “Agile” is a label for a set of ideas shared by a broad church of specific things. Each individual thing is different and unique (XP, DSDM, Scrum, Crystal Clear, Crystal Orange, Lean Software Development, FDD, etc).

The “label” is not the “thing”. The only definitive definition of Agile that I know of doesn’t define any specific practices; it only mentions values and principles. Values and principles have been the focus of my posts so far.

So I agree, posting is a very low bandwidth way to debate, but we need to be clear that we are talking about the same thing.

IMO The branded collections of practices mentioned above (XP, Scrum, etc.) are not Agile, they are a branded collection of practices. Agile is a common collection of values and principles shared by a group of software practitioners and described in the Agile Manifesto.

http://www.agilemanifesto.org/

If you are using a different definition for the Label “Agile”, then I suggest that you clearly state it so that we can be sure that we are debating the same thing.

Using this definition, your approach to Software Development is as likely to be “Agile” as anything else :^).

Paul.

Posted by Paul Beckford  on Sun 24th September 2006 at 08:25 AM | #

Hi Kevin,

I’m fine with this blog and respect the views of all who have participated. I just want to say that I resent some of your blanket statements about “agile defenders/believers.”

It’s important to remember that Agile and traditional folks come from the same place. Sure, there exists skepticism on the internet about agile, and plenty about traditional approaches as well. Skepticism at least means that people are giving it some thought.

I met a guy like you in a class I taught last week. (Consultant, PM, traditional). After some planning exercises and detailed discussions on what works for scaling, he mentioned that there might be something to this stuff, although he wasn’t ready to try it yet. It was through thoughtful and respectful conversation that he could say, “I see your point.” And I was thrilled that he gave it a chance and remained open-minded.

I saw his point on many issues, and did my best to explain the differences in the fundamental beliefs. We agreed that good managers/leaders have “agile skills” regardless of what name you call it. smile

It was a nice discussion with mutual respect and without any blame, finger-pointing, name-calling, bigotry or prejudice.

Posted by ScrumMaster  on Sun 24th September 2006 at 11:05 PM | #

Hi All,

Just to broaden out the debate. I came across this news article:

http://www.infoq.com/news/dangers-agile-adoption

It mentions many of the points reqarding values and organisational culture that I’ve posted about. I know Simon Baker and I am currently working in the same company he talks about, and he is right, the management here are amongst the many that “just don’t get it”.

It should be obvious, but it’s worth stating: Agile is not a nautral fit everywhere! And as pointed out in this article it is no Silver Bullet either.

People solve problems, not off the shelf methodologies.

Paul.

Posted by Paul Beckford  on Tue 26th September 2006 at 03:56 PM | #

Kevin you make some great points here.

Posted by Tim  on Wed 27th September 2006 at 06:24 AM | #

I’ve read this post and the one mentioned above and started to wonder if there isn’t several kinds of people and that one strategy will always fail for some?

Example: Creative people simply cannot produce great results in a restricted environment while nitpickers thrive. In a war-like situation, nobody would propose creative thinking (“let’s stop shooting and start planting some flowers for a change”).

When I’m teaching people Agile methodologies, I insist that they first take their current situation into account, make an assessment and choose those parts of whatever Agile methology that seems to be “en vogue” right now which *helps* them solve one or more of their problems.

But at the end of the day, if the decision makers ignore the itches, who can blame anything but the members of the project for failure?

So usually, I start to introduce people to test-first (or TDD or whatever). The idea is that this will help to drive out bugs of the software, help to document what is really going on and get the specs nailed down (if you can’t write a test for a spec, you *know* that you must do something).

When this works, people start to accept and implement other ideas. Some work, some don’t. Some would be great, everyone agrees, but alas not in this company. Some are so great that the company is forced to adapt, though.

And to come back to one of your criticisms: If there is a person who hogs the knowledge, get the youngsters to add tests to the product. When you have enough tests (ie. when you feel confident), just fire the mini dictator. Sure, you’ll have a rought time ahead of you until you’re up to speed again but a) your product will be well tested by then, b) hogging knowledge won’t mean anything anymore, c) new features will be much easier and faster to implement.

So in the end, it’s not a bad thing that these people exist. It just means that a decision maker has to do his/her homework and make a decision. It’s not that nothing can be done.

Therefore, I think the main point is not whether agile is better or worse than other methodologies but that in agile, you can choose which parts you want to adopt and which not. In contrast, other methods for software development are much more all-or-nothing, or not agile/flexible.

Posted by Aaron Digulla  on Thu 28th September 2006 at 12:25 PM | #

> This Agile thing is a real worry.
> The emotion and rudeness detailed
> in the comments section of this
> blog and the witch-hunt attitude
> of Agile believers is very similar
> to the attitudes I describe in the
> post and only goes to
> support what I am saying

Kevin—

Interesting blog post. What is it about agile that makes people froth at the mouth about it? I personally have been tainted as an agile zealot, although I don’t paint myself with the same bush.

The witch hunt is going both ways. I find your response (quoted here) especially odious, since I’ve seen the same emotion and rudeness from the agile opponents. “This Agile thing is a real worry,” indeed. “This pseudo-intellectual rant against incompletely understood things is also a real worry.” Everyone who tries to correct misperceptions about agile is told that “they just don’t get it.”

Rants against agile, like this one, or blind patriotism to agile, aren’t helping anyone. “My process is bigger than your process.”

All I know is that it’s important for modern organizations to find a way to sustain delivery of constantly changing software over time. Agile starts to give me answers; more specifically, agile development techniques such as TDD start to offer real solutions.

I’m not so dumb, and nor are most software developers, to subscribe to the belief that agile will solve all my problems. I’m also quite aware of the fact that agile can present as many or more problems than it can solve. Yet for all that, I’d promote it over all the other BS methodologies I’ve worked with over almost 25 years of software development. Does my success with agile make me a zealot?

-J-

Posted by Jeff L.  on Tue 3rd October 2006 at 08:48 PM | #

“Everyone who tries to correct misperceptions about agile is told that “they don’t get it”.”

Wow, great piece of irony there Jeff!!!!

Posted by Jiffy  on Tue 3rd October 2006 at 09:34 PM | #

Kevin posts, earlier, “You obviously do not get it” to the frothing CB. His “get it” refers to the bigger “it,” not the specific agile “it.”

It’s a nice, arrogant dismissive phrase.

-J-

Posted by Jeff L.  on Tue 3rd October 2006 at 09:59 PM | #

Hey Kevin this is hilarious. grin

Yahoo Scrum Development

Isn’t Paul Beckford the comment spammer guy?

Posted by Chris  on Tue 3rd October 2006 at 10:00 PM | #

Thanks Jeff,

With all due respect, you don’t seem to get “it” in as far as the irony of your assertion.

Posted by Jiffy  on Tue 3rd October 2006 at 10:21 PM | #

Commenting is not available in this section entry.