(this article was first published in 2015 in LinkedIn)
Mission: Interviewing people from different organisations to find out the reasons behind the prejudice against the agile practices.
On March 3, 2010, the Federal Bureau of Investigation killed its biggest and most ambitious modernization project — the one that was supposed to prevent another 9/11 but that had devolved into one of the biggest software debacles of all time. For more than a decade the FBI had been trying to update its computer system, and it looked as if they would fail. Again. And now it was his baby.
In the last few months, I've made interviews with people from large enterprises about their experience using agile practices in their projects. With the ones who had a bad experience, I dived into a deeper conversation to learn more about it. There were 2 common things they shared in their complaints
“We are too corporate to work agile”
or
“We have business critical applications, it is risky for us”.
I knew neither of these could be the reasons for failure but their projects did really fail and they got back to waterfall, I had to find out why.
Is your company the most corporate enterprise in the world?
First of all I had to convince these people that their claim about being corporate or having business critical projects are not the real causes. Let’s take the corporate structures first. Bank of America, JP Morgan and Goldman Sachs are only a few examples from the corporate world that use agile practices for software development projects. They wrote the books of being corporate, with their employees wearing suits even if they are working as server admins in the data center (this article was first published in 2015, since then dress code might have changed). If they prefer agile practices and had success, then being a large enterprise with complicated corporate structure can not be the reason for failure.
Do you think your mobile application is more critical than FBI or Department of Defense projects?
Let’s discuss the business critical applications. Can you not use agile practices for business critical applications? Absolutely you can. US Department of Defense started using an agile framework in 2014 and they are pretty good at it. They say they are twice as productive compared to how they operated before. It has been more than 5 years since their agile transformation kick-off and they sticked with it which means their defense projects didn't fail because of agile practices. My wording here was intentional, some of their projects might have failed but the root cause wasn't about agile practices. That's why they sticked with the framework. In fact, agile practices helped them to see the root causes and improve.
FBI is yet another example of having business critical operations. The software they use affects lives, and I mean life and death, not lifestyle. It is not business critical, but it is life critical! They failed in their “large scale” computer systems upgrade programme for a long time trying the same ways of working over and over again. Then, they achieved success with the help of agile practices, why can’t you?
So what are the reasons behind the failures?
Unhappy stakeholders, frustrated teams, failure in delivering results, dispute between departments… a total corporate nightmare. And this all happened after the agile transformation. So, if the root cause is not what they think it is, then what is it?
SCRUM is light-weight but not zero-weight
The first thing I realized that they have been working with consultants who didn't have enough practical experience from different industries, different environments, different agile frameworks or methodologies. Just to clarify, I have interviewed 8 organisations, 3 of them failed with their agile transformation and all 8 used SCRUM. However, I am sure that if I had the time and resources to interview more people, I would have heard different frameworks or methodologies that failed because of similar reasons. SCRUM is light-weight which makes practical experience matter even more compared to other framework or methodologies. Inexperience brings the lack of self confidence which results in panic in times of trouble. The consultants they have been working with started mixing the corporate processes of the company with SCRUM artefacts and events. The reasoning they made was "SCRUM is flexible and light-weight". However they were missing the fact that SCRUM is not zero-weight. There are things that you HAVE TO do in order to make SCRUM work. If you don't start making changes in daily routines of people, you can't change existing habits. For example, if you need continuous contribution from someone, this person has to be in your SCRUM team and has to attend the ceremonies. The mistake they have made was bringing people in and out every sprint depending on the sprint plan that the PO has prepared (wait a minute, PO doing planning alone? bringing people in and out?). When I got in touch with the consultant about why he worked that way, his answer was “they told me that people had to work in different projects and they can't be a part of a static team”. This is not something peculiar and most of the enterprises trying to transition to agile using a bottom up approach suffers from similar limitations imposed by the organisation's existing dynamics. This is why practical experience is important for SCRUM implementation. This experience will teach you when to be flexible and where to be flexible and what you can sacrifice when you adjust practices to the organisation rather than the other way around. Yes you can have someone working for different teams and if it is a software product team , this person is usually a UI developer, designer or a tester. However, more than 2 is a physical limitation because that person has to attend the daily stand-ups, plannings,reviews, refinement sessions and sometimes brainstorming, visioning sessions… So, why did the consultant offer bringing people in and out as a solution? Because he did not face this problem before and the safest solution to provide was to get along with company’s existing ways of working.
You need sponsorship from senior leadership and also buy-in from team
Another root cause I've found out from the interviews lied within the approach in landing the transformation. Some organisations used a bottom up approach but lacked senior sponsorship from C-level leaders and some others used a top down approach with strong leadership support but no buy-in from the teams. Both bottom-up and top-down methods are OK as long as you know how to execute each of them. If you don't get the buy in from the teams, you might end up having fake agility where you do the procedure but not aligning with values and principles. If you don't get the leadership support, you won't be able to change siloes, most importantly change the behavious of leaders and managers in the organisation.
Agile as a transformation requires people, technology and process
The last root cause I've seen in some of these organisations was not tackling this change as a transformation. Almost all transformation programmes require changes in people, technology and processes. If you underestimate Agile Transformation and not dive deep into all these 3 pillars, you will leave a lot of blockers and impediments that teams can not overcome themselves without a wider support. Don't get me wrong, you shouldn't plan all of these pillars ahead and execute with a big bang change. However, through out your transformation you need to make changes required in people, process and technology and not only focus on the process.
Agile practices are easy to learn, difficult to implement
I remember the first time I was introduced to Agile practices with SCRUM framework. I learned it from an expert who worked in UK for several years and relocated to Turkey to support software development teams like ours. After implementing SCRUM for a month with him, I considered myself as an expert because it was so easy to learn the rules. However, the more companies I practiced agile ways of working, the more problems I faced, I realized that it takes years, different industries and different organizations to have an expertise.
Take away for enterprises who failed in agile transformation, your answer lies in agile itself; retrospective and inspection.
I had a very similar problem with one of my clients. They were using Nexus framework and happy with it but had minor problems within their processes. They did not blame and leave agile practices for that but tried to fix it. What we did was exactly what agile suggested, incremental delivery in fixing the things going wrong by looking at the process, technology and people as a whole instead of moving to another framework and planning a big bang change. For months, continuously, we checked what went wrong, found out the reasons behind and made small corrections. Targeting one problem at a time, for example; the lack of communication with designers… You can read more about how we tackled these problems at this organisation in this blog post. After a few months everything was going a lot smoother, but of course not perfect. The take away for me was; you need
continuous inspection and retrospective within your organization not only on a team level but also on an organisational level
because the market is changing, your organization is changing and most importantly your people are changing through out time.
Twitter: @can_huzmeli
LinkedIn: https://www.linkedin.com/in/canhuzmeli
References
Scrum: The Art of Doing Twice the Work in Half the Time