Avoiding the cliff edge: Why digital transformation should be agile

Skip to

Part 2: Conventional wisdom

Part 3: The new way

Part 4: Getting started

Part 5: Making it work - the hurdles to expect

Part 6: Why it works

Part 7: Conclusion - freeing the white elephants

Digital services aren’t just important to a business or a service. In today’s world, they’re crucial. Two decades ago, the website for a business, local authority or university was simple. The most complex feature would likely be a field to sign up for a database or mailing list.

Now, the website itself is a complex system, one of the primary tools for lead generation and carefully taking care of people down the conversion funnel. Or, for civil uses, efficiently guiding them to the service they need, informing them and allowing them to manage their services. They capture important data, monitor interest and interlink with other systems.

Of course, an organisation's digital presence no longer stops at the website. Apps — both native, where a user downloads them, and web-based — should integrate seamlessly with the main website. Social media is ever-crucial, and online directories need to be consistently updated.

If you front-load all of the research, planning, designing and building as part of refreshing this huge online presence, with the plan of launching it all at once, you’re going to encounter problems.

Naturally, every project does encounter problems. When you’ve meticulously planned and budgeted for everything, only for something to throw a spanner in the works, adapting to it can be slow, costly or just impossible.

This is the traditional way in which these programmes are managed, because it’s simply the method with which both public and private institutions are familiar. For many, the pitfalls are largely accepted as a necessary part of the process. But do they have to be?

Instead, can adopting an incremental method be better? Developing and changing one small aspect at a time, doing away with the big launch and opting for a process of gradual evolution.

In this guide, we’ll examine how you can do just that. Digital professionals will explain how their past experiences on projects using traditional methods have influenced their thinking of how the programme should be managed. Psychologists will explain why it is that humans struggle to cope with big changes — both in the workplace and in the services they use.

We’ll also explain how you can make an incremental change system work for use, using agile methodology, and show how it has helped the organisations we have worked with. In addition, we’ll also introduce our new system to make the process even simpler and more versatile.

Get ready to change.

The established ‘waterfall’ method of driving a project of digital change, with single, big releases, is something to which many businesses are accustomed. So much so that it can be a challenging habit to break.

So what are the problems and pitfalls of this system, if it’s so widely used, that have prompted others to find new ways? Here, we look at the challenges of conventional practices, and why they occur.

Higher education: habits and vanity projects

Rob Turner, scrum master at Zengenti, previously worked on multiple digital transformation projects for higher education providers over a near 20 year period, including the cyclical revamping of entire websites. These projects are typically renewed every five to seven years, and took around 18 months — on paper, at least.

According to Rob, the higher education sector’s reliance on waterfall project management methods was unshakeable, even though the model doesn’t lend itself to technology-based work. This is because, for many years, it has been effective for various fields to plan everything upfront, and that practice has been carried forward and applied to modern projects.

He said: “The trouble is that such an approach rarely goes to plan because — in any sector — external factors aren’t anticipated and contingency isn’t added in. So, whenever an issue appears that doesn’t fit in with that plan, the project is easily derailed. This situation is apparent to anybody who has watched the TV show Grand Designs: however well-planned the project is, it will always miss the deadline, often with expensive consequences.”

This can be exacerbated by the fact that there is often very little distinction between the website presentation and the actual CMS — it’s often the first thing to go when there is a shift in management. When the site doesn’t look or function how the team expects it to, the CMS is assumed to be bad and put to tender.

The capability of the CMS, its technologies and potential are often thrown out, along with an ageing or underperforming website, because they are seen as inseparable.

What has usually happened is that the initial push in resources to get a great new website subsides. Without the proper long-term investment in resources, a site slips into simply being maintained until it starts to stagnate and the whole cycle repeats.

This can have a heavy toll on users — creating or adding to a general resistance, or fear of change by teams. This can be either the fear of the unknown or simply from remembering the process’s challenges, having been through it before. It therefore becomes difficult to bring people on board.

Lack of flexibility and adaptability

Rob added: “Through lack of proper planning and understanding, stakeholder feedback comes in the form of ‘you make it, and I’ll tell you what’s wrong with it’. So, although an unsuitable approach, the end product is generally begrudgingly accepted.

“The arduous nature of the project means that, by the end, the team’s opinions on it — and their employer — cover the whole spectrum, depending on their individual experience. From ‘not as frightening as I first thought’, through ‘that’s not how we used to do it’, to people who leave their positions because they couldn’t fit with what was being asked of them.

“As a result of so many compromises throughout a long process, and its lack of adaptability, the user experience was often already dated by the time the updated site was released, in my experience. Even though websites in the higher education sector largely follow a standardised style, which somewhat masks outmoded features, the constant technological advances around the software were never able to be incorporated during the build.”

Sudden changes without clear reasoning can frustrate end users — a psychologist’s view

Dr Maria Karanika-Murray, an associate professor in occupational health psychology at Nottingham Trent University, explains that end users and customers often dislike sudden changes to services’ UX when it's too disruptive or they can’t see its relevance to them.

She said: “Humans do not like change that is unexpected. Unwelcome change can be a nuisance and disturb habits, demand adjustment to our daily lives, and require more time and effort to learn how to do things in a different way. A lot will depend on the user experience and how change is presented and what expectations are formed.

“Although there has been too little research to be able to say which aspects of change to a service people struggle to adapt to most, we do know that people find it difficult to deal with change if they do not feel capable, are not motivated, and/or they do not see value in it. A lot of it is about how change is presented, so they see the need and the relevance to them. If the change solves existing problems people may experience with the web service/app, then it will be welcome.”

The alternative to planning one large change, which takes time and involves so much risk — both to the team’s working relationships and outlook and the organisation’s benefits — is instead to swap the waterfall model for an agile one.

This allows you to focus on renewing and launching individual components one-by-one. In place of a journey lasting several months, with nothing visible to the end user until the very end, an individual item can be planned, designed, built, tested and released separately. Each of these increments can take just a matter of days, instead.

For the customer

There are several reasons for doing this, but the most important is to avoid knee-jerk reactions and steadily adapt to improve the customer experience.

If, for example, you want to send an email to a customer when they sign up for a new service, your first increment might be that a support representative sends them an email when they receive a sign-up request.

This approach might cost people time but, if you only have one person sign up a week, it will have achieved the objective of getting important information to a customer, probably without any tangible cost. That customer has also benefited from the email being sent straight away.

Even if your organisation needs to send 100 such emails per week, this scenario of manually sending emails means your team will quickly learn what is involved and even perhaps improve the email due to customer feedback.

The second iteration, or increment, might be to simply set up a Gmail automated reply in response to an incoming email. Again this could be done with little to no cost and may entirely achieve the objective.

Compare that to what might happen with a non-iterative approach. It’s entirely possible that we’d decide upfront that we need an entirely new system that will have 200 or more features. That would require extensive integration, training and costs until, after three months, we finally get our first email out.

After all that, the email itself might be wrong too. It wouldn’t have been iterated and refined, because we spent all our time and effort on the system itself and forgot our original simple intent: just to get an important email to a customer.

Blocks build a flexible agile working setup

Some systems are specifically designed to help teams use this method to maximum effect, as is the case with Contensis. It’s entirely developed for agile working, so that you can change content types, run branches for elements and move content between systems to test and iterate different aspects of your website and content.

A new feature to the system gives non-technical members of your team the control to release changes to parts or all of their website incrementally. Called Blocks, it’s a service of self-contained packets of code that run individual parts of a website or application.

Contensis CEO Richard Chivers explained: “Because they function almost independently, it’s possible to make changes to them without affecting the rest of the website. And, because Blocks integrates completely with Contensis, our content management system, those components can be reviewed, developed and tweaked endlessly.

“The purpose is to allow clients to use a single platform to deliver their agile development or content delivery, rather than integrating a headless cms with a hosting platform, a developing team and source control."

“It essentially gives any customer the ability to iterate and develop their website and applications in the same way as companies like Spotify and Netflix do, without needing a big team and seven or eight different key products.”

Zengenti scrum master Rob Turner’s advice on how to help teams envision the project and work in an agile method.

The product owner is key

The product owner, as the one client member of the scrum team, is crucial to the project’s success. They are important in unlocking the client’s agile mindset.

For a team that isn’t used to working in an agile way, laying out a few important points at the start of the project will help to focus the client, and allow the team to hit the ground running.

• Nominating and explaining the role of the product owner is a crucial step. As the one client member of the scrum team, they are pivotal to the project’s success. It is therefore vital to explain fully the requirements of their availability and expertise. They should also be fully aware that there can only be one of them voicing decisions — even if they need to run it past other internal stakeholders first.
• Be clear on how many days of budget the team has for the project work.
• Ensure everyone is aware of how days are logged. For example, three developers working on a five-day sprint equates to 15 days combined.
• Decide on a Minimum viable product (MVP), and how it can be iterated upon later in the project — or in a later one — if budget is available.

In essence, as a scrum master, you are trying to make a leader out of the product owner and support them. The key role of the scrum master is to encourage the product owner to be a leader. They can then best make decisions, create tasks, and prioritise them in order to achieve the project goal. If this happens, then the rest of the client team behind them will usually be supportive.

Despite this, there are times when the client team simply doesn’t take to agile. This often happens when they are used to working with waterfall frameworks, or there are internal conflicts regarding the end product.

Getting a person out of the waterfall mentality can be a challenge when they are so used to working in other ways, but can be done with regular encouragement and reminders of the process from the scrum team, and reinforcement of the benefits when they occur. It’s important to emphasise to the product owner that it’s their budget that’s being used, and that it’s therefore a financial consideration to stay on schedule.

Dr Maria Karanika-Murray, an associate professor in occupational health psychology at Nottingham Trent University, explains why incremental change is more palatable and manageable for teams, and why it’s crucial to get buy-in from employees if the project is to succeed.

Why staff resist dramatic change

We are creatures of habit, as they say. We like stability around us. We like our world to be predictable. Change challenges all that, even the change that we seek.

This is why establishing lasting change when it comes to behaviours can be difficult. It is not about our own behaviour but also about our environment, which may need to be adjusted to support change.

The mental barriers that people have about change, or create in reaction to it, can be internal or external. They can include worries about:

Why behaviour of the group matters

Social norms have a crucial role in people's acceptance of change, so the views and actions of others, from colleagues to mentors, to managers, are essential. The actions of senior leaders, who are in positions to set an example, can show how they endorse the changes, or indicate what is important and encouraged.

We should not forget how important values and behaviours can be communicated via nonverbal cues like policies and practices, and the culture that people who work in the same workplace share.

How to lessen the impact

Gradual change is essential. Small gains can help to boost motivation, help people prepare, change attitudes easily, make change seem more accessible and more fun, help change social norms, and are better for changing habits in the longer term.

They are also crucial for those who implement the change. It means they’re better able to gauge people’s reactions, especially those who are more resistant to it, and see what elements in the broader environment may need to be altered. Implementation plans can then be adjusted to make the overall change more sustainable in the long term.

Don’t hide the overall objective

Awareness and participation from the start are essential ingredients for sustainable change. Well-designed programmes have failed because the target group was not aware of them.

Participation also leads to ownership. If employees have a sense of ownership of the change, they are more likely to comply and sustain them. So teams should not only know the aims but also be involved in the transformation process. Depending on the nature of the programme, not informing people of the main aim could also have ethical implications.

When it comes to organisational change, failure can cost a lot of money and demotivate people. There are estimates that about 50-70% of interventions or programmes fail, which strengthens the argument for carefully-designed and implemented programmes that aim at sustainable change.

Deciding to go ahead with a programme of agile incremental change doesn’t mean that all the hurdles of updating a system magically vanish. The method comes with its quirks, which organisations have to carefully navigate if they want to reap the benefits.

Each company will have its own culture, requirements and working practices that influence the length of time a project and its individual stages will take.

Here, we explain what you could expect when it comes to taking your process of iterative digital change, with the help of two Contensis scrum masters’ experience.

Most challenging organisations

According to Joe Miller, scrum master at Zengenti and former digital marketer, the biggest challenge with any digital transformation is getting buy-in from team members for an approach they may be unfamiliar or less comfortable with.

He said: “Businesses that have traditionally used waterfall methodologies for all their projects up to this point have particular expectations shaped by that. Everything is set up for a type of feature delivery in which everything is unveiled at once, to great fanfare and carrying a weight of pressure and expectation from both senior stakeholders and customers — the so-called big bang release."

“Many organisations with this ethos have previously relied on printed media, like prospectuses and leaflets. Because print is a one-and-done medium, managers are wary of making costly and potentially embarrassing errors."

“Digital doesn’t have that same level of finality. Online content can be changed almost instantaneously and infinitely. For example, university course pages can be tweaked and changed in real time, as the needs and details of courses adapt. The descriptions can be bespoke to specific audiences, reflect course uptake or show different locations. All with minimal effort — none of which is possible in print.”

Public vs private sector

Of course, there can’t be a one-size-fits-all way of working. The different concerns and aims of public and private sector setups mean that each prioritises different aspects of the digital transformation process.

Drew Dinneen, head of professional services at Zengenti, has spent the past 10 years working across the digital industry in senior leadership positions, heading up operations, project and account management roles. He explains:

“In my experience, private sector clients are more inclined to make swift decisions that give the opportunity to release more frequently. Often, those approving the feature of change will accept that it is ‘good enough’ and will return value quickly, and can be enhanced later.

“For public sector clients, on the other hand, there’s often more layers of process to get through before a decision on whether to release or not can be made. There’ll often be several management hoops to jump through, rollback plans to be agreed, and a greater desire to ensure something is perfect before being exposed to the public.”

This could be down to the additional scrutiny that public services are under whenever a change is made, and the difference between the approaches has advantages either way. Extensive due diligence, consideration and being conscious of quality is simply what works best for those bodies, but it often slows down the speed of the project as a whole.

Time is a sliding scale

The length of time taken to deliver such projects varies considerably, with so many variables to allow for. Complexity accounts for a large part of this, but on two levels. Firstly is how complicated the project is as a whole, the nature of the business and its services.

Second is the complexity of each particular feature, which has a sliding scale for each component and is dependent on its purpose.

Take an element that needs to allow users to submit their feedback as an example. The most simple version of this might just involve displaying an email address so users can send their feedback. A more complex implementation might be to provide a form that the user can submit.

As an even more complex solution, that form might post to a third party API and trigger a subsequent action, such as downloading a report or unlocking a further feature of the website.

There are some time estimates that can be made on generalities, though, as Drew explains:

“For the lion’s share of projects, a typical UI component like an accordion or a card could be designed, developed, tested and released within the space of a day or two."

“The first components we tackle are always more time consuming, as we’re establishing conventions and creating elements for the first time. As the design system and the codebase builds, there’s more to inherit from, so the timeframes are gradually reduced. Towards the end, a simple component can be turned around in an hour.”

Whatever the practices and structure is in your organisation, helping your service provider understand it means that timescales can be much more accurately understood and projected for your project.

So what impact has an incremental approach to digital change had so far, and what is its potential for the future? Here, Contensis CEO Richard Chivers says how it has helped organisations the company has worked with, and explains why it could — and should — become the new normal.

Do more organisations looking for change already include a digital capacity in most of their services, which they need to update? Or do they still involve manual processes?

Most organisations include the digital capability in many of their services, but the basic expectations are often not met. Often, paper processes have just been moved over to a digital replacement, which is a great first iteration, but typically these can be reconsidered and improved.

Is the main hurdle to making big changes the psychological barriers of staff and users?

Yes, and I think we saw this especially during the pandemic. People are scared of change and have been almost stuck in a rut without moving things forward. That is unless their business has required changes to operate, in which case the opposite has occurred.

For the organisations you have worked with on incremental change, what has been the predominant theme of their feedback and experience?

We have learned a great deal in our trials for Blocks with specific clients, which we have been running for three years. Typically speaking, when making incremental changes, the feedback from organisations is that the team feels empowered and able to deliver value quickly and efficiently. The practice of making multiple iterations and learning from them makes the teams more confident, and able to unlock ideas that had once felt too scary.

Often, the biggest revelation is that what they thought they wanted is not what is ultimately delivered, and this surprises people positively and allows open-minded approaches. In terms of Blocks specifically, customers love that they can completely control their choice of web development frameworks and methodologies, whilst still having a SLA and support.

What are the other benefits you’ve seen for businesses?

We have typically found that team members are more engaged and just happier. Rather than one big change, we make a tiny alteration to start with, and then gradually build on it. People then realise that change is easy if managed in small chunks, and risks are smaller which gives more freedom to innovate.

Having happier team members, who are not scared or worried, means they can focus on business goals and objectives rather than worrying about “what if”, which hinders change.

Is this still a minority approach? How widely is it used at the moment?

Iterative development is now much more common in mainstream organisations. What we do often find, however, is that people say they are practising agile or iterative development but are not.

This is the most dangerous scenario because the team will normally fail, which can destroy morale. In this situation, senior management usually introduces a deadline, delivery specifications and a set time budget. They then say “let’s deliver this using an agile methodology”.

The trouble is that, because all of the flexible aspects are already decided, it will either work or fail based on how good the original time estimate was. Quality will nearly always be lower, if it does happen to work at all. There’s no room for innovation or new thinking: the team must follow the requirements and deliver them.

In a parallel world, a team actually delivering agile for real says “we have four sprints, and we need to ensure our customers can register their place on a webinar”, for example. In this case, there are no pre-set criteria and the team is free to solve problems in the way they choose, and to a high quality. For the latter example, they might decide to build a form from scratch, or they may integrate with Eventbrite and focus on training teams internally.

Should the incremental approach be adopted to all digital updates, or just for specific places?

Typically, iterative cycles are the most efficient way of working because changes can be managed and limited to a size the team is happy with, ensuring changes continue to flow and reactions to market and customer needs can be realised very quickly.

There are, however, some circumstances where iterative changes are not so good, but these are typically few and far between.

Where do you see the approach heading in the future? Are there still innovations to be made?

Absolutely. I think ultimately companies of the future will just use composable API first services, which can integrate to suit their needs. The innovation in my mind is around empowering people who understand customers’ needs, to deliver upon them in a way that can move even more quickly and efficiently.

It’s clear that the traditional method of updating digital operations, using a waterfall project management style, is a poor fit for modern organisations and can potentially do more harm than good in the long term.

With rigid, unwavering requirements that cannot properly accommodate solutions to problems, meaning you’re forced to ignore advances in technology that could improve the programme, the exercise can turn into a white elephant — a high cost and missed opportunity.

That’s before you consider the risk of putting all your eggs in one basket, keeping everything for one significant launch. And, of course, the disillusioning effect it can have on your teams, not to mention the all-too-human reaction of the end user, who can baulk at big changes they consider unnecessary or poorly thought through.

Using the agile model of incremental, iterative changes, one small step at a time and without the big shock of drastic upheaval, means that you have the flexibility to consider different approaches to challenges you will inevitably encounter.

This cycle of designing, building and releasing parts of a website, app or other digital function in bite-size stages means that, although each stage can use knowledge from those before it, everything can be adapted in turn, where it will benefit.

Perhaps most importantly, your employees are happier. They have control over the process, and can be heard if they raise concerns or alternative approaches about particular components — because it won’t be too late to adapt it. The resulting system will reliably be of a higher standard and, therefore, so will your end users’ experiences.

So, should digital transformation be a cliff edge? It can be, if old project management methods are used. With an iterative approach, though, it absolutely needn’t be.