Driving acceleration: the importance of observability in public sector digital transformation

When Prime Minister Scott Morrison announced a $1.2 billion investment in Australia’s digital future in the May 2021 Federal Budget, it was a clear signal that while the Covid-19 pandemic may have triggered a serious acceleration in the pace of transformation, his government has no intention of curbing its speed any time soon.

Andrew Goodall

“We must keep our foot on the digital accelerator to secure our economic recovery from Covid-19,” said Mr Morrison. 

Almost half of the new funding will be spent on improvements to online government services, including the myGov website and My Health Record system. That makes sense, given the increased uptake of such services during lockdowns and higher public expectations around their delivery.

In other words, Australians increasingly hold public-sector organisations to the same standards of seamless online efficiency as they do e-commerce retailers, food delivery apps, and ridesharing services. There’s no distinction for them between their online interactions as citizens and their online interactions as consumers.

But there are plenty of obstacles for individual government agencies and departments to navigate if they are to meet those high standards. For public-sector IT teams in particular, a major challenge is a distinct lack of observability. Without observability, the acceleration of digital transformation could be a risky journey, resulting in poorly performing digital services and frustrated, resentful users.

Australians increasingly hold public-sector organisations to the same standards of seamless online efficiency as they do e-commerce retailers

Observability: Keeping a watchful eye on digital services  

The term ‘observability’ refers to real-time performance monitoring that helps IT teams detect, diagnose and resolve issues that might otherwise hamper the performance of digital services.

The problem here is that government digital services tend to straddle both cloud-based and on-premise systems. A new online portal or app for citizens, for example, will likely run in the cloud, but may depend on close integration with older, on-premise systems to access data or to perform certain functions. 

Legacy monitoring and security tools, which tend to focus on individual IT siloes, will consistently fail to give IT admins a complete view of digital services’ end-to-end business and IT performance. In short, admins can’t see the whole picture – a distinct disadvantage when it comes to spotting issues and dealing with them.

Certain characteristics of modern digital application stacks – such as microservices and the shift to containerised workloads – simply add to the challenge of observability by dramatically adding to the volume of individual technology components that support a digital service. And digital transformation of government services also has important implications for cybersecurity and fraud, since the launch of a new portal, service or app potentially exposes a whole range of new attack surfaces.

All this points to a need for IT teams to rethink how they achieve the observability they need to keep digital services up and running.

The term ‘observability’ refers to real-time performance monitoring that helps IT teams detect, diagnose and resolve issues

From data to insights to action

Modern approaches to observability typically depend on every piece of technology making up a digital service being rigorously instrumented. This involves equipping the underlying software with the code it needs to report on errors and performance issues. By accessing this data, IT teams can mount a proactive and predictive detection and response capability – one that is equipped to respond quickly and efficiently to both performance issues and security threats.

But when a great number of instrumented digital services are running – as is increasingly the case in government IT environments – a vast number of signals and data can be generated. In Australia, for example, the myGov portal already provides access to more than 900 online services, according to the World Bank’s recent GovTech Maturity Index (GTMI) report.

This is where modern observability solutions come into play, by collecting all of that data and bringing it into one place for enrichment, analysis, visualisation, reporting and alerting.

While siloed IT monitoring tools can only ever provide visibility within their narrow domains, an observability solution can not only collect and present data from across different siloes via a single console, but also layer it with IT and business service context. In other words, an observability solution tells IT teams what’s not working, why it’s not working and what needs to be done to fix it, as well as the likely impact on user experience if the issue isn’t addressed.

For example, data relating to application performance trends can be supplemented by additional context, such as social media sentiment, call centre queue length and IT configuration management changes, giving teams the insight they need to respond to incidents before they impact service delivery.

The right observability solution also enables teams to take a deep dive and get more granular in their trouble-shooting processes, based on information that might otherwise remain hidden. This way, they have a fresh opportunity to look at how individuals are interacting with government services, understand what is popular and effective and identify pain points early on.

Another key benefit of modern observability technology is automation. These solutions use machine learning to detect anomalies, and in some cases act on them, based on their understanding of acceptable performance thresholds. By allowing technology to take the strain in this way, IT admins are freed up to work on more valuable and fulfilling activities.

An observability solution tells IT teams what’s not working, why it’s not working and what needs to be done to fix it

Navigating the road ahead

The move to cloud-based software and digital service delivery offers a huge opportunity for the public sector to improve citizen experience and engagement – but observability will be a key ingredient for making the most of that opportunity.

Many government organisations worldwide are already using observability technology. In the UK, for example, they include the Driver and Vehicle Licensing Authority (DVLA), where engineers are migrating IT systems to new cloud-based platforms, as part of a wider modernisation initiative at the agency. They use a search platform to centralise logs and metrics from cloud infrastructure, to analyse and quantify the performance and usage of apps located there and deliver a better experience, for both internal users and UK motorists in general.

While observability can be applied to a wide range of situations and digital services, what these projects have in common is that they enable IT teams to centralise information from many different sources and at scale, allowing them to ‘connect the dots’ and spot trends and anomalies that might otherwise remain under the radar – until a digital service fails and complaints come flooding in.

At a time when poorly performing digital services are likely to be a major source of frustration and disappointment to many sectors of the population, the race is on to deliver the digital experience they expect – and soon.

Andrew Goodall is the Federal Director, Australia, at Elastic

Comment below to have your say on this story.

If you have a news story or tip-off, get in touch at editorial@governmentnews.com.au.  

Sign up to the Government News newsletter

Leave a comment:

Your email address will not be published. All fields are required