Data, AI, and the future of the Public Sector

by Victoria Thomason

09 Jul 2024

Andrew Taylor, Information Architect at City of York Council (CYC), talks to us about data in the Public Sector, the impact of AI, and what the future of the industry looks like.

Tell us a bit about your team’s role at the City of York Council.

Andrew: Our team manages the flow of data and information at City of York Council, which includes performance indicators, and evidence-based decision making. We’re responsible for the operational and management reporting that allows staff from all levels to see what’s going on with their data.

We’re heavily involved in ensuring the council meets transparency legislation around public data, supporting with Freedom of Information (FOI) requests and running our open data platform that allows the public to see our data.

We’re also responsible for the statutory returns, which have evolved over the last few years from aggregated returns to individual item level return, dealing with data on individual clients, customers, and citizens.


Has that shift from aggregated statutory returns to individual level statutory returns, made the need for accurate data at the individual level more important? 

Andrew: Absolutely, the accuracy and density of the data has grown massively more important. Previously, central government was purely interested in the collection of aggregate level statistics from local government. It was a high-level percentage or average figure required. Now the data collections are at the client or item level, so the person themselves and the data items they’re connected with. Along with the time resource required to get at this data, we’ve had to build trustworthy processes for collecting data and ensuring data consistency and accuracy, so that we don’t have a horrendous process to clean up the data when we get to the deadline.


How do you find the process of getting data sharing agreements over the line for multi-disciplinary working, in-house and externally?

Andrew: Internally, it’s a lot easier than it ever used to be and I think we’re seeing the benefits of statutory roles like DPOs. But externally, data sharing is still quite difficult to achieve. That’s not because external partners, such as the police or the NHS don’t need or want to share information where appropriate – they do. The issue is resource.  Funding for public sector data or governance roles isn’t adequate and the levels of staff aren’t sufficient for the demand. Getting sign-off within organisations, therefore, takes quite a long time; so, it’s not that people don’t see the relevance of it, they just don’t have the capacity to move it along at the pace we’d like.


You and your team have done some brilliant things with data and your ambitions have often been greater than what time and resource would allow. What’s the focus for City of York Council right now?

Andrew: Like all local authorities right now, we’re facing lots of budgetary pressures, so it’s difficult to look too far ahead into the future and see it clearly. I’ve always seen it as a challenge to innovate with little resource, work hard and deliver something successful. But, right now, with budgets as they are, we must be grounded and focused on “keeping the lights on” and prioritising data projects that are statutory or adequately funded within the organisation. As a data service, we’ve still got our ambitions, but with limited budgets and resources, we’ve got to be pragmatic and make sure we keep things running before anything else.


The million-dollar question that everyone’s talking about right now, is how would AI cope with replacing your manual decisions, whether in data quality or service provision?

Andrew: AI feeds off consistent data and the quality of its decision making on service provision is only as good as the quality of the data you give it. Right now, I think AI would struggle to come up with effective decisions based on public sector data. Take for example an Educated Health and Care Plan – there’s statutory guidance on what must be in that plan but no guidance at a national level on a homogenised data structure. There’s so much variety in the structure, file types, layout that mining the data would be a challenge. Aside from the individual plans themselves, AI would need the education, health and care data at population level where, again, there’s a multitude of data unstructured data.

We’ve only just started seeing the first few iterations of AI use in local authorities. But the technology still has a long way to go before it can be used comprehensively and consistently. For instance, AI that identifies defects in road surfaces really only works when the surface is dry because rain changes the colour and texture of a road surface. In the rainy northern UK, that’s a lot of days where the product is essentially useless!


In an ideal world, if money, resource, and time were no object, what would utopia at City of York Council be from your perspective?

Andrew: That’s quite an exciting thought! Personally, I feel we would benefit from more data analysis. With limited budgets, we’ve had to be pragmatic and focus on the statutory obligations – which have themselves increased in number and complexity over time! So, if money were no object, I’d want us to engage in more data analysis and deliver more intelligence about what data shows us about the City of York.

We have some solid core technologies in our architecture – MDM, Single Customer View, an OpenData platform and a consolidated way of reporting organisational performance. With unlimited resource, we’d take that to the next level. We’d harness all data and architecture to give us near real-time, systematic intelligence on our City and use that evidence base to direct policy making to improve customer satisfaction and service.


Pamela: Do you have any projects upcoming that you’re excited about, perhaps because you’re able to do something a bit innovative?

Andrew: Most projects we’ve got upcoming are what you would consider “bread and butter” projects, but I’m always looking for opportunities to add a bit of ‘jam’; creating added value in a standard or necessary project. As a large public sector organisation, there is a cycle or systems reaching their end of life that we need to replace. We’ll be looking at our CRM with the decision to maintain or replace; and we’re moving our HR system to the Cloud imminently. We have highways related projects looking at road condition, and live time circular ownership and feedback on assets.

For our CRM, for me the “jam” will be connecting the data back into Infoshare ClearCore MDM and the opportunity to see if we can improve the data to help us better deliver for our citizens. Digital for all local authorities is challenging; Councils are like a collection of mini organisations all under one roof. A CRM is a piece of core architecture that provides a fantastic opportunity to bind the parts of the organisation together through the data and processes.


Is it hard, especially when budgets are tight, to deliver projects that don’t break the bank, but are maintaining the service for customers?

Andrew: Yes, especially where the understanding of the cost of projects is underestimated. It’s too easy at the start of a project to underestimate the cost of work in a system change, for example, when it is a complicated, intricate process. Part of our job is helping people to understand the true cost of work involving data to ensure successful projects and highlight the value it can deliver for customers.

That’s why Infoshare, and ClearCore, has been an easy solution to position, as it’s such a good value product. In a recent peer review with directors from other local authorities we demonstrated our Single Customer View, which relies heavily on the data produced by ClearCore. There’s a presumption that costs for MDM products are way higher than they are, and they were pleasantly surprised at the true cost – ClearCore is incredible value for money, especially considering the number of applications it can underpin.

This is where ClearCore and Infoshare are very different from other suppliers on the marketplace – the costs are fair, it delivers excellent tangible value, and you’ve got a brilliant support team available for after it’s set up.


What does the technological future look like for local authorities?

Andrew: I think the future looks increasingly Cloud based with a variety of underpinning technology. Where local authorities may have been able to control the fundamental DMBS, I think we’ll start to see suppliers introducing NoSQL products like MongoDB. There’s the potential for shared services around technology where a balance of risk and cost is shared.

It’s already happening to some extent, and there are some services that are naturally easier to share than others. Microsoft support this with Teams for example, where you can affiliate two organisations and see each other’s Active Directory lists and collaborate directly within Teams. However, I think the software market needs to adapt further to accommodate the kind of affiliation and shared services local authorities are going to need.


Read more

CASE STUDY: Building a Single Citizen View for City of York Council

News: City of York Council shortlisted for Public Service Award