Transforming Data Use at the UK Energy Regulator

Office of Gas and Electricity Markets (Ofgem) are an economic regulator, a core member of the UK Regulators Network and a non-ministerial government department. Their principal objective is to protect the interests of existing and future electricity and gas consumers. This remit includes the 28 million households who consume electricity and gas today, as well as managing the development of the future energy system.

Ofgem works with a broad range of stakeholders from across the energy sector (retail companies, network companies and market participants), academia, the UK government and the European Union.

Valtech partnered with Ofgem over a period of 30 months to deliver digital transformation. Our co-located, co-sourced team faced three main challenges; internal data storage and use, data collection and organisational change. We dedicated ourselves to solving these challenges incrementally, initially using small 3-week engagements to de-risk the work and up to 9-month pieces once problems and direction were better understood.

We sought to change the organisation through building a new digital service that would manage data; the Data Hub (an organisational data store) would put data at the heart of Ofgem’s approach to regulation. This platform provides common methods to validate, organise and securely store data giving a single source of truth to regulatory users.

External to Ofgem, regulatory data collection represented a huge burden to industry with many different touch points (online portals, USBs, emails) creating confusion. We created a single Data Exchange service as the one-stop shop for the energy industry. This portal reduced the burden on those who submit relatory data to Ofgem. A complete user journey from data submission to validation and acceptance is now automated and takes under an hour, instead of many weeks, as part of discrete manual processes.

Throughout our work, we focused on building a coalition of change to show how adoption of new ways of working and technologies can give value back to the user. The primary drivers were new efficiencies through automation, greater transparency of regulatory processes and, ultimately, greater insight from the data available. Valtech helped to hire and establish the core Data Services team and collaborated with leaders across the organisation to place a new focus on agile ways of working and organisational agility.

Over many objective and key result cycles, sprints and thousands of tickets, we leave behind Ofgem as an organisation hardly recognisable from the one we joined, the data services team is established and new technology in place. This is an Ofgem still relevant to a fast changing world and hopefully capable of delivering against the challenges of decarbonisation, decentralisation and increased digitalisation.