Data is critical connective tissue for the electricity system, but are we on track to unlock its power?

Image from Canva used to illustrate data related post on the Wattwatchers Blog

Long overdue reforms like action on 7-star home energy performance standards, vehicle fuel standards and giving emissions and the environment legal status in the electricity market carry a bigger message: transformative change is coming within reach for millions of household and small business electricity customers. So how do we collect, manage and share the data for that?


It’s hardly news in 2022 that technology and data are critical enablers for the energy transition, supporting an accelerated shift from dirty, unstable, expensive and analogue, to clean, affordable, reliable and digitally-smart.

The real question is how do we make this happen in ways that are consistent with ever-evolving data capabilities and culture, matching the economy’s and the energy system’s operational and market needs on one side, and the online-world expectations of consumers on the other.

A big problem is that ‘Old Energy’ is still trying to bend technology and data to its traditional, centralised, industry-dominated, command-and-control model, Grid 1.0. What we need is a paradigm change to a ‘New Energy’ model, Grid 2.0, which is real-time, flexible, decentralised and customer-empowering.

This is not a new challenge

Back in 2017, in my first year at Wattwatchers, I went to Canberra to talk about consumer electricity data with the office of then Minister for the Environment and Energy, Josh Frydenberg.

To my surprise, it was quite a refreshing experience, given how our nation was bogged down politically on climate and energy policy reform, with chronic inaction on solutions. 

I didn’t get an audience with the Minister himself. But the departmental representatives I met were informed, engaged, and focused on early data-related opportunities: the CSIRO’s Energy Use Data Model (EUDM), which later morphed into the National Energy Analytics Research (NEAR) program; and the Australian Treasury-led Consumer Data Right (CDR).

Now, over five years later, an initial version of CDR for Energy will go live this November, following CDR for Banking last year, and ahead of CDR for Telecommunications in 2023, with another industry sector to be added each year ongoing. 

It’s instructive to me that datasets from Banking, Energy and Telecommunications will all be part of empowering consumers to participate successfully in the decarbonised, digital and distributed electricity systems of the near future. And that’s just a start.

A whole ‘dataverse’ surrounds energy, stretching from home appliances and the daily weather, all the way to major infrastructure, the economy at large, and the sustainability of the planet. The graphic below is an early, inevitably incomplete attempt to map all of the things that will play a part in this evolving data ecosystem. 

FIGURE 1: Relevant data will come from numerous sources, both energy-related and non-energy. The real challenge is to integrate and coordinate data so it is broadly useful, safe and value-creating for multiple stakeholders: customers, energy system operators and market players, researchers, diverse service providers and more.

What CDR Version 1 does, and what it won’t do

CDR is meant to standardise how energy utilities and industry bodies provide access to customers’ usage and billing data. Done well, it will streamline customer-approved sharing of data with third-party service providers, and shift the data dial towards more open access while maintaining security and privacy protections.

But CDR as it stands only addresses a subset of useful energy and energy-relevant data. Its data is sourced overwhelmingly from utility ‘smart meters’ – which are yet to even be installed at over half of Australian homes – and is blind to the new growth space: all manner of smart monitoring and remote control devices, solar and battery inverters, electric vehicle chargers, and intelligent machines and appliances.

Data from these new sources is critical connective tissue for the ‘New Energy’ era, and unlocking the multiple layers of value it offers – to consumers, service providers, the system, and the wider society and economy – is key work for an orderly and accelerated clean energy transition. 

The passage of time on CDR, more than half a decade, affirms a number of things for me:

  • The wheels of change continue to turn far too slowly – even torturously – when you have to rely on government processes, especially in a highly-regulated, politically-contentious and excessively-siloed sector like energy.
  • This lag time for much-needed change is especially problematic when you look at it through the lens of technology, with five years representing a generation in the context of fast-evolving information and communications technologies, and also ‘New Energy’ technologies.
  • And while there is recognition of the importance of energy-related data – including its accessibility and share-ability, and its security and privacy, as well as its functionality, integrability and core role for technology interoperability – the pace, reach and scale of reform is still too little, too narrow and too late.

Looking beyond the energy ‘establishment’

What I see now, in 2022, is a lingering contest between the legacy players of the energy sector, whose influence remains even as their relevance fades or at best changes, and the solution enablers of the future. (Among which I include Wattwatchers, but also see many other players ranging from new early-stage local startups to global innovators like Tesla.)

It remains an unequal contest, because the legacy players still hold considerable power – based on incumbency, propped up by control of billing data, deep pockets, and embedded recognition as being the go-to stakeholders for regulatory reforms – while even proven technology and data-driven solution providers are still operating at the fringes of the ‘energy establishment’.

At the big picture level, emerging solutions – the enablers of the customer-centric future we need for Net Zero priorities of decarbonisation and electrification – are still being held at arm’s length as mere ‘alternative technologies’ by this conservative, risk-averse and self-protective establishment. 

As an example, when CDR for Energy does finally start in just a few months, it will be restricted initially to energy customer data from technologically-outdated utility meters held by the Australian Energy Market Operator (AEMO), and the nation’s traditional big three energy retailers, AGL, Origin and Energy Australia.

This happens to coincide with the nascent ranks of ‘New Energy’ retailers, the emerging online and community-based challengers to the business models of the top tier and also more-traditional mid-tier players, being decimated by the latest round of supply crises and soaring prices (which in turn can be blamed on past energy and climate reform failures).

Meanwhile, many among the 10 million-plus households and enterprises which make up the customer base of the low-voltage mass-market electricity system in Australia are left wondering: When will we feel some real benefits from the energy transition?

So what can we do to fast-track data-driven solutions?

Australia lacks a vibrant, broad-based dialogue on the role of enabling technologies and data-driven innovation for the energy sector, particularly where it meets increasingly tech-savvy consumers, in the home or business, and behind the utility’s billing meter.

I say ‘dialogue’ deliberately, because we need much more than just another program or working party. The data dialogue needs to be sustained, elevated and expanded across the board, with many touchpoints in reform agendas that are already underway or mooted, including but not limited to: 

  • The Consumer Data Right (for Energy and for everything).
  • The Australian Energy Market Commission (AEMC) Metering Review, which follows less than stellar outcomes from the Power of Choice metering reform process that began in 2011.
  • The Energy Security Board (ESB) Data Strategy, which has the ultimate objective of making the energy system fit-for-purpose for the 21st century.
  • The National Construction Code, with the extended stakeholder campaign to move to a 7-Star baseline for residential buildings having just succeeded.
  • The Distributed Energy Integration Program (DEIP), coordinated by the Australian Renewable Energy Agency (ARENA), and involving all of the peak bodies across the energy sector.
  • The long-dormant opportunity for a national energy-saving scheme, similar to state-based models in NSW and Victoria, which can include using data for real measurement and verification of results.

Much to talk about, but it can’t just be a talkfest

There’s a lot to include in a cross-cutting dialogue that reaches beyond the usual suspects. 

Methodologies. Standards. Taxonomies. Integration. Interoperability. Apps. Application Programming Interfaces (APIs). Communications. The Internet of Things. Matter, the upcoming global smart home protocol backed by Apple among others. The Green Button and Orange Button initiatives in the US. Open data. Public interest. System and market flexibility. Proprietary data and privately-controlled ‘data walled gardens’. Balancing cybersecurity with over-the-air aggregation models like Virtual Power Plants (VPPs). 

Consumer data rights, broadly defined, and also the rights of enterprises that add value to data and make it more accessible and useful. And the UK’s strategy for modernising energy data, based on the belief that: Transparent, accessible, interoperable and accurate data will allow the markets to develop (in ways) that will put consumers at the heart of this change while allowing networks to support the proliferation of new business models and technologies.

There’s so much more to cover in this dialogue, although just talking about energy data won’t be anywhere near enough. 

We need the dialogue to be opened up, as well as the data, and not locked away in often-complex intra-industry processes and forums. We’ll need well-targeted policies and a suite of collaborations to entice or compel diverse parties to participate, with particular emphasis on consumers, the data they are creating from their sites, and the equipment and devices that can help them, and also the energy system and the wider economy.

And we can’t just incrementally retrofit today’s data potential to the ‘Old Energy’ paradigm. We have to really shake things up, with an aggressive agenda for step change. We need to move quickly, because there’s no time to waste when there’s a real opportunity to deliver Net Zero for electricity within a decade, by the early 2030s, enabled by data, technology and empowered customers.

FOOTNOTE: Back in 2017, I also was in Canberra for an ARENA A-Lab innovation session where Wattwatchers presented a data-driven project opportunity for a ‘people’s energy data bank’. That idea led to our $9 million-plus My Energy Marketplace (MEM) project, supported by a $2.7 million grant from ARENA, which is creating new customer-sourced datasets and tools to make consumer data portable and shareable, supporting greater choice of services and solutions for customers, and useful access to customer-generated data for the industry, communities, researchers and solution developers, and commercial service providers. This year Wattwatchers has added energy data as a service as a third pillar of our business model, alongside hardware sales and subscriptions. We’re walking our data talk.

ARENA logo


The My Energy Marketplace project is receiving funding from ARENA as part of ARENA’s Advancing Renewables Program. The views expressed herein are not necessarily the views of the Australian Government, and the Australian Government does not accept responsibility for any information or advice contained herein.