Share Share this | Facebook Twitter YouTube LinkedIn Flipboard

IEEE: The expertise to make smart grid a reality

Subscribe to the Newsletter


*Required field

Publishing Information

Slicing and Dicing Smart Grid Data

Capturing all the benefits of the smart grid will require vast amounts of information to be communicated, collected, and processed. It will only be manageable if some basic differences between types of data are recognized and respected. Some information needs to be consulted frequently and in large quantities, but only locally, by the consumer; other information can be broadcast.

Smart meters were originally deployed to obtain the operational benefit of automated meter reading, but they rapidly became almost synonymous with the smart grid. It is now widely recognized, however, that the smart grid encompasses a much broader set of technologies that will deliver a much wider range of benefits, including improved reliability, power quality, efficiency, balance of supply and demand, and integration of renewable generation. And it is understood that garnering those benefits will depend on the use of information technologies and the collection of data from smart meters and other sensors in the grid.

That will require much more intensive data processing than just automated meter reads. To better understand data processing needs and plan for a scalable smart grid’s IT infrastructure, let us consider the different types of data related to end-customers:

Broadcast data. To balance the grid, data concerning grid status need to be communicated to market participants. That includes information on price changes, critical peak events, reliability signals, and--in the future-carbon content. Availability of this data makes the electricity market efficient by allowing consumers to make informed decisions, leading to better allocation and utilization of resources.

Billing interval data. For price signals to make markets efficient, some form of variable pricing must be in effect (time of use, critical peak pricing, peak time rebates, and so on). A utility needs to take readings at the beginning and end of a period during which a particular price is in effect, so that the utility can bill its customers properly.

Detailed consumption data. Energy customers need to know their current rate of electric consumption so that they can adjust it; they may also use automation controls to set their preferences so as to avoid having to constantly take actions.

Aggregate statistical data. Energy service providers may show customers their consumption by month, give them comparisons with neighbors or with historical time series, and other analytic results based on historic usage.

Distinguishing between these categories of data can be helpful in multiple ways. First, each type of data poses different latency and bandwidth requirements. Broadcast data typically has low volume because groups of players are getting the same information but the latency has to be low enough to allow end-users or their proxy systems adequate time to react in a meaningful way. Such specifications vary somewhat from group to group: For example, the ancillary services market has much tighter latency requirements than the demand response market, where peak prices are often declared hours or even a day in advance. Statistical data, on the other hand, is used relatively infrequently and does not need to be up-to-the minute.

There is often conflation between billing interval data and detailed consumption data, quite possibly because they both originate from the meter. An analogy may help here: In a car, both the speedometer and the odometer derive their readings from a device that counts wheel “ticks.” A car rental company uses odometer readings at car pick-up and return to determine the mileage driven during a rental agreement and bill accordingly. A driver needs to look at the speedometer continually to keep the vehicle’s speed below the limit; if the driver sets the cruise control, the cruise control reads the speedometer continuously to adjust the vehicle’s speed. The interval billing data is the “odometer” and is used by the utility to charge customers; the detailed consumption data is the “speedometer” and is used by customers (or their building management systems, or home energy manager devices) to adjust consumption.

Advanced metering systems were originally designed as "odometers;" the general requirement was for a system to be able to collect data from almost every meter between 12:00 AM and 6:00 AM, and it was tolerable for a few meters to be inaccessible for a few days (the data is stored in the meter and not lost). With the desire to understand consumption better and with the advent of meters that can take measurements of multiple attributes (such as power, reactive power, peak power, and voltage) at frequent intervals (typically every 15 minutes), the advanced systems started collecting load profiles, which can help with static (non-real-time) optimization.

But that is still a "batch" process, in which information is communicated and handled only periodically. New applications envisioned entail much heavier data requirements; for example, there is a belief that in-home displays will help consumers modify their behavior and reduce their peak and overall consumption (though research in this area is still inconclusive). In countries where a pre-pay option is popular, there is often the need or mandate to display a consumer’s balance continuously. For these applications, it may make more sense not to centralize the data, but to transmit it directly to the customer premises.

If a utility tried to collect detailed consumption data at a central site in real-time (interactively) once every 15 minutes (let alone more frequently), each customer would generate almost 100 transactions per day. This number is a multiple of the number of banking, credit card and airline transactions that a household generates per day, which implies that the smart grid would necessitate an IT infrastructure (computing and communications) larger than that of the banking, credit card, and airline industries together.

Other considerations also argue against uniform treatment of all information. Different categories of data have different life spans: Billing data needs to be kept long enough to satisfy auditing requirements and statistical data long enough to provide meaningful historical comparisons; but detailed consumption data loses its value fairly quickly--do you remember exactly what you were doing at 7:15 PM yesterday that might have caused a spike in your energy consumption?

The different nature of each type of data naturally lends itself to different communications media. Thus, for broadcast data, a broadcast medium may be most appropriate; prices and similar information can be posted on the Internet. But an end-user's consumption during a particular interval is specific to that end-user and requires a communications medium (such as an advanced metering network) that can address individual endpoints. Aggregate statistical data can be displayed on a portal, as centralized storage and processing allows comparisons to be made.

Finally, different types of data pose different privacy and security concerns. Prices are public knowledge, while detailed consumption information is sensitive, so that concerns about its confidentiality have occasionally led to the suspension of meter deployments, as seen last year in the Netherlands.


  • Christos PolyzoisDr. Christos A. Polyzois, former Director of Technology Strategy at Honeywell Automation and Control Solutions, is an independent consultant. Previously he has held technical and business positions with IBM, Bell Labs, McKinsey & Co. and Sarnoff. He studied at the National Technical University of Athens, Princeton University, and the University of Pennsylvania’s Wharton School.

    Read more

About the Smart Grid Newsletter

A monthly publication, the IEEE Smart Grid Newsletter features practical and timely technical information and forward-looking commentary on smart grid developments and deployments around the world. Designed to foster greater understanding and collaboration between diverse stakeholders, the newsletter brings together experts, thought-leaders, and decision-makers to exchange information and discuss issues affecting the evolution of the smart grid.


Melvin I. OlkenMelvin I. Olken is the editor-in-chief of the IEEE Power& Energy Magazine. Prior to assuming this position, he was the first executive director of the ... Read more


Christos PolyzoisDr. Christos A. Polyzois, former Director of Technology Strategy at Honeywell Automation and Control Solutions, is an independent ... Read more


Ronnie BelmansRonnie Belmans is a full professor at the Katholieke Universiteit (KU) Leuven in Belgium, where he teaches electric power and energy systems. ... Read more


Massoud AminMassoud Amin is director of the Technological Leadership Institute, professor of electrical and computer engineering, and ... Read more