Synchrophasor Data Anomaly Detection and Classification Considering Data Drift and Contextual Information

Written by Sagnik Basumallik, Anurag Srivastava, Arman Ahmed and Yinghui Wu

Motivation for Synchrophasor Data Analytics and Impact: Uncertainty of renewables, penetration of inverter-based resources, and varying load dynamics have increased the operational complexity of the grid. Monitoring the power system dynamics under such highly complex scenarios is essential, and Phasor Measurement Units (PMU) provide synchronized high-resolution measurements. PMUs enable coordinated system-wide situational awareness and faster post-event analysis by providing real-time Global Position System (GPS) time-stamped voltage and current phasor data [1]. Analyzing PMUs data comes with multiple challenges, such as data drift, data anomalies, and missing data. For data quality, efficient approaches are needed to detect, process, and remove such inconsistencies from the PMU measurements before being provided to critical PMU data analytics applications.

 

Challenges: Data anomalies, PMU estimation, Missing Labels, Data Drift

PMU data quality varies widely as measurements are transferred through the communication network. Inconsistencies in data may arise due to actual changing grid operating conditions, measurement inaccuracies, or errors in end-to-end synchrophasor systems including communication networks. To prevent low-quality data from producing anomalous results leading to incorrect operator decisions, it is imperative that PMU data be monitored for availability and accuracy by addressing the following challenges.

1. Data anomalies: Sources of measurement anomalies include biases and calibrations errors in instrument transformers, noise, device and channel errors, incorrect data alignment, missing timestamps, loss of packet data, network congestion, problems during data compression, and harmonic interference. The anomalies may occur at multiple PMUs or at the data concentrator level [2].
2. PMU estimation: These are induced errors introduced by the PMU estimation and filtering process which include synchrophasor total vector error, magnitude error, phase error, frequency error, and rate of change of frequency error typically higher during system dynamics [3].

In addition to data anomalies and induced errors, two additional challenges can arise during data analytics of PMU data.

1. Missing labels: Machine-learning based events or anomaly detection algorithms require a considerable amount of labeled PMU measurements, which are often not available due to the excess manual labor and costs involved.
2. Data Drift: Drifting in PMU measurements occurs due to dynamic changes in operating conditions such as asset tripping, line reconnection, or switching events. The drift can be gradual or sudden, and in both cases, the statistical properties of the PMU data change over time. As a result of drift, the definition of PMU data anomaly changes. The data that was considered normal at time t may not be considered normal later. Any machine learning or data analytics algorithm that was trained on one set of PMU data may no longer make accurate anomaly predictions, leading to model degradation and reliability over time.

 

Solution Strategies: Dynamic Change Driven Learning with Data Change Detection Algorithm

We aim to address this key challenge of integrating contextual information in anomaly detection. For context sensing, we introduce the dynamic Data Change Driven Learning (DCDL) method that incorporates data drift [4].

Consider the case of online anomaly detection when data drift is detected. In this situation, the dynamic DCDL trains an unsupervised deep-learning algorithm that takes additional input from a non-parametric multivariate Data Change Detection Algorithm (DCDA). The objective of the DCDA module is to identify when the data statistics have changed. The entire algorithm is implemented at the PMU device level, shown in Figure 1 (below), and is explained as follows. Consider a single PMU streaming n measurements. A total of N data points for one measurement is taken as the length of one single window. To detect data drift, the data window is first divided into two equal halves - from 1:M and from M+1:N. Histograms are generated for PMU data in each window, and for each bin q, the ratio of the total number of PMU measurements in bin q over the total number of measurements in each window is calculated. This is repeated for each measurement in both windows. Finally, a log-likelihood measure is calculated over all bins for each measurement and the maximum of this measure is compared against the threshold for data drift detection. The intuition is that under data drift, the log-likelihood measure successfully captures statistical differences between the two windows by quantifying the differences in the histogram bins. To filter anomalous measurements under data drift, a pre-trained unsupervised deep-learning module is used for prediction and generation of pseudo-labels for re-training. The anomalous data is filtered out using intermediate labels and the deep-learning module is retrained on the new data statistics as shown in Figure 1. Anomaly classification is subsequently carried out.

 

Figure 1: PMU data anomaly detection with data drift module

 

Future Needs: Integrated Data Anomaly Detection and Classification, Cybersecurity, Data Value

The proposed approach addresses the challenge of integrating contextual information when the definition of PMU data anomalies varies over time due to changes in power system operating conditions. In the future, integrated anomaly detection and classification for cyber-physical data is required. Also, contextual information can further be incorporated into cyber anomaly detection when changes in the inter-dependent PMU communication network result in data drift. Changes in the cyber data distribution may occur under dynamic changes in the network routing table through software-defined networking, or server and link failures, which need to be accounted for. Also, PMU data needs to be considered for multiple applications enhancing the data value. Integrating context in both cyber and physical anomaly detection modules will result in building a secure and resilient grid.

 

 

References

1. Phadke, Arun G. "Synchronized phasor measurements in power systems." IEEE Computer Applications in Power 6, no. 2 (1993): 10-15.
2. Khaledian, Ehdieh, Shikhar Pandey, Pratim Kundu, and Anurag K. Srivastava. "Real-time synchrophasor data anomaly detection and classification using isolation forest, kmeans, and loop." IEEE Transactions on Smart Grid 12, no. 3 (2020): 2378-2388.
3. NASPI, "PMU data quality: A framework for the attributes of PMU data quality and quality impacts to synchrophasor applications." (2017).
4. Ahmed, Arman, K. Sadanandan Sajan, Anurag Srivastava, and Yinghui Wu. "Anomaly detection, localization and classification using drifting synchrophasor data streams." IEEE Transactions on Smart Grid 12, no. 4 (2021): 3570-3580.

 

This article was edited by Ali Nabavi.

To view all articles in this issue, please go to January 2023 eBulletin. For a downloadable copy, please visit the IEEE Smart Grid Resource Center.