Document Type : Reseach Article
Abstract
The integration of advanced metering technology in power systems has enabled real-time data access for every node in a smart grid. As a result, the power system can now access large volumes of data. This vast amount of data requires an alternative method of analysis. Machine learning-based load forecasting technologies are being applied in this scenario. However, this massive data collection needs to be processed through the appropriate data pre-processing method, such as the removal of noise, outliers, and erroneous data, the detection of missing data, the normalization of widely divergent datasets, etc., to improve the effectiveness of the load forecaster. Thus, to eliminate the various kinds of errors and outliers present in the data that was directly obtained from smart meters, this study analyses and compares the efficacy of eight distinct smoothing and filtering techniques as a novel contribution of this work. Using the processed data acquired, a neural network-based load forecasting model was developed to compare the efficacy of the various
preprocessing approaches. This study makes use of real-time data obtained from the smart meter placed at a node within the NIT Patna campus. The proposed moving average filter surpasses the other methods for filtering and smoothing the raw data by an average MAPE of 2.66, according to the load forecasting results that were obtained.
Keywords
“Tuning data preprocessing techniques for
improved wind speed prediction.”. Energy Reports, 11:pp. 287–303, 2024. DOI:
https://doi.org/10.1016/j.egyr.2023.11.056.
[2] A. Parashar, A. Parashar, W. Ding, M. Shabaz,
and I. Rida. “Data preprocessing and feature selection techniques in gait recognition:
A comparative study of machine learning and
deep learning approaches.”. Pattern Recognition Letters, 172:pp. 65–73, 2023. DOI:
https://doi.org/10.1016/j.patrec.2023.05.021.
[3] B. Boashash and Ed. “Chapter 11 - TimeFrequency Synthesis and Filtering.”. in TimeFrequency Signal Analysis and Processing (Second
Edition), Oxford: Academic Press, pages pp. 637–
691, 2016. DOI: https://doi.org/10.1016/B978-0-12-
398499-9.00011-X.
[4] S. Rai and M. De. “Effect of Filtering in Big Data
Analytics for Load Forecasting in Smart Grid in
Machine Learning, Image Processing, Network Security and Data Sciences, A. Bhattacharjee, S. Kr.
Borgohain, B. Soni, G. Verma, and X.-Z. Gao, Eds.,
in Communications in Computer and Information
Science..”. Singapore: Springer, pages pp. 125–134,
2020. DOI: https://doi.org/10.1007/978-981-15-6315-
7-10.
[5] M. Aouad, H. Hajj, K. Shaban, R. A. Jabr,
and W. El-Hajj. “A CNN-Sequence-to-Sequence
network with attention for residential shortterm load forecasting.”. Electric Power Systems Research, 211:p. 108152, 2022. DOI:
https://doi.org/10.1016/j.epsr.2022.108152.
[6] V. Chinta, G. Song, and W. Zhang. “Validation of
the medium-range and sub-seasonal forecast of solar irradiance and wind speed using ECMWF.”.
Energy Reports, 10:pp. 3908–3913, 2023. DOI:
https://doi.org/10.1016/j.egyr.2023.10.058.
[7] A. S. F. Rocha, F. K. de O. M. V. Guerra,
and M. R. B. G. Vale. “Forecasting the Performance of a Photovoltaic Solar System Installed in other Locations using Artificial Neural Networks.”. Electric Power Components
and Systems, 48(1-2):pp. 201–212, 2020. DOI:
https://doi.org/10.1080/15325008.2020.1736211.[8] J. W. Taylor and R. Buizza. “Neural network load forecasting with weather ensemble predictions.”. IEEE Transactions on
Power Systems, 17(3):pp. 626–632, 2002. DOI:
https://doi.org/10.1109/TPWRS.2002.800906.
[9] S. Chapaloglou and et al. “Smart energy management algorithm for load smoothing and peak shaving based on load forecasting of an island’s power
system. ”. Applied Energy, 238:pp. 627–642, 2019.
DOI: https://doi.org/10.1016/j.apenergy.2019.01.102.
[10] P. Mishra, A. Biancolillo, J. M. Roger, F. Marini,
and D. N. Rutledge. “New data preprocessing trends based on ensemble of multiple preprocessing techniques.”. TrAC Trends in Analytical Chemistry, 132:p. 116045, 2020. DOI:
https://doi.org/10.1016/j.trac.2020.116045.
[11] E. Escobar-Avalos, M. A. Rodr´ıguez-Licea, H. RostroGonzalez, A. G. Soriano-S ´ anchez, and F. J. P ´ erez- ´
Pinal. “A Comparison of Integrated Filtering and
Prediction Methods for Smart Grids.”. Energies, 14
(7), 2021. DOI: https://doi.org/10.3390/en14071980.
[12] E. R. Davies. “CHAPTER 3 - Basic Image Filtering
Operations.”. in Machine Vision (Third Edition), E. R.
Davies, Ed., in Signal Processing and its Applications.
Burlington: Morgan Kaufmann, pages pp. 47–101,
2005. DOI: https://doi.org/10.1016/B978-012206093-
9/50006-X.
[13] E. Hussein. “Preprocessing of Measurements.”. pages pp. 97–123, 2011. DOI:
https://doi.org/10.1016/B978-0-12-387777-2.00009-
4.
[14] C. Becker and U. Gather. “The Masking Breakdown
Point of Multivariate Outlier Identification Rules.”.
1997. DOI: https://doi.org/10.17877/DE290R-15061.
[15] D. J. Robb and E. A. Silver. “Using Composite Moving Averages to Forecast Sales.”. The Journal of the
Operational Research Society, 53(11):pp. 1281–1285,
2002.
[16] “Introduction to Modern Time Series Analysis.”.
SpringerLink, 2023. URL https://link.springer.com/
book/10.1007/978-3-540-73291-4.
[17] J. Luo, K. Ying, and J. Bai. “Savitzky-Golay smoothing and differentiation filter for even numberdata.”.
Signal Process, 85(7):pp. 1429–1434, 2005. DOI:
https://doi.org/10.1016/j.sigpro.2005.02.002.
[18] W. S. Cleveland and S. J. Devlin. “Locally Weighted
Regression: An Approach to Regression Analysis
by Local Fitting.”. Journal of the American Statistical Association, 83(403):pp. 596–610, 1988. DOI:
https://doi.org/10.1080/01621459.1988.10478639.
[19] G. E. P. Box. “Time Series Analysis.”. 5th edition,
2015.
[20] Z. Qu, J. Xu, Z. Wang, R. Chi, and H. Liu. “Prediction
of electricity generation from a combined cycle
power plant based on a stacking ensemble and its
hyper parameter optimization with a grid-search
method.”. Energy, 227:p. 120309, 2021. DOI:
https://doi.org/10.1016/j.energy.2021.120309.
[21] S. Raschka. “About Feature Scaling and Normalization.”. 2014. URL https://sebastianraschka.com/
Articles/2014 about feature scaling.html.
[22] S. Rai and M. De. “Analysis of classical and machine learning based short-term and mid-term load
forecasting for smart grid.”. International Journal
of Sustainable Energy, 40(9):pp. 821–839, 2021. DOI:
https://doi.org/10.1080/14786451.2021.1873339.
)