Research on Market Power Platform for Power Generation Based on Spark Distributed Framework Computing

The Spark distributed algorithm is a distributed computing implementation based on the map reduce algorithm, which has the advantages of Hadoop MapReduce. However, unlike MapReduce, the intermediate output and results of jobs can be saved in memory, eliminating the need to read and write HDFS. Therefore, Spark is better suited for iterative map reduce algorithms such as data mining and machine learning. Spark distributed algorithm has been widely used in the Internet, advertising, finance and other industries, but less in the power industry. At present, the research on market power risk of power generators has been very in-depth, but there is little research on simulation platform algorithms for identifying market power risk. This article proposes a market power identification platform for power generation enterprises based on the Spark distributed framework, which can effectively accelerate the calculation speed of market power indicators. Furthermore, using a certain regional power grid as a simulation example, the monopoly situation of various power generation enterprises and the entire wholesale market is analyzed, proving the effectiveness of the Spark distributed framework algorithm. As the number of grid nodes increases, the advantages of the distributed algorithm will become more prominent.