Research on the Application of Big Data Technology in the Field of Coal Mining

With the in-depth application of various intelligent sensors in the coal mine, although the coal mining industry is gradually transforming from automation to intelligence, a large amount of operational monitoring data is generated in the production process, and the total amount of data is still on a growing trend every year, and the traditional decentralized data management system can no longer satisfy the data storage and processing requirements under the condition of big data due to the limitation of computing resources. To address the above issues, big data technology is applied to coal mine data management system, develop coal mine intelligent big data management system, and utilize big data frameworks such as Hadoop and Spark to build a distributed parallel processing environment, which constitutes a computing cluster with a unified resource scheduling of dispersed computing resources, and makes full use of idle storage resources and computing resources. Firstly, the data collected by various subsystems in the underground coal mine are standardized, and the data of various subsystems are unified and integrated, and then the collected data are analyzed and utilized using big data processing technology, and the invalid data and abnormal data are compressed, cleaned and stored hierarchically according to the access timeliness, so as to expand the horizontal and vertical storage capacity of the current decentralized data management system in the coal mine. The actual application effect shows that applying big data technology to the coal mine management system can integrate the scattered data resources of the coal mine and improve the reading and writing ability and processing ability of the data.