Design and Implementation of a Spark Based Product Recommendation System

In order to improve the accuracy of platform product push and increase marketing revenue, the author proposes a product recommendation system based on Spark. The author adopted a hybrid recommendation method based on Spark memory computing model, combining alternating least squares (ALS) algorithm and item based collaborative filtering algorithm to train multiple models and select the candidate set of the optimal model. Designed and implemented a Spark based product recommendation system. Spark is used for distributed computing, covering various recommendation functions such as offline recommendation, real-time recommendation, and popular recommendation. Through experimental analysis from multiple dimensions, the completeness of system functionality and excellent performance have been verified. The results indicate that in system testing, when the concurrency is below 500, the corresponding time of the system is less than 1 second, which is sufficient to meet the required corresponding time. The throughput of the system will gradually increase with the increase of concurrency, but when the concurrency scale increases from 600 to 700, the throughput of the system does not increase, but instead decreases. Therefore, it can be inferred that the reasonable concurrency of the system should be within the range of 400-500. Basically, it can ensure the online performance of the system. Conclusion: The recommendation algorithm designed by the author has been applied in the system and meets the requirements of various functions, which has certain reference value for the design and development of product recommendation systems.