Machine learning based classification and regression approaches for return and risk prediction in the stock market using fundamental feature engineering

Portfolio management relies on accurate stock prediction. While fundamental analysis is crucial for long-term investing, it is often underemphasized, and the importance of feature engineering and leveraging temporal dependencies within company data has been further overlooked. To enhance prediction efficiency and effectiveness, feature engineering and lightweight models are critical. This work introduces a new stock dataset with fundamental features. We analyze the significance of feature engineering by evaluating sample-based (treating each company’s time-stamped reports as independent samples) versus trend-based (modeling each company’s sequential reports as a time-series to capture temporal trends) approaches. Aiming for effective and efficient predictions, we propose ML/DL models categorized into four groups: sample-based classification, trend-based classification, sample-based regression, and trend-based regression, designed to predict security return and risk. We conduct extensive experiments, including feature engineering, ranking, parameter tuning, and model evaluation. Notably, our lightweight models outperform state-of-the-art heavy attention-based, deep LSTM architecture while underscoring the importance of feature engineering—particularly trend-based techniques—for forecasting return and risk, demonstrating practical efficiency. Additionally, distinct feature sets are needed to effectively predict returns versus risk.