Risk-adaptive transaction security framework integrating hybrid anomaly detection, post-quantum cryptography and permissioned blockchain

As trust, auditability, and the long-term security of cryptographic mechanisms become increasingly critical, modern digital payment systems are adopting intelligent decision-making techniques for fraud detection under stringent latency and reliability constraints. However, existing fraud detection approaches primarily focus on improving classification accuracy within isolated systems, relying on traditional security rules, basic cryptographic schemes, and centralized authentication mechanisms. Such approaches are insufficient to address evolving fraud patterns, holistic system trust requirements, and emerging post-quantum threats. To address these limitations, this work proposes an end-to-end, risk-adaptive transaction security framework that reconceptualizes fraud detection as a dynamic and continuous security process rather than a stand-alone prediction task. The framework introduces Hybrid Anomaly-Guided Adaptive Detection (HAGAD), which integrates supervised fraud prediction, unsupervised anomaly detection, and temporal behavior analysis to generate a continuous risk score. This risk score dynamically governs Adaptive Key Ratcheting (AKR), risk-triggered association quorums, and fault-tolerant post-quantum threshold signing, while selectively employing a permission blockchain for high-risk transactions. Furthermore, the system incorporates an Adaptive Assurance Scoring (AAS) mechanism to evaluate trustworthiness beyond conventional accuracy-based metrics. Experimental results demonstrate that the proposed approach achieves high reliability with balanced precision and recall, while activating enhanced security mechanisms only when necessitated by transaction risk. This adaptive enforcement ensures robust decision validation without introducing additional overhead for low-risk transactions. Overall, the proposed framework establishes an integrated, risk-adaptive transaction security pipeline that unifies intelligent detection, advanced cryptography, auditability, and assurance evaluation, providing a resilient and future-ready foundation for next-generation electronic payment systems.