Development of a brief test protocol for assessing cognitive change in the peri‐operative period in older adults undergoing elective surgery

Many patients are afraid of permanent cognitive impairment after general anaesthesia [1]. However, there is no uniform method to diagnose postoperative neurocognitive disorders extending beyond 1 month after surgery [2]. The gold standard is a time-consuming neuropsychological assessment [3]. We aimed to identify and validate internally the best predictive subset of neuropsychological assessment tests, balancing accuracy and utility suitable for both research and clinical purposes. This study used data from a previous study [3]. After ethical approval and informed consent, patients aged ≥ 65 y undergoing elective surgery were enrolled. Patients with hearing impairment, multiple procedures under anaesthesia or pre-existent cognitive impairment were not studied. Neuropsychological assessment was done pre-operatively and 4–8 weeks postoperatively. It covered five cognitive domains, with outcomes for 17 individual subtests and four combined scores (online Supporting Information Appendix S1). Outcomes were reported in T-scores, corrected for age and educational level compared with a healthy Dutch patient group [4]. Missing outcomes were imputed. Our previous study used a definition based on composite cognitive domain scores, yielding an 18% incidence of postoperative neurocognitive disorder [3]. However, as predictors would be based on the results of individual tests, we defined postoperative neurocognitive disorder in this analysis as a decline of ≥ 1 SD on ≥ two tests in at least one cognitive domain, or a decline of ≥ 1 SD in total cognitive domain score, aligning with the most used research definition [5]. After checking for multicollinearity, we aimed to create a logistic regression model (online Supporting Information Appendix S2). All 17 individual delta test scores were included, and backward selection based on the Akaike Information Criterion was performed, using different cut-offs. When backward selection included the middle or last subtest from a test, we deemed it necessary to also include the other subtest (e.g. Stroop 1, 2 and 3). When this would make the test protocol too time-consuming, we selected the tests with the lowest p values and their subtests, while also checking AUROC, so time of administration would be acceptable. Internal validation was done using bootstrapping, where AUROC and regression coefficients were corrected uniformly for measured optimism. Calibration was assessed and thresholds determined. Predictions were made using the formula 1 1 + e − β 0 + β 1 * x 1 + … $$ \frac{1}{1+{e}^{-\left({\beta}_0+{\beta_1}^{\ast }x1+\dots \right)}} $$ . Sensitivity and specificity of the final selected subset were tested against the composite domain definition. Full statistical analyses can be found in online Supporting Information Appendix S3. In total, 77 patients had complete data and all baseline characteristics can be found in online Supporting Information Appendix S4. Neuropsychological assessment diagnosed 32 (42%) patients with postoperative neurocognitive disorder. After backward selection the selected test protocol was approximately 30 min and deemed too time-consuming (Table 1). Therefore, we included the three tests with the lowest p values, as this gave the best balance between administration time and AUROC. The final model included the Digit Span Test (forward, backward and sorting), Stroop (1, 2, 3), and Auditory Verbal Learning Test with an approximate duration of 20–24 min and yielded an AUROC of 0.91 (95%CI 0.83–0.98) before and 0.86 (95%CI 0.77–0.97) after internal validation (Table 1). Entering the delta T-scores into the model gave a predicted probability on the presence of postoperative neurocognitive disorder. Calibration is shown in Fig. 1. Thresholds and comparison with the other definition can be found in online Supporting Information Appendix S5. We believe we have identified a short test protocol with good properties providing a promising alternative for the current neuropsychological assessment, which can be administered before and after surgery. Interestingly, we found a postoperative neurocognitive disorder incidence of 42%, which is relatively high [6, 7]. This might be attributed to the inclusion of patients undergoing cardiac surgery and the definition used in this study. This again emphasises the need for a uniform definition for postoperative neurocognitive disorder and the corresponding diagnostic tool. Limitations include the small sample size which might have led to possible overfitting of the data, and part of this study was conducted during the COVID-19 pandemic which may have led to a higher dropout rate as procedures were cancelled in our vulnerable population. However, this does not appear to be an expression of underlying frailty, as dropouts did not have lower neuropsychological assessment scores or higher ASA physical status classifications. In conclusion, this short test protocol is a promising alternative to the extensive neuropsychological assessment. This is initially of interest for research, and we recommend future studies to externally validate these results. Consensus on the precise definition of postoperative neurocognitive disorder and the recommended test protocol should be generated in a multidisciplinary working group. The authors would like to thank Rik van Eekelen, statistician, Amsterdam UMC, for his advice on the development of the model. MvZ is a board member of the Dutch Association of Anaesthesiology Data and code will be made available upon reasonable request to the corresponding author. This work was supported by the Amsterdam University Fund. No competing interests were declared. Appendix S1. Composition of the neuropsychological assessment. Appendix S2. Correlations between delta test scores of all 21 individual tests of the neuropsychological assessment. Appendix S3. Statistical methods. Appendix S4. Baseline characteristics of included patients. Appendix S5. Sensitivity and specificity of the final short test battery per threshold and comparison with other definition of postoperative neurocognitive disorders. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.