From singing in the rain to tears in the rain: socio-demographic trends and pessimism during new Hollywood

New Hollywood (NH) was a transformative period in American cinema from 1967 to 1982, marked by a shift towards more experimental, morally ambiguous, and pessimistic storytelling. Film studies surmise these shifts were influenced by socio-economic challenges such as declining movie revenues, stagnating income, rising distrust in institutions, and increased college-educated audiences. However, it is unclear whether these factors generally lead to ambiguous and pessimistic storytelling. In this study, we use Natural Language Processing (NLP) to quantify psychological dimensions of pessimism, ambiguity, stress, and negativity across 5948 movie scripts spanning 1950–2000 and comprising the periods NH (1967–1982), pre-NH (<1967), and post-NH (>1982). We contrasted the dynamics of these psychological dimensions across periods and assessed the temporal precedence of socio-economic variables, such as movie revenues, median income, trust in institutions, and college education, using lagged regression, cross-correlation, and network analysis. This confirmed, first, that pessimism, stress, and ambiguity increased during NH, while pessimism and ambiguity decreased afterward; negativity, on the other hand, consistently rose throughout all periods. Second, declining trust in institutions and movie revenues predicted higher levels of pessimism and ambiguity; moreover, higher education correlated with lower pessimism and ambiguity, but higher negativity and stress. Third, median income and income growth had complex effects: pessimism, negativity, and stress correlated with a decline in median income growth but a rise in median income. Finally, a decline in income growth temporally preceded a significant increase in pessimism, suggesting a lagged effect of economic downturns on cultural expression. The study provides a framework for understanding how socio-economic conditions influence cinematic content, emphasizing the complex interplay between material conditions and cultural outputs. Our findings confirm that declining trust and income growth can contribute to the rise of a cultural zeitgeist of pessimism and ambiguity.