Pinning down wording effects: How NLP can separate valence from content in personality-wellbeing research

The positivity or negativity of item wordings, or ‘item valence’, is a potential confounder for research into Personality and Wellbeing, but one that might be fixable without major disruptions. We analysed item text using computational methods (NLP) to tackle a known problem in this line of research: how to separate substantive item content from item valence. We calculated item valence scores for 454 regular-keyed items from Subjective Wellbeing instruments and Personality inventories (Big Five/Big Six) by combining several existing item valence dictionaries. These dictionaries are based on human-derived word positivity and negativity ratings. We compared the different Personality traits and Wellbeing variables' item valence scores using Kruskal-Wallis tests and Exploratory Graph Analysis (Golino & Epskamp, 2017). Our findings indicated Neuroticism items are substantially more negatively valenced than other personality trait items and Positive Wellbeing items. Valence network communities revealed strong similarities between Extraversion, Openness, and Positive Wellbeing items. We argue our word-by-word approach primarily measures item valence wording similarities, rather than item content (semantic similarities). Our methodological contribution can be used to advance research clarifying what shared variance is due to item valence, as opposed to other biases or substantive relationships. This method could eventually be used to reduce item valence effects via targeted score adjustments, without the need to re-write items.