Assessing Subjective Probabilistic Expectations in Household Surveys with Audio Records

Probabilistic questions about future income or housing price are increasingly used in household surveys. However, there is still much to learn about the elicitation process itself. Do households fully understand this kind of questions? Do interviewers comply with protocols, so that potential biases are minimized during the expectations elicitation process? One interesting source of information to shed light on the measurement process is provided by audio records. In this paper, we develop a state-of-the-art audio transcription machine learning pipeline to process audios recorded in the Spanish Survey of Household Finances (EFF) for a subjective probabilistic expectations question on housing prices. From these data, we measure the incidence of interviewers’ compliance with protocols and respondents’ difficulties. Our results show that the incidence of deviations from protocols such as the verbatim reading of questions and non-neutral probing is non-negligible. Besides, the incidence of respondents asking for clarifications or being reminded that points should be add up to ten is also high. In addition to the heterogeneity of these measures by demographic groups, we also document how they are associated to different measures of future house prices uncertainty observed in the EFF data. In particular, speech rate, verbatim reading and non-neutral probing seem to have substantial effect on the incidence of those responses. Finally, we document the in person mode and panel conditioning effect on such uncertainty measures, being the first positive and significant once we control for the aforementioned extracted audio characteristics.