From analysis-by-synthesis to predictive coding and beyond: The Martingale Dynamics of Human Speech

Ken Stevens is best remembered for his seminal contributions in developing theories of speech perception based on detection of acoustic features emerging from quantal properties of speech. Equally influential, however, is his earlier proposal for speech perception through analysis-by-synthesis, whereby hypotheses generated by an internal model of speech production are evaluated against sensory input. These ideas are recapitulated in recent theories of predictive coding, which also rely on monitoring discrepancy between simulations of potential action and actual sensation. Analysis-by-synthesis and invariant feature detection are often considered as alternative top-down and bottom-up approaches to explaining perception. Both viewpoints can, however, be reconciled within mathematical theories of optimal estimation, which offer two alternative, equivalent solutions to the problem of extracting information about the state of a dynamical system from partial observations. Both solutions rely on internal models, but one involves calculating the prediction error, as in analysis-by-synthesis, whereas the other involves calculating the relative salience of the sensory field, akin to landmark detection. This presentation provides a historical overview of analysis-by-synthesis, from cybernetics to predictive coding, and also reviews parallel developments in martingale dynamics that unite analysis-by-synthesis with landmark detection.