Beyond Words: Memetic Theory as a Lens to Expose Ideological Bias in Language Models

This study builds upon existing research examining ideological propagation and the integration of memetic theories within Language Models (LMs). We investigated four memetic hypotheses in LMs - competition among ‘selfish memes’, the presence of memeplexes, the varying susceptibility of agents, and the influence of human guardrails. We fine-tuned twelve LMs on the MediaNLP dataset, derived from GPT-2 and DistilBERT base models, across thematic subsets of religion, politics, and opinions on AI. Subsequently, we conducted seven experiments in an attempt to disprove three null hypotheses. These experiments included measuring the variation between the models and human ground truth, evaluating the effects of exposure to AI-generated content using epidemiological modeling, analyzing cosine distances among key words in LM embeddings, applying cryptographic benchmarks, and scrutinizing changes in model weights and biases. We disproved Null Hypothesis I, but did not disprove Null Hypotheses II and III. Furthermore, our findings underscore the significant role of human designers in shaping the guardrails of ChatGPT, particularly in generating data containing potentially ‘sensitive’ content. This study represents a novel investigation in applying memetic theory to highlight potential unexplored risks and vulnerabilities inherent in LMs and their output.