Deep Learning voor mens-computercommunicatie

With the rapid advancement of artificial intelligence (AI) technologies, machines are increasingly becoming intelligent assistants that enhance productivity across various human activities. In this context, human computer communication has gained both theoretical importance and practical value. Early chatbot systems were typically based on a modular pipeline architecture, including components for information extraction, intent detection, dialogue policy, and response generation. More recently, end-to-end systems powered by large language models (LLMs) have become dominant. Beyond functional performance, the underlying infrastructure, such as web load balancing to ensure reliable and scalable user interactions, also plays a critical role. This thesis addresses both the model-level functionalities and the infrastructure-level challenges in human-computer communication. Specifically, it tackles: (1) improving dialogue policy strategies, (2) effective information extraction under data scarcity, (3) fine-grained intent detection, (4) enhancing LLM reasoning under non-ideal conditions, and (5) enabling scalable load balancing under high user traffic. We present systematic experiments exploring how advanced deep learning techniques can improve the aforementioned components, analyze limitations in LLM reasoning, and develop infrastructure-aware solutions for large-scale deployment. First, we address the overestimation bias in reinforcement learning-based dialogue policy optimization. We propose a dynamic partial average estimator (DPAV) that computes a weighted average between predicted maximum and minimum action values, where the weights adapt dynamically based on the task. DPAV is integrated into a deep Q-network and theoretically shown to offer provable convergence and tighter bounds on estimation bias compared to existing methods. Second, we propose a few-shot named entity recognition (NER) pipeline that improves information extraction in low-resource scenarios. The method includes a steppingstone span detector pretrained on open-domain Wikipedia data, which reduces redundant feature learning. In addition, an LLM is leveraged to generate reliable type referents, eliminating the sample dependency problem. Third, we develop a clustering method for fine-grained intent detection. The method uses semantic similarities in a logarithmic space to guide sample distributions in the Euclidean space and to form distinct clusters that represent fine-grained categories. A centroid-based inference mechanism is introduced to support real-time applications. Fourth, we introduce and empirically explore a new research direction, evaluating LLM reasoning under the summarizing inference scenario. We fine-tune three state-of-the-art LLMs and a large vision-language model using a policy gradient algorithm and evaluate them on five benchmark datasets. Despite improvements in the setting without summarizing information, model performance degrades significantly in summarizing inference, highlighting limitations in advanced reasoning. We further propose a training method that explicitly trains models to consider multiple possible outcomes during learning. Finally, inspired by causal dynamics in ecological systems, we investigate causal dependencies among web services to improve load balancing. We propose CCMPlus, a neural module that extracts causal relationships between services and integrates seamlessly with time series models to enhance traffic prediction. We also provide theoretical explanation that the generated causal correlation matrix captures causal relationships among services. In summary, this thesis presents a series of contributions that advance the capabilities of deep learning models in human-computer communication. By combining reinforcement learning, few-shot learning, contrastive learning, LLM reasoning and time series analysis, our work provides both methodological innovations and practical solutions validated through extensive experimentation.