Content Safety and Response Quality in LLMs: A Data-Centric Refinement Approach

Recent advancements in large language models (LLMs) have significantly impacted natural language processing. However, ensuring the safety and quality of responses generated by LLMs remains a challenge. Building on previous work with a Corrective-BART Model, which demonstrated significant reductions in toxicity, this paper addresses the trade-of between safety and response quality. A data-centric refinement paradigm is introduced, proactively generating high-quality, safe responses during training. A dynamic dataset is curated using Llama2, where toxic prompt-response pairs are contextually regenerated into safe, relevant alternatives. The enhanced Corrective-BART model employs a multi-threshold correction pipeline, leveraging multiple metrics to detect implicit and explicit harms. For the Type-Token Ratio, the enhanced model paired with GPT-4 achieves an 11% improvement. Similarly, improvements of 8.2% and 8.6% are observed when paired with Gemma-2b-it and Mistral-7B, respectively. In terms of Readability Score, the enhanced BART model paired with GPT-4 shows an 8% improvement while demonstrating a 21% improvement when paired with Mistral-7B and an 8.4% improvement with Gemma-2b-it. For Coherence Score, the enhanced BART model achieves a 6% improvement when paired with GPT-4, a 7% improvement with Mistral-7B, and a 12.7% improvement with Gemma-2b-it. Notably, the Refusal Rate exhibits a 23.4% improvement when the enhanced BART model is paired with GPT-4. Furthermore, impressive increases of 10.5× and 21.3× are observed when paired with Mistral-7B and Gemma-2b-it, respectively. These results demonstrate that the enhanced BART consistently enhances performance across all LLMs, improving lexical diversity, readability, coherence, and safety.