Prompt Refinement or Fine-tuning? Best Practices for using LLMs in Computational Social Science Tasks

This paper presents PROTECT, a novel policydriven organizational value taxonomy designed to enhance ethical compliance and trust within organizations.Drawing on established human value systems and leveraging large language models, PROTECT generates values tailored to organizational contexts and clusters them into a refined taxonomy.This taxonomy serves as the basis for creating a comprehensive dataset of compliance scenarios, each linked to specific values and paired with both compliant and non-compliant responses.By systematically varying value emphasis, we illustrate how different LLM personas emerge, reflecting diverse compliance behaviors.The dataset, directly grounded in the taxonomy, enables consistent evaluation and training of LLMs on value-sensitive tasks.While PROTECT offers a robust foundation for aligning AI systems with organizational standards, our experiments also reveal current limitations in model accuracy, highlighting the need for further improvements.Together, the taxonomy and dataset represent complementary, foundational contributions toward value-aligned AI in organizational settings.This work provides a practical framework for companies to ensure that AI systems, especially LLMs, align with their policies and values, contributing to a more secure, ethical, and compliant organizational environment. Related WorkVarious theoretical frameworks have been developed to categorize human individual values and to explain the underlying motivations driving human actions.The Schwartz Theory of Basic Human Values (Schwartz, 2012) organizes 10 basic human values into four high-order categories: openness to change, conservation, self-enhancement, and self-transcendence, providing a comprehensive framework for understanding value-based motivations.The Moral Foundation Theory (Graham et al., 2013) offers a complementary perspective through five fundamental moral dimensions: care/harm, fairness/cheating, loyalty/betrayal, authority/subversion, and sanctity/degradation. Traditional frameworks such as Rokeach Values (Rokeach, 1967) distinguish terminal values as desired life goals (e.g., happiness, freedom) and instrumental values as preferred behaviors (e.g., honesty, ambition) to achieve those goals and contribute additional perspectives on value classification.However, unlike basic human values, organizational values have received less attention as often the focus is towards policy and operational guidelines.In the following subsections, we provide a 1 https://github.com/AvniMittal13/PROTECT