Hybrid net powered large scale audit of dataset licensing and attribution practices for enhanced transparency and compliance

In the era of data-driven research and artificial intelligence, proper dataset licensing and attribution practices are crucial for legal compliance and ethical data usage. Open-access datasets are often shared under various licensing schemes, such as Creative Commons (CC) or MIT, each with distinct usage and attribution requirements. However, ensuring adherence to these requirements across vast repositories poses a significant challenge. This study presents a HybridNet-powered system combining RoBERTa and InceptionV3 models to audit large-scale dataset licensing and attribution practices for enhanced transparency and legal compliance. The system leverages RoBERTa for natural language processing (NLP) to classify licensing terms and detect attribution requirements in textual metadata, while InceptionV3 handles visual attribution embedded in images. A comprehensive dataset audit was conducted on 5,000 datasets from the OpenML repository, resulting in an overall accuracy of 95% for detecting and classifying licenses and attributions. Cross-validation with OpenML metadata showed 94% consistency in license classification and 90% consistency in attribution detection. License violations were identified in 5% of the datasets, while attribution violations were flagged in 6%, leading to an overall compliance violation rate of 11%. The system’s ensemble approach significantly outperformed traditional models, such as Logistic Regression (accuracy: 72%) and Support Vector Machines (accuracy: 75%), demonstrating its effectiveness in auditing multi-modal dataset content. By flagging datasets with missing or inconsistent license and attribution information, the system enables corrective action, improving the transparency of dataset repositories.