Domain-Adapted Distilbert for Semantic Representation of Bibliographic Records on Library and Scopus Datasets

Bibliographic records incorporate conventional language (text) and structured data, with the domains of all the text fields involving vocabulary and norm conventions. General-purpose pretrained language models might capture some more general regularities of language while missing out on terminology occurring in library catalogues and scholarly-indexing records. In this paper, we introduce Domain-Adapted DistilBERT (DA-DistilBERT), a domain-adaptive text-representation method which further continues masked-language-model pretraining based on two different sources of corpus fields, a self-collected Library Dataset and a Scopus Dataset. In the Library corpus, titles, author strings and call numbers were utilized and in the Scopus corpus, titles and publisher names were utilized. The contextual representations are obtained from the candidate Transformer layers and scores are taken by using Cosine similarity and Spearman's rank correlation. According to Synopsis-reported results, for Library Dataset, DA-DistilBERT scores a cosine similarity of 0.934 and a Spearmen-ρ score of 0.912 while for Scopus Dataset, the scores of DA-DistilBERT is a cosine similarity of 0.921 and a Spearmen-ρ score of 0.896. These values exceed the corresponding reported scores for Vanilla BERT (0.891/0.856 and 0.874/0.831) and FakeBERT (0.913/0.883 and 0.899/0.862). The absolute gain in cosine similarity with Vanilla BERT are 0.043 and 0.047 and the absolute gain in Spearman ρ are 0.056 and 0.065 for the Library and Scopus data sets respectively. The results show the potential of using domain adaptation for representation of bibliographic works in the given evaluation protocol. However, the provided synopsis does not give out the corpus size, training hyperparameters, manner in building the evaluation-pair and specification on confidence intervals and repeated-runs statistics are not provided, which are required for independent replications, and statistical interpretations. It gives the mathematical formation of the same and a broadcasting technique that can be copied without the addition of any missing trial information.

Authors

Institutions

Publication Details

Journal
International Journal of Innovative Research in Engineering
Published
2026-11-13
DOI
https://doi.org/10.59256/ijire.20260706008
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Domain-Adapted Distilbert for Semantic Representation of Bibliographic Records on Library and Scopus Datasets

Kavitha V, Geetha J
International Journal of Innovative Research in Engineering
Topic Modeling
article

Domain-Adapted Distilbert for Semantic Representation of Bibliographic Records on Library and Scopus Datasets

Kavitha V, Geetha J
article en

Abstract

Bibliographic records incorporate conventional language (text) and structured data, with the domains of all the text fields involving vocabulary and norm conventions. General-purpose pretrained language models might capture some more general regularities of language while missing out on terminology occurring in library catalogues and scholarly-indexing records. In this paper, we introduce Domain-Adapted DistilBERT (DA-DistilBERT), a domain-adaptive text-representation method which further continues masked-language-model pretraining based on two different sources of corpus fields, a self-collected Library Dataset and a Scopus Dataset. In the Library corpus, titles, author strings and call numbers were utilized and in the Scopus corpus, titles and publisher names were utilized. The contextual representations are obtained from the candidate Transformer layers and scores are taken by using Cosine similarity and Spearman's rank correlation. According to Synopsis-reported results, for Library Dataset, DA-DistilBERT scores a cosine similarity of 0.934 and a Spearmen-ρ score of 0.912 while for Scopus Dataset, the scores of DA-DistilBERT is a cosine similarity of 0.921 and a Spearmen-ρ score of 0.896. These values exceed the corresponding reported scores for Vanilla BERT (0.891/0.856 and 0.874/0.831) and FakeBERT (0.913/0.883 and 0.899/0.862). The absolute gain in cosine similarity with Vanilla BERT are 0.043 and 0.047 and the absolute gain in Spearman ρ are 0.056 and 0.065 for the Library and Scopus data sets respectively. The results show the potential of using domain adaptation for representation of bibliographic works in the given evaluation protocol. However, the provided synopsis does not give out the corpus size, training hyperparameters, manner in building the evaluation-pair and specification on confidence intervals and repeated-runs statistics are not provided, which are required for independent replications, and statistical interpretations. It gives the mathematical formation of the same and a broadcasting technique that can be copied without the addition of any missing trial information.

International Journal of Innovative Research in Engineering
Sri Ramakrishna College of Arts and Science (IN)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon