Unveiling the Secrets of Embeddings: Does the Importance of Agricultural Terms Relate to the Context They Occur In?

Advances in language modeling have provided affordances for term extraction based on the capture of the lexical context and its semantics and encoding this in real-valued vectors (embeddings). Term importance is usually computed using quantitative measures, ignoring the semantic context. Until now, there has been limited or no research on the effect of context on term importance using machine learning methods. In this paper, we investigate whether there is a relation between the importance of agricultural terms and the context of their occurrence as represented by text embeddings. Using a dataset of almost 33.7 K AGRIS abstracts containing 50 concepts randomly extracted from AGROVOC, we computed the correlation between the concept tf-idf scores and each of three semantic distances (cosine similarity of embeddings) used as a proxy to context: (i) the semantic distances of the various occurrences of the concept; (ii) the semantic distances between the concept and the abstracts that it appears in; (iii) the semantic distances of the abstracts in which the concept occurs. Embeddings were generated using Agriculture-BERT. We present a methodology and initial results from the computation of correlations. The novelty of our work is in the systematic investigation of the relation between term importance and semantic context.