AI Tools and Text Embedding for Session Organization at ASABE's Annual International Meeting

Highlights Text embedding models enable automated determination of the similarity between presentations and sessions. Procedures to use metrics based on similarity scores to identify focused and unfocused sessions and outlier presentations are provided. Model clustering ability is more critical for this application than the maximum length of input text. ABSTRACT. Four text embedding models (nomic-embed-text-v1.5, cde-small-v1, all-mpnet-base-v2, and all-MiniLM-L6-v2) were used to create dense vector representations of the topics of the titles and abstracts of all 810 oral presentations that were given at the American Society of Agricultural and Biological Engineers’ 2024 Annual International Meeting. Similarities between all these presentations were then calculated through cosine similarity. Metrics for characterizing presentations and their relationship to their assigned session were created: Presentation Session Fit, Presentation Raw Deviation, and Presentation Standardized Deviation. To describe the level of similarity within a session, session metrics were established: Session Coherence and Session Standard Deviation. Examples of using these metrics to identify very focused sessions as well as unfocused sessions are provided. Outlier presentations are identified using the Presentation Standardized Deviation and potentially more suitable sessions for the presentation are identified using similarity scores. A final metric, Session-Session Similarity identifies the similarity in topics between different sessions, which can assist in scheduling to ensure highly related sessions are not scheduled at the same time. The outputs of the four different models were compared through Pearson correlations of the similarity scores, which ranged between 0.79 and 0.88. Since the models differed on the maximum quantity of input text and thus the length of the abstracts that they could process, all models were again tested with inputs truncated to the first 256 tokens. These input limits only had slight impacts on outputs with Pearson correlations above 0.96 between truncated and non-truncated tests of the same model. The differences in clustering quality as measured by model benchmark scores were more impactful than the length of the abstract that the model could process. Based on these results, efforts will continue with the more powerful embedding models, and a web app has been built that will enable session organizers to consider similarity metrics as they organize AIM 2025. Keywords: Artificial intelligence, ASABE annual international meeting, Natural language processing, Session organization, Similarity metrics, Text embedding.