CORPUS OF TEXTS FROM PUBLIC CHANNELS OF THE TELEGRAM MESSENGER: CREATION AND PRACTICAL APPLICATION

The article presents the process of creating and practical applications of a text corpus based on public channels of the Telegram messenger. The aim of this study is to develop a corpus of texts from public Ukrainian-language Telegram channels. The research objectives are as follows: to develop a methodology for building a text corpus from public Telegram channels, to create and test scripts for the automated collection, cleaning, and analysis of textual data, to upload the processed texts into a corpus manager, and to explore the potential for further applications and improvements of the corpus. To achieve these objectives, automated methods for text collection and cleaning were employed using Python scripts and text processing libraries (e.g., json, pandas, SpaCy). The Sketch Engine platform was utilized for the creation and management of the corpus. Data analysis involved the use of CQL (Corpus Query Language) search queries, regular expressions, and basic statistical analysis. The source data consisted of texts from a selected public Ukrainian-language Telegram channel (Serhiy Sternenko’s channel). The corpus development process was conducted in three stages: (1) selecting a public Telegram channel as the source of textual data and downloading its publication history; (2) converting the downloaded data into a format compatible with specialized corpus creation and management software; and (3) uploading the pre-processed texts into the chosen corpus manager. Two Python scripts were developed to prepare the data. The first script extracted the text of posts and saved them to a separate file, while the second script performed text cleaning and statistical analysis. The cleaning process involved the removal of unnecessary symbols, emojis, and links. To illustrate the potential applications of the corpus in online discourse research, several queries were performed: searching for verbs exceeding 15 characters in length, identifying hashtags, extracting proper names, and analyzing attributive phrases. The queries for attributive phrases focused on two patterns: adjective-noun combinations denoting proper names or entities, and phrases matching the structure ‘so-called + noun’ or ‘so-called + adjective + noun.’ These queries were formulated using CQL and regular expressions. The findings indicate that the created corpus is a valuable resource for studying online discourse. Future research directions include expanding the corpus by incorporating texts from additional Telegram channels, refining the text preparation and filtering processes, and evaluating the applicability of alternative software solutions for corpus creation and management. This research contributes to the field of computational linguistics and offers a novel resource for the analysis of Ukrainian-language online discourse.

Authors

Publication Details

Journal
Odessa National University Herald Series Philology
Published
2024-12-27
DOI
https://doi.org/10.18524/2307-8332.2024.2(30).320406
Primary Topic
Natural Language Processing Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

CORPUS OF TEXTS FROM PUBLIC CHANNELS OF THE TELEGRAM MESSENGER: CREATION AND PRACTICAL APPLICATION

Mariia Malysheva
Odessa National University Herald Series Philology
Natural Language Processing Techniques
article

CORPUS OF TEXTS FROM PUBLIC CHANNELS OF THE TELEGRAM MESSENGER: CREATION AND PRACTICAL APPLICATION

Mariia Malysheva
article en

Abstract

The article presents the process of creating and practical applications of a text corpus based on public channels of the Telegram messenger. The aim of this study is to develop a corpus of texts from public Ukrainian-language Telegram channels. The research objectives are as follows: to develop a methodology for building a text corpus from public Telegram channels, to create and test scripts for the automated collection, cleaning, and analysis of textual data, to upload the processed texts into a corpus manager, and to explore the potential for further applications and improvements of the corpus. To achieve these objectives, automated methods for text collection and cleaning were employed using Python scripts and text processing libraries (e.g., json, pandas, SpaCy). The Sketch Engine platform was utilized for the creation and management of the corpus. Data analysis involved the use of CQL (Corpus Query Language) search queries, regular expressions, and basic statistical analysis. The source data consisted of texts from a selected public Ukrainian-language Telegram channel (Serhiy Sternenko’s channel). The corpus development process was conducted in three stages: (1) selecting a public Telegram channel as the source of textual data and downloading its publication history; (2) converting the downloaded data into a format compatible with specialized corpus creation and management software; and (3) uploading the pre-processed texts into the chosen corpus manager. Two Python scripts were developed to prepare the data. The first script extracted the text of posts and saved them to a separate file, while the second script performed text cleaning and statistical analysis. The cleaning process involved the removal of unnecessary symbols, emojis, and links. To illustrate the potential applications of the corpus in online discourse research, several queries were performed: searching for verbs exceeding 15 characters in length, identifying hashtags, extracting proper names, and analyzing attributive phrases. The queries for attributive phrases focused on two patterns: adjective-noun combinations denoting proper names or entities, and phrases matching the structure ‘so-called + noun’ or ‘so-called + adjective + noun.’ These queries were formulated using CQL and regular expressions. The findings indicate that the created corpus is a valuable resource for studying online discourse. Future research directions include expanding the corpus by incorporating texts from additional Telegram channels, refining the text preparation and filtering processes, and evaluating the applicability of alternative software solutions for corpus creation and management. This research contributes to the field of computational linguistics and offers a novel resource for the analysis of Ukrainian-language online discourse.

Odessa National University Herald Series PhilologyVol. 29(2(30))
Openalex Percentile: Top 74%
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.