Contextual Semantic Text and Image Annotation in the MARKUS Environment

In this article we discuss the rationale behind the design of COMARKUS and IMMARKUS, two browser-based annotation services for the creation of complex semantic text- and image annotations, in the form of both individual entities or tags with properties and clusters of entities or tags also with their own set of properties. We first discuss the challenges we faced with existing text and image annotation platforms, using as our case a longue-durée social history of Chinese material infrastructures including city walls, roads, and bridges. We explain how methodological design and redesign constituted an urgent need issuing from both theoretical and empirical findings. In the other two sections we set out how COMARKUS and IMMARKUS were designed to address those challenges and facilitate an event- or cluster-based cross-media history of infrastructures. Throughout we aim to illustrate the more general use of COMARKUS and IMMARKUS annotation methods and show how annotation results can be visualized and analyzed in the cross-media information retrieval platform X-MARKUS and spatial analysis service MUNDa, or exported for analysis in a broad range of data analytical environments. We also discuss how machine learning and Generative AI are included in these services and can be made part of a humanities research flow aimed at the contextualization of research data. We see this methodological design as part of a broader effort to face the challenge of the contextualization of data, focusing on the semantic modeling of image and text sources, the traceability of image regions and textual entity and data cluster locations, and the uses of piece and source metadata in data analytics.