Librarians at the Center of Peer Review Training: Increasing Collaboration Among Scholarly Communication Stakeholders

My background in scholarly publishing and peer review brought me to the United States to teach Scholarly Communication at the School of Information Sciences of the University of Illinois Urbana-Champaign (UIUC). Offering a course dedicated to the scholarly communication process is a fairly new endeavour for iSchools. In 2022, I was entrusted with a Scholarly Communication course composed of masters students in the Library & Information Science and the PhD in Information Sciences programs. The Scholarly Communication course was created by Prof. Dr. Maria Bonn, who is one of the authors of The Scholarly Communication Notebook (https://oercommons.org/hubs/SCN), ‘an active, inclusive, empowered community of practice for teaching scholarly communication to emerging librarians’. As Professor Bonn used to do with her library science students at the University of Illinois (Bonn 2014), in one of the first class sessions the students and I spent some time browsing job advertisements for Scholarly Communication librarians published on the ALA Job List (https://joblist.ala.org/). In these job advertisements posted from 2006 to 2014, we found positions titled ‘Scholarly Communications Librarian’, or others with a more specific focus such as ‘Copyright Librarian’, ‘Data Librarian’, or ‘Open Access and Intellectual Property Librarian’, and analysed the prevalence of scholarly communications terms, concepts, and activities, as identified by Finlay et al. (2015, 21), namely ‘instruction; digital products; outreach and liaison work; publishing; repositories; copyright, policy, and licensing; preservation; metadata, standards, and data management; and open access’. My students and I were particularly intrigued by the absence of peer review as one of the concepts requested of a scholarly communication position, since peer review is clearly represented in the schematic of the scholarly communication cycle created by the Association of College and Research Libraries (Figure 1, ACRL). Furthermore, the definition of scholarly communication librarianship of the ACRL includes the evaluation of scholarly work in peer-reviewed. journals: ‘the system through which research and other scholarly writings are created, evaluated for quality, disseminated to the scholarly community, and preserved for future use. The system includes both formal means of communication, such as publication in peer-reviewed journals, and informal channels, such as electronic listservs’ (ACRL 2003). In addition, as Meadows (1998, preface) says ‘Communication lies at the heart of research. It is as vital for research as the actual investigation itself, for research cannot properly claim that name until it has been scrutinized and accepted by colleagues’. This statement highlights that peer review is a fundamental aspect of the communication of research in scholarly publishing. Thus, I asked my students: Why is peer review not listed as a required competency for Scholarly Communication Librarians? Do you think that one day we will have a Peer Review Librarian? What can librarians do to support education in peer review? Motivated by these ideas, I (a peer review researcher with a PhD in Information Sciences, but no library science degree) designed and delivered an in-person and online peer review workshop curriculum for the Savvy Researcher workshop series sponsored by the University Library and supported by the School of Information Sciences at UIUC. The Savvy Researcher focuses on a variety of topics related to the lifecycle of research including advanced research and information management topics, for example, introduction to library resources, using tools to perform qualitative and quantitative research, data management, understanding copyright, and much more. All campus affiliates are invited to participate in the synchronous in-person and online sessions, including undergraduate and graduate students, campus staff, researchers, and faculty. Many of the workshops are concept-based around an assortment of scholarly communication topics, while others focus on a particular tool or library resource. The workshops are taught by librarians, graduate students in information science, and several campus partners such as the Center for Innovation in Teaching and Learning, the Writers Workshop, and the UI Press. In this article, I introduce the six-part peer review series ‘Peer Review in Scholarly Journals: History, People, and Models’ delivered at UIUC, and the lessons learned from the experience of conducting them. Finally, my coauthors and I offer recommendations on how peer review training could be improved and discuss the fundamental role of librarians in supporting peer review training. The six-part peer review series ‘Peer Review in Scholarly Journals: History, People, and Models’ was presented as a pilot offering during the fall semester and took place at the University Library from October 11, 2023 to November 15, 2023 (Figure 2). These new workshops were included on the Library's calendar and advertised broadly across campus throughout the fall semester. According to research programs liaison and instruction librarian and associate professor Merinda Kaye Hensley, who supported this initiative, the Peer Review Workshop Series was the first of its kind offered by the University Library. The purpose of the series was to educate a diverse audience and raise awareness regarding the goals and challenges of the peer review process implemented by scholarly journals, including current models and the role of reviewers. Since the sessions were not designed around any specific discipline, myriad examples of peer review were highlighted. The workshops were held for a duration of 60–90 min. Please see the Appendix A for details regarding the content of the workshops. Despite our efforts to promote the peer review workshops and be inclusive by opening registration to any member of the UIUC community, the first set of workshops had low attendance (Figure 3A). In the past 15 years, this is not an unusual occurrence for newly added workshops. Of the 26 total registrations across 6 scheduled sessions, only 10 registrants attended the workshops (Figure 3B). All attendees were graduate students with little to no experience with publishing and/or reviewing papers. The single exception was an editor that reached out directly by email to schedule a one-on-one appointment via Zoom to discuss the last workshop of the series, ‘Understanding and Implementing Open Peer Review.’ However, even with low attendance, offering workshops and sharing their descriptions across campus is a powerful tool to better inform campus stakeholders about the wide variety of topics, including complex scholarly communication issues, where librarians can help them with sharing, presenting, and archiving their scholarly work. We expected to see increased attendance at future workshops, as word-of-mouth spread. We also conducted a second set of non-consecutive sessions in Spring and Fall 2024. These were training sessions for undergraduate students working for an undergraduate research journal. As there was limited interest in the open peer review session previously, this was not conducted. Thus, in the undergraduate sessions, we had a total of 46 attendees split between five sessions (Figure 4). These sessions were better attended compared to the open Savvy Workshop ones. Even for an audience actively engaged as editorial staff and peer reviewers in a journal, interest waned as the specificity of themes progressed, and attendance at the last session was reduced to a third of the attendance of the introductory one. To evaluate the workshops, an auto generated form was sent to all registrants two days after each workshop. The workshop, ‘Reviewers and Their Biases’ was the only session that received feedback from an attendee. In the form, when asked ‘Did the workshop cover the content you expected?’ the attendee replied, ‘Sort of. The workshop helped show how to identify bias, but I was looking for techniques to address bias as well. For example, specific actions or ways I can avoid it when I review papers. Overall the workshop provided a good overview of what bias is and common types of biases during the peer-review process.’ When asked about their preference for the format of workshops, this attendee answered ‘in-person’. Additionally, one faculty member sent an email to the workshop instructor stating that one of their students attended a workshop and as a faculty member they would recommend it to their students. After this first experience in offering a peer review workshop series in an academic library, my coauthors and I reflected on how we could improve this initiative to better serve our campus stakeholders. Our improvement strategies for the pilot series are based on the identification of issues and questions that arose and how they could be addressed, namely by updating workshop content according to information needs of researchers at different career stages, providing additional opportunities for hands-on experience, offering workshops online and asynchronously, inviting scholarly communications stakeholders to speak of their perspective on the peer review process, identifying avenues to engage with undergraduate researchers, and possibly rethinking workshop titles to be more descriptive and actionable. During the workshop series, the workshop instructor observed that it is fundamental to reintroduce attendees to the foundational concepts of the peer review process for each workshop. This introduction contained definitions of the peer review process, the roles of editors, peer reviewers, and authors in relation to the manuscript, and introduction to peer review models, from traditional to open. This was imperative because workshop attendees were different each week, self-selected into the sessions, and the group did not share a common level of knowledge and understanding of the peer review process. Moreover, when showing examples of peer review policies and reviewer guidelines, it was noted that concepts such as originality, validity, and reproducibility were not clear to graduate students, a similar observation as that of Chong (2021, 464). Klebel et al. (2020) found that 75% of the top 100 journals (as per their 5-year h-index metric) surveyed had no clear policy with respect to whether co-reviewing was allowed, whether preprints could be cited, or if reviewer identities were published. This was true in subjects spanning Art, Economics, and Medical Sciences. Additionally, 30% of these journals did not offer clear information on which type of peer review was employed. However, this study was published in 2020 (Klebel et al. 2020), and the prevailing situation may have changed in recent years. After the torrent of retractions in 2023 and fears pertaining to paper mills and the use of generative artificial intelligence (AI), publishers have revised policies and instructions for authors to include ever more detailed declarations, in an effort to promote clarity and guidance. Of the 20 openly available online peer review training identified by Willis et al. (2022), only one had early career researchers as a target audience. Academics engaged with research in teaching of peer review have pointed out that scholarly publishing and peer review are components neglected in undergraduate science education (Otto et al. 2023). Although the instructor's intention was to bring researchers at different career stages for a peer review training and conversation at the library, the workshop design may inadvertently have ignored issues of power imbalance in peer review. In a study of the implementation of peer review curriculum in undergraduate education, one student described considering it safer and more comfortable to share her thoughts on paper with other undergraduate students than in journal clubs with graduate students (Otto et al. 2023). This way, future peer review workshops need to take into consideration not only the overall gaps in peer review education to better meet the information needs of researchers at different stages of their career, but also the competitive, hierarchical, and sometimes oppressive academic environment. Furthermore, we observed that some attendees signed up for a peer review workshop, but asked questions spanning other aspects of scholarly publishing. Some attendees had never heard of open access, others asked about strategies to identify predatory journals, and still others asked where to find a ‘good’ journal to submit a paper. A study conducted by Hurrell et al. (2024) has shown that, in fact, graduate students are interested in learning about scholarly communication topics such as publishing timeline, peer review, journal quality/prestige, appropriate and ethical use of generative AI tools in research and writing, predatory publishers, publishing costs, writing, and disciplinary norms. Taking this into account, the peer review workshops may also provide an invaluable opportunity to advertise other services offered by the Library, and provide insights leading to the creation of other workshops with a focus on the scholarly communication cycle. Some attendees came to the workshop series with the expectation of reviewing a paper. This experience was not offered to them. However, they were provided with an activity to review the peer review reports published by journals guided by the outline for writing a review report created by PLOS (https://plos.org/resource/how-to-write-a-peer-review/). This activity was conceived to explore the educational potential of peer review reports, to encourage discussions about what a good review looks like and the role of an effective reviewer, and to share with attendees typical queries authors receive to improve their manuscripts. During the open discussion portion of the workshop, attendees evaluated this activity positively and were surprised to find that journals have started to publish peer review reports. For the future peer review workshops, we would like to explore partnerships with initiatives such as PREreview—a community and open platform for preprint review—to provide hands-on experience in peer review of preprints. In addition, it might be useful to reach out to journal clubs to see if they are interested in hosting a workshop in partnership with the library. with the opportunity of journal clubs on preprint review A journal is group that to review and and and UIUC has journal clubs in academic from to to of University that could support the library in providing hands-on in peer review. can also advertise available to about the peer review process, such as the and The of the peer review workshops offered to campus stakeholders were It was common to see some attendees because they had or had to We that workshops would be a better for interested in for these In addition, offering the workshops online would of on and A with campus stakeholders would help to about the appropriate and of the workshop. to and promote and learning might be to the workshop content and any available on the would be to the learning an open such as open science or where these could a and be The workshop ‘Understanding and Implementing Open Peer did not have any Although the provided an of each content to campus the instructor observed that attendees of other sessions were not with the peer This peer review was only during the for future we could the we introduce the to campus stakeholders. We to learning where attendees could explore journals that models of peer review, the conversation and to the of peer review related to and Furthermore, the target audience for this workshop of graduate students, editors, and the academic This audience may not be of how open peer review could them. As a the workshop by providing information to campus stakeholders regarding the of open peer review might help to additional interest in the workshop. may not be that peer review reports published they were invited to peer review can the of their peer review work in early career researchers may not that the publication of peer review reports researchers a the of to help them as they to do peer review In this future peer review workshops can on tools such as the Open a for the University of & University College The workshop instructor with only one editor through a scheduled Zoom This editor was interested in learning about open peer review and how to were particularly interested in the current peer review used by the journal, and After the provided information and a of journals from their which are open peer review, the editor will reach out to the editor of to discuss open peer review In this for future workshops, we to offering one-on-one to editors, but also it would be fundamental to promote initiatives where could and with other research et al. et al. on peer review training has also shown the of who are learning about peer review with other in different roles of the academic publishing This way, future peer review workshops in the library may have the potential to promote peer review education but also a between scholarly communication stakeholders to promote of and in the are also in a to teach and support learning in scholarly communication because they can with different campus stakeholders. can identify the information needs of campus stakeholders pertaining to the peer review process and other scholarly communication topics through community In a community of the library and its data be on and and on the librarians can offer peer review workshops at the library, and also identify partners and the at the UIUC would be the Research which undergraduate students with little or no research experience the opportunity to work with graduate students and on their research Offering these peer review workshops to may out of but there is on students in all to share their scholarly work through whether through or Moreover, UIUC undergraduate research journals where undergraduate student researchers can publish their work who are in student journals are more to and show increased knowledge of the research process and In addition, research has for the of peer review in undergraduate science education, how the of undergraduate students in the review of preprint review can and their as researchers (Otto et al. 2023). Furthermore, librarians can also faculty on how to peer review a paper. A study on co-reviewing and of peer review has how much faculty are to their provided by may from a teaching but on the et al. can also support faculty with to teach peer review in the or peer review workshops faculty learning Additionally, librarians can support the process of learning peer review by offering one-on-one the library. with other scholarly communication stakeholders of librarians are to programs and services that can awareness and interest in scholarly publishing. In this we that librarians could partnerships with such as to provide hands-on experience in peer review to campus and to support the of reviewers at any of their the of peer reviewers, one would be to with such as engaged with scholarly communication research also with librarians to improve peer review training by including them in their research process and them roles in peer review training. and librarians can work to design data and training in peer review. This partnership could help to by future workshops as the curriculum in in other how academic of and class that may who is and out and In the of the open educational and Peer that are in an when it to peer review. We may be teaching students what peer review or we may be students techniques on how to identify peer-reviewed We may be or publishing programs that include peer-review In a paper published by ACRL has how information and scholarly communication and has pointed out librarian is an academic is a and roles in an academic library are and by the of scholarly communication and the of the of (ACRL 4). Bonn that communication has a competency for academic librarians’. However, scholarly communication is a competency for librarians, there is of formal training on scholarly communication to include librarians in peer review education and promote future scholarly communication programs include scholarly communication topics into their curriculum and the six-part series ‘Peer Review in Scholarly Journals: History, People, and Models’ we learned that, to better serve campus stakeholders regarding education in peer review, it is to the training to the information needs of researchers and at different stages in their Furthermore, effective peer review workshops in into consideration the time and day to the workshops, their format and It is also to that instruction in an academic is it is including and students with have been to for open access, we they can be to for peer review training as well. we have a Peer Review one how the role of the Peer Review will like and how it will from other scholarly communication librarian We will you with these we programs include scholarly communication topics into their curriculum and librarians to be at the of peer review training and other We researchers and publishers partnerships including librarians in their research and publishing offering and training in peer review based on community and in with scholarly communication stakeholders the librarians have a role in and publishing more inclusive, and do writing writing review and writing writing review and Merinda Kaye writing review and formal Maria formal writing review and The first would like to of Illinois School of Information and Maria Bonn of Illinois School of Information for reviewing the workshop We Maria Bonn for her and on this its In addition, we our peer reviewers for their and on the The authors no of to be of the curriculum and the learning from published research to teaching peer review by early career researchers Peer The and We also such as a community designed to more and to scholarly peer review by supporting and of researchers, particularly at early stages of their career and to review preprints in a process that is to to peer review as of their Open The The and The Review For this pilot series, we content only from The In addition, we several specific from a variety of the of Open Access the A of for content related to peer review, and and and several scholarly publishing and recommendations by scholarly such as the of Science for in or such as the Information for Peer The for of were based on the the of of campus stakeholders working different of scholarly communication, and a between and of peer review in scholarly and Furthermore, while there were used to teach the peer review process, was All workshops included a using to set the by learning and group and learning were created based on of effective learning composed of and sharing is not to this as no new data were created or in this