Category: NPS Page 1 of 8

Simple Checklists to Verify the Accuracy of AI-Generated Research Summaries

On August 18, 2025

In AI, emerging technologies, GenAI, generative artificial intelligence, my research, networked scholars, NPS, papers, research shorts

Do you share AI-generated audio/video summaries of your research with students? or with the broader public on social media? Below is a short article I wrote encouraging researchers to share a checklists alongside those summaries verifying their accuracy and noting their limits (the final version is at Veletsianos, G. (2025). Simple Checklists to Verify the Accuracy of AI-Generated Research Summaries. Tech Trends, XX(X), Xx-xx but here’s a public pre-print too).

Simple Checklists to Verify the Accuracy of AI-Generated Research Summaries

Picture this: An educational technology researcher shares a seven-minute AI-generated audio or video of their latest paper on social media. It sounds engaging and professional. But buried in that smooth narration, the AI has quietly transformed “may suggest” into “proves,” dropped crucial limitations, and expanded the study’s claims beyond what the data supports. The listeners, including students, policymakers, and journalists, have no way of knowing.

The proliferation of AI-generated audio and video summaries of research papers—through tools like Google’s NotebookLM and others—represents both an opportunity and a challenge for scholarly communication. These summaries are promising as they can expand the reach, accessibility, and consumption of our research for diverse audiences (cf. Veletsianos, 2016). They also allow us to efficiently engage with literature outside of our expertise. A seven-minute podcast consumed during a commute may reach audiences who would never read a 30-page paper.

Yet this convenience comes with risks. Peters and Chin-Yee (2025) for example, found that summaries generated by Large Language Models omitted study details and made overgeneralizations. Such risks can propagate misunderstandings, particularly when summaries circulate without clear indicators of their accuracy or limitations.

While some technical solutions to address this problem exist (e.g., including fine-tuning models and implementing algorithmic constraints) these approaches remain inaccessible to most researchers. We need a low-barrier intervention that empowers authors to assess and communicate the quality of AI-generated summaries to listeners.

I propose that researchers who share AI-generated summaries complete and publish a brief verification checklist alongside their summary. This practice serves two purposes: it encourages authors to critically review AI output before dissemination, and it provides audiences with transparency about the summary’s accuracy and limitations. Just as we expect ethics approval for research, we should normalize quality assurance for AI-generated scholarly content.

To facilitate this practice, below are two verification checklists, one for academic audiences and another for the general public, even though the latter could serve both audiences. Both are deliberately concise to enable sharing across digital platforms where these summaries circulate, from social media to publishers’ websites to course management systems.

Checklist 1: For Academic Audiences Checklist 2: For the General Public

Author verification: This summary of [paper title] was AI-generated using [tool name] on [date] and reviewed by the author(s). It accurately represents our work. For full details, nuance, and context, please refer to the original work at [URL].

The following items were verified:
✓ Research purpose or questions stated correctly
✓ Study design described correctly
✓ Summary matches study results (no fabricated data)
✓ Conclusions are explicitly limited to the study’s scope and context
✓ Key terminology used properly
✓ Theoretical, conceptual, and/or methodological frameworks are framed appropriately and are neither omitted, nor misrepresented
✓ Major limitations are included
✓ Context and scope are clear
✓ There summary does not omit anything of significance
✓ The tone is consistent with the original work

Issues noted: [Note any issues]
Author verification: This summary of [paper title] was AI-generated using [tool name] on [date] and reviewed by the author(s). It accurately represents our work. For full details, nuance, and context, please refer to the original work at [URL].

What we checked:
✓ Main findings are correct – nothing made up
✓ Doesn’t overstate what we found
✓ Includes what we studied and who participated
✓ Mentions important limitations
✓ Uses language appropriately
✓ Matches our original tone and message

Issues noted: [Note any issues, using plain language]

These checklists are a starting point, not a comprehensive solution. I have attempted to make them flexible enough to accommodate different research paradigms, but if you do use them, you should refine them to fit your needs and orientation. The point is not to develop the perfect checklist, but to provide a flexible tool that can be adapted and improved to minimize the risks of AI-generated research summaries. As AI tools become increasingly integrated into research dissemination, we must develop community standards for responsible use. Normalizing transparency practices now contributes toward maintaining the integrity that underpins scholarly communication.

In an academic landscape saturated with contested claims, particularly in education and educational technology where myths and zombie theories persist (e.g., Sinatra, & Jacobson, 2019; Suárez-Guerrero, Rivera-Vargas, & Raffaghelli, 2023), our commitment to accuracy and transparency must remain constant. Verifying AI-generated summaries constitutes a form of reputational stewardship. This quality assurance practice encourages authors to critically review AI output before it circulates, signaling to colleagues, institutions, and the public that they take seriously their role as knowledge custodians. By proactively verifying summaries, researchers can protect the integrity of their findings and build a reputation for reliability that enhances the trustworthiness of their entire body of work. At the end of the day, the few minutes invested in verifying AI-generated summaries of one’s work pale in comparison to the time that might be required to correct a misleading summary that gains traction on social media. Once an AI-generated misrepresentation goes viral, no amount of clarification can fully revise it. In this sense, verification checklists function as both quality control and professional insurance. They are a small investment that yield returns in credibility and peace of mind.

I encourage researchers to adopt versions of these checklists, journals to consider requiring them for AI-generated supplementary materials, and the broader academic community to refine and expand upon this framework. In an era of rapid AI developments, our commitment to scholarly accuracy and transparency must remain constant.

Author notes and transparency statement, as suggested by Bozkurt (2024): This editorial was reviewed, edited, and refined with the assistance of ChatGPT o3 and Gemini Pro 2.5 as of July 2025, complementing the human editorial process to address grammar, flow, and style. I critically assessed and validated the content and assessed potential biases inherent in AI-generated content. The final version of the paper is my sole responsibility.

References

Bozkurt, A. (2024). GenAI et al. Cocreation, authorship, ownership, academic ethics and integrity in a time of generative AI. Open Praxis, 16(1), 1-10.

Peters, U., & Chin-Yee, B. (2025). Generalization bias in large language model summarization of scientific research. Royal Society Open Science, 12(4), 241776. https://doi.org/10.1098/rsos.241776

Sinatra, G. M., & Jacobson, N. (2019). Zombie Concepts in Education: Why They Won’t Die and Why You Cannot Kill Them. In P. Kendeou, D. H. Robinson, & M. T. McCrudden (Eds.), Misinformation and fake news in education (S. 7–27). Information Age Publishing, Inc.

Suárez-Guerrero, C., Rivera-Vargas, P., & Raffaghelli, J. (2023). EdTech myths: towards a critical digital educational agenda. Technology, Pedagogy and Education, 32(5), 605-620.

Veletsianos, G. (2016). Networked Scholars: Social Media in Academia. New York, NY: Routledge.

Public & networked scholarship and its challenges

By George Veletsianos

On January 10, 2022

In my research, networked scholars, NPS, scholarship

Much of my work on public/networked/participatory scholarship approached the topic with the understanding that

scholarly practices impact how scholars use technology (e.g., institutional metrics and rewards systems shaping what kinds of activities faculty participate in, and thereby seek to amplify or improve via technology)
technology impacts scholarly practices (e.g., the adoption of a particular technology at an institution shaping what kinds of practices academics use; this can be anything, ranging from proctoring tools that encourage adoption of traditional assessment practices to institutional websites that ‘nudge’ faculty to include their social media profiles).

Note: “scholarship” here includes teaching, and isn’t just a synonym for research.

Much of this work was framed within a broader context of forces that shape how scholars enact digital and networked scholarship. Over the last few years, I’ve become more interested in the broader context and the broader forces. Of particular interest are three forces (or problems)

online harassment
systemic inequities (that impact online participation)
the mediating roles of ranking, sorting, and attention economy algorithms

There three areas overlap in unique ways as well (e.g., the case of an an op ed going viral and its author being on the receiving end of particularly vitriolic forms of abuse based on their identity).

I’d like to develop this framework of challenges further.

Tri-council guidance on using online public data in research

By George Veletsianos

On March 9, 2018

In emerging technologies, NPS, open, scholarship

I am often asked whether there are Canadian ethics guidelines on the use of online public data in research. The relevant section from the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans is provided below. I believe that researchers should take further steps to protect privacy and confidentiality pertaining to public data, but with regards to accessing and using public online data, this is a start.

A sample project to which these guidelines may apply is the following: The researcher will collect and analyze Twitter profiles and postings of higher education stakeholders (e.g., faculty, researchers, administrators) and institutional offices (e.g., institutional Twitter accounts). This research will use exclusively publicly available information. Private Twitter accounts (ie those that are not public and involve an expectation of privacy) will be excluded from the research. The purposes of the research is to gain a better understanding of Twitter metrics, practices, and use/participation.

=== Begin relevant Tri–council guidance ===

Retrieved on December 12 2014 from http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/chapter2-chapitre2/

REB review is also not required where research uses exclusively publicly available information that may contain identifiable information, and for which there is no reasonable expectation of privacy. For example, identifiable information may be disseminated in the public domain through print or electronic publications; film, audio or digital recordings; press accounts; official publications of private or public institutions; artistic installations, exhibitions or literary events freely open to the public; or publications accessible in public libraries. Research that is non-intrusive, and does not involve direct interaction between the researcher and individuals through the Internet, also does not require REB review. Cyber-material such as documents, records, performances, online archival materials or published third party interviews to which the public is given uncontrolled access on the Internet for which there is no expectation of privacy is considered to be publicly available information.

Exemption from REB review is based on the information being accessible in the public domain, and that the individuals to whom the information refers have no reasonable expectation of privacy. Information contained in publicly accessible material may, however, be subject to copyright and/or intellectual property rights protections or dissemination restrictions imposed by the legal entity controlling the information.

However, there are situations where REB review is required.

There are publicly accessible digital sites where there is a reasonable expectation of privacy. When accessing identifiable information in publicly accessible digital sites, such as Internet chat rooms, and self-help groups with restricted membership, the privacy expectation of contributors of these sites is much higher. Researchers shall submit their proposal for REB review (see Article 10.3).

Where data linkage of different sources of publicly available information is involved, it could give rise to new forms of identifiable information that would raise issues of privacy and confidentiality when used in research, and would therefore require REB review (see Article 5.7).

When in doubt about the applicability of this article to their research, researchers should consult their REBs.

=== End relevant Tri–council guidance ===

Recent SSHRC awards

By George Veletsianos

On December 20, 2017

In my research, networked scholars, NPS, online learning, open

SSHRC recently announced the awards of the latest round of the Insight and Insight Development grants, and we can now announce that we were awarded two grants for our research. Both grants are collaborations. The first with Dr. Royce Kimmons and the second with Dr. Jaigris Hodson. I’m a true believer in people’s ability to collaborate to go farther together. More than 93% of the funding will go to student research assistants. Here’s the work that these two awards will support:

SSHRC Insight grant #435-2017-160. PI: Veletsianos; Collaborator: Kimmons, R. Faculty members’ online participation and expression of self over time.

Summary: Researchers’ understanding of longitudinal aspects of digital technology use in education is limited. While many researchers, policymakers, and businesspeople are hopeful about the potential positive impacts that academics’ use of digital technology may generate, the empirical evidence describing the nature of academics’ online participation over time is scant and is largely predicated on small-scale studies. We will address this problem by studying whether, how, and why academics’ online participation and presentation of the self change over time. We will use a mixed methods approach combining descriptive/inferential analyses with basic qualitative studies using data collected from interviews and data mining of social media sites.

SSHRC Insight Development grant #430-2017-00104. PI: Veletsianos; Co-PI: Hodson, J. Female academics’ experiences of harassment on social media.

Summary: Prior research shows that some female academics, especially those who are in the public eye and use technology to promote their work, are at great risk of harassment. To gain a greater understanding of this issue, this mixed methods investigation seeks to investigate women scholars’ experiences of online harassment. The proposed research will use data arising from interviews, social media posts, and surveys to gain a deep and multidimensional understanding of harassment aimed at academics.

Institutional Use of Twitter – national analyses

By George Veletsianos

On November 5, 2017

In my research, NPS, open, scholarship

We recently wrote two papers that examined institutional uses of Twitter in Canada and the United States. As part of that work, we identified similar analyses taking place in other countries. These are listed below:

Country	Citation
Australia	Palmer, S. (2013). Characterisation of the use of Twitter by Australian Universities. Journal of Higher Education Policy and Management, 35, 333–344.
Canada	Veletsianos, G., Kimmons, R., Shaw, A., Pasquini, L. & Woodward, Ss. (2017). Selective Openness, Branding, Broadcasting, and Promoting: Twitter Use in Canada’s Public Universities. Educational Media International, 54(1), 1-19.
Turkey	Yolcu, O. (2013). Twitter usage of universities in Turkey. Turkish Online Journal of Educational Technology, 12, 360–371.
UK	Jordan, K. (2017). Examining the UK higher education sector through the network of institutional accounts on Twitter. First Monday, 22(5). doi:http://dx.doi.org/10.5210/fm.v22i5.7133
USA	Kimmons, R., Veletsianos, G., & Woodward, S. (2016). Institutional Uses of Twitter in Higher Education. Innovative Higher Education, 42(2), 97-111.

Lola Olufemi and student/faculty social media harassment

By George Veletsianos

On October 26, 2017

In learner experience, my research, networked scholars, NPS, open, scholarship

Below is a short interview with Lola Olufemi. The description from the BBC reads “Lola Olufemi is 21 years old and Cambridge University Students’ Union Women’s Officer. She found herself on the front page of a national newspaper, the face of a campaign to “decolonise” the English curriculum at Cambridge University. She discusses with Jenni Murray how she feels she’s been scapegoated by the media and her fears for the impact this could have on other young, black women wanting to speak out.”

I was watching this unfold yesterday, and witnessed the racist and misogynistic tweets fly by. One of which came from a professor at a well-known unversity, and as I responded at the time, what sort of academic responds in such a vile way to a person, let alone a student. As was shared on Twitter the institution has policies processes to deal with the harassing faculty member, but the questions that have been preoccupying my thinking over the last few months is the following: In what ways should our universities respond to the harassment that their students and faculty receive online, and on social media in particular? What are the institutional and individual responsibilities when we encourage students and faculty to be present on social media?

Discreet Openness: Scholars’ Selective and Intentional Self-Disclosures Online

By George Veletsianos

On August 17, 2016

In emerging technologies, NPS, open, papers, scholarship, sharing

What do scholars share on social media? Like the jelly jars below, some topics shared/discussed are familiar. The center jelly nn the top row? I’ve seen many of those. A scholar sharing a link to a paper? I’ve seen many of those, too. Other jellies, and scholarly activities online, are more complex and require a closer look. The bottom right jelly? I’m not quite sure what to make of it. Some scholars disclose challenging professional and personal issues on social media. That’s what Bonnie Stewart and I set out to understand in a our paper Discreet Openness: Scholars’ Selective and Intentional Self-Disclosures Online. Popular literature tends to offer conflicting advice on this topic. Scholars are encouraged to share both personal and professional aspects of their self online, but at the same time they are advised to “watch what they say.” The empirical literature examining scholars’ online self-disclosures and the reasons for making these disclosures remains limited.

DGJ_5184 – Jelly Jars by Dennis Jarvis

Research into emergent forms of scholarship focuses on academics’ use of technology for learning, teaching, and research. Very little attention has been paid in the literature to scholars’ uses of social media to disclose challenging personal and professional issues. This article addresses the identified gap in the literature and presents a qualitative investigation into the types of disclosures that 16 scholars made online and their reasons for doing so. Results identify wide-ranging personal and professional disclosures. Participants disclosed not only about academia-related issues but also about challenges pertaining to family, mental health, physical health, identity, and relationships. Some scholars disclosed as a way to grapple with challenges they faced; others disclosed tactically, sharing information for political rather than personal reasons. Yet others disclosed as a way to welcome care in their lives. In all instances, though, disclosures were selective, intentional, and approached with foresight.

Unlike popular literature that suggests that scholars are “naive users of social media” and must exercise caution, our research shows that people might be thinking deeply about the the ways that the share aspects of their lives.

You can retrieve the paper from here:

Veletsianos, G. & Stewart, B. (2016). Scholars’ open practices: Selective and intentional self-disclosures and the reasons behind them. Social Media + Society, 2(3). doi: 10.1177/2056305116664222