Professor & Canada Research Chair in Innovative Learning and Technology at Royal Roads University

Category: online learning

Making sense of the Digital Learning Research Network gathering (#dLRN15)

Posted on October 20th, by George Veletsianos in emerging technologies, my research, online learning, sharing. No Comments

I was at a small gathering last week, called the Digital Learning Research Network. It was hosted at Stanford and it aimed to explore the messiness of digital learning. This was not representative of Silicon Valley’s uncritical love affair with technology. Many colleagues wrote reflections about it: Catherine CroninKristen Eshleman,  Josh KimJonathan Rees, Tim Klapdor,  Alyson Indrunas, Adam Croom, Whitney Kilgore, Matt Crosslin, Laura Gogia, Patrice Torcivia, and Lee Skallerup Bessette (to name a few). When was the last time you were at a small conference, other than the ones focusing on blogging, and this many people took time after the event to blog about it?

The messiness of digital learning isn’t a new development. It is something that educational technology evangelists ignore, but as a researcher who has an affinity for qualitative data, and as one who is increasingly using data mining techniques on open social data, I can tell you that mess is the norm and not the exception. I’m not the only one.

For me, the conference questioned educational technology but looked to it for empowerment. It critiqued universities but saw them as places to create a more just and equitable society. It brought attention to the US-centric conversations happening in this space, but recognized that we can learn from one another. It sought research, but did not seek to emulate research-focused conferences. It allowed Dave to share his thoughts but called him out on it when it was time to stop. ;)

I see the conference as the start of a longer and larger conversation. Many of us are doing research in this space and many were missing. Let’s expand the conversation.

Strategies for designing student- centered, social, open, and engaging digital learning experiences

Posted on October 6th, by George Veletsianos in emerging technologies, learner experience, my research, online learning, open, scholarship. No Comments

Recently, I had the privilege of organizing a workshop for the Faculty of Humanities and Social Sciences at Athabasca University. The goal was to help the organization work through what they might need to do to put in practice a new strategic plan which calls for student-centered and open digital learning. I used the slides below to assist faculty, instructors, and instructional designers translate theory into practice.


Digital Learning, Emerging Technologies, Abundant Data, and Pedagogies of Care

Posted on September 3rd, by George Veletsianos in my research, online learning, scholarship. No Comments

I was at the Emerging Technologies in Authentic Learning Contexts Conference in Cape Town this week, where I gave one of the keynotes. In my talk, I highlighted some of the assumptions of the Educational Technology evangelists and explained how educational technology as an industry departs from educational technology as a field of study. I argued for context-driven innovation, and gave some examples from our current/upcoming research to explain these arguments. My slides are below.

My visit in Cape Town included many highlights – amongst them, meeting Jan Herrington, Gilly Salmon, Dick Ng’ambi and revisiting with Johannes Cronje and Nick Rushby. Spending time with Laura Czerniewicz though, was a real treat. Here’s proof (selfie #1 by Laura, photo of selfie by Jolanda Morkel)

The buzzwords of #edtech

Posted on August 26th, by George Veletsianos in emerging technologies, my research, online learning, scholarship. No Comments

An interesting article this morning from Jeff Young at the Chronicle of Higher Education notes:

One of the obstacles to bringing “adaptive learning” to college classrooms is that professors, administrators, and even those who make adaptive-learning systems don’t always agree on what that buzzword means. That was a major theme of a daylong Adaptive Learning Summit held here on Tuesday. Several people interviewed at the summit, held by the education-innovation group National Education Initiative, noted that part of the problem is a proliferation of companies that make big promises based on making their technologies adaptive, yet all use the term slightly differently.

I would counter that the big (and unsubstantiated) promises are a greater problem than the buzzwords, but the lack of clarity on what these concepts refer to are an issue, too.

The introductory sentences from Online Learning: Emerging Technologies and Emerging Practices (the second edition of the Emerging Technologies in Distance Education book I edited, which is forthcoming in 2016), make a similar argument:

Many of these (new) approaches to education and scholarship can be categorized as either emerging technologies (e.g., automated grading applications within MOOCs) or emerging practices (e.g., sharing instructional materials online under licenses that allow recipients to reuse them freely).

The terms “emerging technologies” and “emerging practices” however, are catchall phrases that are often misused and haphazardly defined. As Siemens (2008, ¶ 1) argues, “terms like ‘emergence,’ ‘adaptive systems,’ ‘self-organizing systems,’ and others are often tossed about with such casualness and authority as to suggest the speaker(s) fully understand what they mean.”

A clearer and more uniform understanding of emergence and of the characteristics of emerging technologies and practices will enable researchers to examine these topics under a common framework and practitioners to better anticipate potential challenges and impacts that may arise from their integration into learning environments.

Multidisciplinary, interdisciplinary, and cross-disciplinary research on MOOCs and digital learning

Posted on July 2nd, by George Veletsianos in moocs, my research, online learning, scholarship. No Comments

Multidisciplinary, interdisciplinary, and crossdisciplinary research represent promising approaches for studying digital learning. Prior research however, discovered that research efforts directed at digital learning via MOOCs were dominated by individuals affiliated with education (Gašević, Kovanović, Joksimović, and Siemens, 2014). In their assessment of proposals submitted for funding under the MOOC research initiative (MRI), Gašević and colleagues show that more than 50% of the authors in all phases of the MRI grants were from the field of education. This result was interesting because a common perception in the field is that the MOOC phenomenon is “driven by computer scientists” (p. 166).

We were curious to understand whether this was the case with research conducted on MOOCs (as opposed to grant proposals) and used a dataset of author affiliations publishing MOOC research in 2013-2015 to examine the following questions:

RQ 1: What are the disciplinary backgrounds of the authors who published empirical MOOC research in 2013-2015?

RQ 2: How does the disciplinary distribution of the authors who published MOOC research in 2013-2015 compare to that of the submissions to the MRI reported by Gašević et al. (2014)?

RQ 3: Is the 2013-2015 empirical research on MOOCs more or less interdisciplinary than was previously the case?

Results from our paper (published in IRRODL last week) show the following:

– In 2013-2015, Education and Computer Science (CS) were by far the most common affiliations for researchers writing about MOOCs to possess
– During this time period, the field appears to be far from monolithic, as more than 40% of papers written on MOOCs are from authors not affiliated with Education/CS.
– The corpus of papers that we examined (empirical MOOC papers published in 2013-2015) was less dominated by authors from the field of education than were the submissions to the MOOC Research Initiative.
– A comparison of affiliations with past published papers shows that recent MOOC research appears to be more interdisciplinary than was the case in research published in 2008–2012.

We draw 2 implications from these results:

1. Current research on MOOCs appears to be more interdisciplinary than in the past, suggesting that the scientific complexity of the field is being tackled by a greater diversity of researchers. This suggests that even though xMOOCs are often disparaged for their teacher-centric and cognitivist-behaviorist approach, empirical research on xMOOCs may be more interdisciplinary than research on cMOOCs.

2. These results however, also lead us to wonder whether the trend toward greater interdisciplinarity of recent research might reflect (a) the structure and pedagogical model used in xMOOCs, (b) the greater interest in the field of online learning, and (c) the hype and popularity of MOOCs. Could it be that academics’ familiarity with the xMOOC pedagogical model make it a more accessible venue in which researchers from varying disciplines can conduct studies? Or, is increased interdisciplinary attention to digital education the result of media attention, popularity, and funding afforded to the MOOC phenomenon?

We conclude by arguing that “The burgeoning interest in digital learning, learning at scale, online learning, and other associated innovations presents researchers with the exceptional opportunity to convene scholars from a variety of disciplines to improve the scholarly understanding and practice of digital learning broadly understood. To do so however, researchers need to engage in collaborations that value their respective expertise and recognize the lessons learned from past efforts at technology-enhanced learning. Education and digital learning researchers may need to (a) take on a more active role in educating colleagues from other disciplines about what education researchers do and do not know about digital learning from the research that exists in the field and, (b) remain open to the perspectives that academic “immigrants” can bring to this field (cf. Nissani, 1997).”

For more on this, here’s our paper.

Learners’ experiences in MOOCs: Notetaking, social networks, and content consumption

Posted on May 26th, by George Veletsianos in learner experience, moocs, my research, online learning, papers. 27 comments

What do learning experiences in MOOCs look like? Amy Collier, Emily Schneider and I have just published a paper that provides some in-depth answers to this question. Here is a copy of the paper in pdf. The paper is part of a special issue published by the British Journal of Educational Technology which can be found here (there are many excellent pieces in that issue, so be sure to read them).

In addition to trying to understand learner experiences, in the paper we describe that we did this study because “ease of access to large data sets from xMOOCs offered through an increasing number of centralized platforms has shifted the focus of MOOC research primarily to data science and computational methodologies, giving rise to a discourse suggesting that teaching and learning can be fully analyzed, understood and designed for by examining clickstream data”

Our abstract reads:

Researchers describe with increasing confidence what they observe participants doing in massive open online courses (MOOCs). However, our understanding of learner activities in open courses is limited by researchers’ extensive dependence on log file analyses and clickstream data to make inferences about learner behaviors. Further, the field lacks an empirical understanding of how people experience MOOCs andwhy they engage in particular activities in the ways that they do. In this paper, we report three findings derived by interviewing 13 individuals about their experiences in MOOCs. We report on learner interactions in social networks outside of MOOC platforms, notetaking, and the contexts that surround content consumption. The examination and analysis of these practices contribute to a greater understanding of the MOOC phenomenon and to the limitations of clickstream-based research methods. Based on these findings, we conclude by making pragmatic suggestions for pedagogical and technological refinements to enhance open teaching and learning.

We reported 3 main findings:

1. Interactions in social networks outside of the MOOC platform

A number of learners alluded to interactions they have had with individuals who are part of their social networks. These include digital connections with other participants in a MOOC, face-toface interactions with friends and family, and face-to-face interactions with new connections in a MOOC.

2. Notetaking

Despite the fact that none of the popular MOOC platforms support integrated notetaking at the time of writing this paper, nearly all interviewees reported taking notes while watching lecture videos. Only one interviewee never took notes. However, the tools used to take notes and the subsequent use of notes varied substantially by learner.

3. Consuming content

All individuals participating in this study discussed factors that shaped the ways they consumed MOOC content, shedding light on the context surrounding their participation. Scholars in the learning sciences have long highlighted the critical role of the environment, arguing that learning must be understood as a sociocultural phenomenon situated in context and culture (Brown, Collins & Duguid, 1989). Patterns of MOOC content consumption can be examined by clickstream data, but these contextual factors help explain why learners exhibit particular patterns of participation.



Veletsianos, G., Collier, A., & Schneider, E. (2015). Digging Deeper into Learners’ Experiences in MOOCs: Participation in social networks outside of MOOCs, Notetaking, and contexts surrounding content consumption. British Journal of Educational Technology 46(3), 570-587.

Who studies MOOCs? (or, thinking with Siemens & Downes)

Posted on May 2nd, by George Veletsianos in my research, online learning, open, papers. 28 comments

If it wasn’t abundantly clear by now, George Siemens and Stephen Downes are two individuals that are making significant contributions to the field. I respect them both and I enjoy engaging with their work. They have been having a conversation  regarding the research and academic diversity in MOOCs, (here and here and here) as a result of a report that George and colleagues released on the history and current state of blended, distance, and online education.

I am writing to add to that conversation because my colleagues and I analyzed some parts of the literature published on MOOCs, and have some results that are relevant and interesting. The paper is under review but the editor gave me permission to share our findings.

We studied the disciplinary distribution of the authors who published MOOC research in 2013-2015 and compared it to the submissions to the MOOC Research Initiative (MRI) reported in Gašević et al., (2014). Our tests showed that the MOOC literature published in 2013-2015 differed significantly from the MRI submissions: our corpus had a greater representation of authors from Computer Science and the Gašević et al., corpus had a greater representation of authors from Education and Industry. In other words, our corpus was less dominated by authors from the field of education than were the MRI submissions. One of Downes criticisms is the following: “the studies are conducted by people without a background in education.” This finding lends some support to his claim, though a lot of the research on MOOCs is from people affiliated with education, but to support that claim further one could examine the content of this papers and identify whether an educational theory is guiding their investigations. 

We also compared author affiliation information in our papers with the papers used in Liyanagunawardena et al.’s (2013) review of the 2008-2012 MOOC literature. We found that the two samples differed significantly. For example, the Liyanagunawardena et al. corpus was relatively over-represented in the Independent Researcher category. This result suggests that the bodies of literature published in 2008-2012 and 2013-2015 differ in significant ways. This may or may not hold true for the writing that has occurred in blogs, unpublished reports etc. We don’t know and we haven’t studied that. 

Finally, and most significantly, we found that the disciplinary makeup of the literature is changing over time: there’s greater interdisciplinary activity in MOOC research now than in the past. This result is very interesting and its implications are worth examining. Suffice to say that this provides opportunities (can we capitalize on the expertise of one another to improve education?) and challenges (are newcomers to the field capitalizing on what we now about the science of learning?). The move to greater inter- and cross- disciplinarity in the field is evident in other initiatives. See for example, the Digital Learning Research Network.

Keep in mind that this research faces some of the same limitations raised by Downes (i.e. like Siemens, our inclusion criteria mean that some research is included while other is excluded). However, it also addresses some of those criticisms. For example, we tried to verify whether some of our results could have arisen by chance by running 10,000 computer simulations on the samples. The computer is confident that they could not have arisen by chance.

I’m hoping this paper will be out of peer-review soon so that I can share, but I’m thankful to the editor that allowed us to share our findings.

The Defining Characteristics of Emerging Technologies and Emerging Practices in Online Education

Posted on April 14th, by George Veletsianos in my research, online learning, scholarship. 12 comments

My amazing colleagues Amy Collier and Jen Ross have been blogging about the chapter they wrote for the second edition of my Emerging Technologies in Distanced Education book, and have thus encouraged me to blog about it :)

First, the good news: Athabasca University Press will be publishing Online Learning: Emerging Technologies and Emerging Practices in an open access format.

One of the interesting aspects of editing a book is interacting with chapter authors (15 in this case) and helping to create a coherent narrative. In this case, we were starting with the first edition of the book as a foundation, but more importantly, we were building on the first chapter of the first book in which I defined “emerging technologies.” I’ve clarified in the second book that this collection focuses as much on the emerging practices of online learning/participation (e.g., networked scholarship, open education, learning analytics) as much as it focuses on the technologies that underpin them. Hence the new title.

In the first edition of the book, I argued that emerging technologies are “not yet fully researched” and “not yet fully understood.” Thus, you can quickly begin to see that emerging technologies and emerging practices are intertwined, as, open scholarship for example, or learners’ experiences with automated assessment, are not yet fully research and not yet fully understood.

In the second edition of the book, I revisit this work, and argue that what makes technologies and practices emerging are not specific technologies or practices, but the environments in which a particular technology or practice operate. This definition recognizes that learning, teaching, and scholarship are sociocultural phenomena situated in specific contexts and influenced by the cultures in which they take place. Emerging technologies and practices appear to share four characteristics:

  • Emerging Technologies and Emerging Practices are Not Defined by Newness
  • Emerging Technologies and Emerging Practices Are Evolving Organisms that Exist in a State of “Coming into Being”
  • Emerging Technologies and Emerging Practices Are Not Yet Fully Understood or Researched (what Amy and Jen branded as not-yetness)
  • Emerging Technologies and Emerging Practices Have Promising But as Yet Unfulfilled Potential

There’s 8 more great chapters in the book, and I’m looking forward to sending them off to the publisher. More in the coming months!