Category: emerging technologies
At AERA this week, Amy Collier, Emily Schneider, and I will be presenting a paper that makes a series of arguments regarding learner activities and experiences in MOOCs in relation to clickstream-based MOOC research. One of the implications of our work is the following: learners’ participation and experiences in these courses resist binary and monolithic interpretations as they appear to be mediated by a digital-analog continuum as well as a social-individual continuum. In other words, learning and participation in MOOCs are both distributed and individually-socially negotiated. The following visual (which provides some hints on our results) makes this point clearer:
* and since the work of peer reviewers often goes unrecognized, let it be known, that this insight was prompted by a comment from one anonymous reviewer. So, whoever you are, thank you for your input.
I joined Audrey Watters, Philipp Schmidt, Stephen Downes, and Jeremy Friedberg in Toronto last week, to give a talk at Digital Learning Reimagined, an event hosted and organized by Ryerson University’s Chang School. I presented some of our latest research, and tried to highlight research findings and big ideas in 15 minutes. Below are my slides and a draft of my talk.
Welcome everyone! It’s a pleasure and an honor to be here. Even though I’m the person giving this talk, I’d like to acknowledge my collaborators. A lot of the work that I am going to present is collaborative and it wouldn’t have been possible without such amazing colleagues. These are: Royce Kimmons from the University of Idaho, Amy Collier and Emily Schneider from Stanford University, and Peter Shepherdson from the University of Zurich. The Canada Research Chairs program, the National Science Foundation and Royal Roads University have funded this work.
I want to start my talk by telling a story.
This castle that you see here is one of the most recognizable parts of Royal Roads University (RRU). But, don’t let the castle fool you. RRU was created in 1985. It’s purpose was to serve the needs of a changing society by serving working professionals through graduate digital education and multidisciplinary degrees. It has grown since 1985. It has matured, developed a social learning model that is now infused in all courses, developed new areas of focus, forged global partnerships, and continues to explore how to improve what it does through pedagogical and technological approaches.
Why am I sharing this short story about RRU?
Because this story, minus the specific details, is a common story. It’s also a Ryerson story, a story that is played out at the University of Southern New Hampshire, a story that Open Universities around that world have gone through. It is a story that repeats itself over and over for years and years.
What is the essence of the story?
It is often assumed that universities have been static, unchanging since the dawn of time. The short story I shared illustrates that universities are, and have always been, part of the society that houses them, and as societies change, universities change to reflect those societies. The economic, sociocultural, and technological pressures that universities are facing are sizable, and for better or for worse, usually for both, there’s a continuous re-imagination of education throughout time. Throughout time. Universities have always been changing.
As universities are changing and exploring different ways to offer education, faculty, researchers, and administrators engage in a number of practices that I like to describe as emerging. Emerging practices & emerging technologies are those that are not necessarily new, not yet fully researched, but appear promising.
Online learning and openness are example of emerging practices.
Online learning has a long history. But it also has a new history, with the development of multimedia platforms, media that can be embedded across platforms, syndication technologies that enable learners to use their own platforms for learning and so on. So, even though some of the problems that online learners are facing in contenmporary situations are not new (eg dropout), learners abilities’ to congregate in online communities is expanded through newer technologies and that poses different sorts of challenges and opportunities.
Another emerging practice is openness. Openness refers to liberal policies for the use, re-use, adaptation, and redistribution of content. Openness is also a value: It refers to adopting an ethos of transparency with regards to access to information. And this ethos ranges from academics publishing their work in open formats, to teaching open courses, to creating open textbooks. And it doesn’t stop at individual academics or institutions. In 2014 the Premiers of Alberta, British Columbia, and Saskatchewan signed a Memorandum of Understanding to facilitate creation, sharing, and use of Open Educational Resources. In the same year, SSHRC, NSERC, and CIHR have drafted a tri-agency open access policy to improve access to and dissemination of research results (NSERC, 2014);
There is a growing interest in and exploration of online learning and openness, practices which are still emerging. Next, I will share four recent results from our research into these practices that I believe are interesting to consider because they reveal the tensions that exist when dealing with emerging topics.
First, research into online learning is becoming more interdisciplinary
Interdisciplinary research into online learning means that individuals from a diverse range of disciplines, not just education, are interested in making sense of online learning. It is hoped that more research into online learning and more research from multidisciplinary groups will help us learn more about online learning and about learning in general.
We have evidence to show that research into online learning is becoming more interdisciplinary. I won’t bore you with the statistics, but we measure diversity in published research using a nifty measure and found that the period 2013-2014 can be described as more interdisciplinary than the period 2008-2012.
This is a positive trend, but before I explain its significance, let me explain to you how I view technology.
My perspective on online learning centers around the idea that technology is socially shaped . That means that technology always embeds its developers’ worldviews, beliefs, and assumptions into its design and the activities it supports and encourages.
What does this mean for interdisciplinarity? This means that we have both an opportunity and a challenge.
Our opportunity: to use our respective expertise to improve education.
Our challenge: to actually do interdisciplinary thinking and to go into the study and design of future educational systems with an open mind and the realization that our own personal experiences of education may not be generalizable. A lot of educational technology is produced by people of privilege and to develop educational technology that matters and makes societal difference, we need diversity in thinking and experience.
Our second finding refers to the increasing desire to collect, mine, and analyze data trails to make inferences about human behavior and learning. This practice is often referred to as learning analytics and educational data mining. This practice is a reflection of a larger societal trend toward big data analytics. The idea is that by looking at what people do online one can understand how to improve education.
A couple of things that researchers discovered for example are:
Data trails. Nearly everything that learners do online is tracked. Can we understand learners and improve learning by analyzing their data trails?
While these approaches can help us explain what people do, they often don’t tell us why they do they things they do nor how they actually experience online education.
My colleagues and I are interviewing MOOC students to learn about their experiences in MOOCs.
I am now going to tell you about our third result. We find that learners schedule their learning, use of resources, and participation to fit their daily life. This is in stark contrast to the idea of undergraduate education situated at a university and happening at particular time periods.
One retired individual in Panama that we interviewed works on his class early in the morning every day. Why does he do that? He does that because at that time his daughter is asleep. She is homeschooled and once she wakes up she needs access to the 1 computer that they have in the household to do her own schoolwork. In this case a lack of resources necessitates this scheduling.
One individual that we interviewed moved from the UK to the USA to be with her partner. She is currently waiting for her work permit, driver’s license, and so on, and she was enrolled in multiple MOOCs at the same time. She would work on her courses on Monday because she just “wanted them out of the way,” and so she would work on these courses straight throughout the day.
The fourth and final finding that I have for you today, is that MOOC platforms to date have not offered learners the ability to keep notes, so that particular activity, by virtue of being unsupported by the platform goes undetected when researchers only look at data trails.
Unsurprisingly, learners keep notes. A number of students that we talked to described that they keep notes on paper, frequently keeping a notebook for particular courses and returning to them during exams or during times that they needed them. Learners of course also keep notes in digital format. Usually in word documents, but again documents are dedicated to particular courses, but sometimes they are dedicated to particular topics across courses.
To give you an example, of how we believe this activity could be supported in the future and how we believe innovations can contribute to learning, we recommend designers support this practice by pedagogical innovations such as scaffolding notetaking, but also by technological innovations, by developing online systems for notetaking. What is important here is that such systems should support learning by being interoperable, by allow learners full and unrestricted access to their notes, supporting them to be able to import & export their notes between platforms. Such a design is in line with emerging ideas in the field which call for learners to own their data.
Thank you for being a great audience. I am really excited to hear the speakers that follow me, as I am sure you are!
A visualization of my talk, created by Giulia Forsythe
The Networked Scholars course starts in two weeks, on October 20th, with 2 options for participation.
1. Through the Canvas Network.
2. Through personal blogs and twitter accounts, syndicated, via the…. drumroll…. Networked Scholars Syndication hub. With special thanks to Alan Levine who has been helping a number of people implement this design, all readings and activities will be publicly-available, and this site syndicates blog/twitter feeds used as discussion/reflection spaces. The official Twitter hashtag for the course is #scholar14. If you are interested in this option, feel free to head over to the syndication hub, and connect your blog to the site!
Image: STS-131 Discovery Launch
I will be visiting my colleagues at the University of Edinburgh in mid-June to give a seminar on MOOCs, automation, artificial intelligence and pedagogical agents. This is a free event organized by the Moray House School of Education at the U of Edinburgh and supported by the Digital Cultures and Education research group and DigitalHSS. Please feel free to join us face-to-face or online (Date: 18 June 2014; Time: 1-3pm) by registering here.
This seminar will bring together some of my current and past research. A lot of my work in the past examined learners’ experiences with conversational and (semi)intelligent agents. In that research, we discovered that the experience of interacting with intelligent technologies was engrossing (pdf). Yet, learners often verbally abused the pedagogical agents (pdf). We also discovered that appearance (pdf) may be a significant mediating factor in learning. Importanly, this research indicated that “learners both humanized the agents and expected them to abide by social norms, but also identified the agents as programmed tools, resisting and rejecting their lifelike behaviors.”
A lot of my current work examines experiences with open online courses and online social networks, but what exactly does pedagogical agents and MOOCs have to do with each other? Ideas associated with Artificial Intelligence are present in both the emergence of xMOOCs (EdX, Udacity, and Coursera emanated from AI labs) and certain practices associated with them – e.g., see Balfour (2013) on automated essay scoring. Audrey Watters highlighted these issues in the past. While I haven’t yet seen discussions on the integration of lifelike characters and pedagogical agents in MOOCs, the use of lifelike robots for education and the role of the faculty member in MOOCs are areas of debate and investigation in both the popular press and the scholarly literature. The quest to automate instruction has a long history, and lives within the sociocultural context of particular time periods. For example, the Second World War found US soldiers and cilvilians unprepared for the war effort, and audiovisual devices were extensively used to efficiently train individuals at a massive scale. Nowadays, similar efforts at achieving scale and efficiencies reflect problems, issues, and cultural beliefs of our time.
I’m working on my presentation, but if you have any questions or thoughts to share, I’d love to hear them!
MITx and HarvardX deserve huge congratulations for making data associated with a number of their MOOCs publicly available. Four months ago, I wrote that the “community would benefit from access to the data that HarvardX and MITx have, as other individuals/groups could run additional analyses. Granted, I imagine this might require quite a lot of effort, not least in the development of procedures for data sharing.” It seems that the researchers at MITx and HarvardX have tackled the issues involved to make the data available, and have developed thoughtful procedures to ensure de-identification. While some of the steps taken may limit analyses (e.g., the de-identification process document notes that “rows with 60 or more forum posts were deleted,” thus eliminating highly active users), this is a big step in the right direction and it should be celebrated.
Now… can we have some qualitative data? If any institutions are interested in making those available, I’d love talk to you, give you input, and work with you toward that goal.
I spent part of last week in Dallas at the annual Emerging Technologies for Online Learning conference, organized by SLOAN-C. I describe my presentation at the conference in this post, but the sessions below were all relevant to my work:
Jim Groom’s keynote. Jim’s Domain of one’s own work resonates with me. Providing students with digital tools that will enable them to learn the ways of the web is significant, but the idea also resonates with me in the context of digital scholarship, which is one of my research strands. In particular, I see Jim’s project being applicable for PhD students who should be equipped with the tools, skills, and experiences to understand networked, open, and digital scholarship. I’ve met Jim briefly in the past, but we never had a chance to chat much, so it was great to be able to spend some more time together.
Amy Collier’s and Jen Ross’ plenary. The session focused on giving insightful descriptions of the messy and compromised realities of learning in contrast to the narratives of efficiency and ease suggested by numerous educational technology providers.
Andy Saltarelli’s study of belongingness and synchronicity in cooperative learning settings. What’s not to like about rigorous, theory-driven, large-scale evaluations of the socio-psychological constructs that make a difference in online learning contexts?
The session on distributed flips, or re-using MOOC resources in face-to-face/blended courses. MJ Bishop, Mike Caulfield, and Amy Collier came together to share their research on the topic. Mike was unable to join unfortunately, but he shared his thoughts here.
Rolin Moe organized a number of fantastic panels on issues pertaining to the field and I was excited to participate in the one focused on academics in educational technology, along with Jen Ross, Amy Collier, Jill Leafstedt, Jesse Stommel, and Sean Michael Morris. We had a wonderful conversation, but 50 minutes are never enough to cover this topic. The Sloan-C organizing committee should consider making this session a longer (free-to-attend) workshop.
I gave a presentation at the annual Emerging Technologies for Online Learning conference last week focusing on aspects of networked participatory scholarship. I kept track of other sessions of interest here.
The concept of networked scholarship is expressed in different ways in the literature, ranging from digital scholarship to social scholarship to open scholarship. In my presentation, I discussed two themes that have arisen from my 3+ years of qualitative and ethnographic studies into the practices of higher education scholars.
Both of these themes help us make better sense of scholars’ digital participation and networked scholarship. They also help us better describe online scholarly networks and the lives and practices of digital scholars.
The first theme refers to the notion of scholars using networks to enact digital/open scholarship and circumvent restrictions to the sharing of knowledge. I have a recent publication on this that you can read here.
The second theme is one that I am still developing. Specifically, in my research I found that social media and online social networks function as places where some academics express and experience care. While debates about the use of digital scholarship and social media use in education have so far largely focused on the professional experiences of scholars, with frequent suggestions to limit personal sharing, professional and personal identity are difficult to separate, and academics frequently collapse the boundaries between personal and professional sharing. Academics demonstrate vulnerability and express care online in many forms. In my presentation, I showed and discussed examples of what these very personal and intimate instances of sharing look like. A version of my slides appear below:
Below are my notes from the AERA 2014 session Rigor and Realism: Doing Educational Science in the Real World. The ideas are significant for learning technologies researchers focused on impacting practice and developing real-world innovations/interventions. While I share many of these values and discussed a number of them in past work (e.g, here and here), what appears below are Catherine Snow’s ideas.
Rigor and Realism: Doing Educational Science in the Real World
The Wallace Foundation Distinguished Lecture: Catherine Snow, Henry Lee Shattuck Professor of Education, Harvard University
Some educational researchers are adopting new models for doing educational research, models that start from problems of practice, prioritize the challenge of utility to educators, and presuppose partnership relationships between researchers and practitioners. In attempting to implement such approaches, we often find that attention to the conditions of real-world practice may compete with attention to the constraints of rigorous design. That familiar problem can be exacerbated by the conflicting epistemologies of real-world decision-making vs. rigorous scientific knowledge-building. This conflict, in its multiple forms only some of which will be discussed, is a dilemma rather than a problem; it demands careful consideration of approaches to balancing the desirable features of rigor and of realism when they conflict.
Notes from the presentation and twitter feed
A conflict exists between rigorous research and the demands of educational systems. Educators and researcher must work together as partners.
Researchers have to acknowledge the realities of practice.
Researchers should start with urgent problems of practice – not simply gaps that exist in the literature
Education research/science can be highly rigorous, but it needs to be relevant. If it’s not impacting students in schools, it doesn’t matter. Focus of ed research is often quality, but we have to make the case for utility. It should be about utility and relevance.
Education research should move from questions of “whether” to questions of “how” ( GV side note: especially relevant to learning technologies, as noted by past research – e.g., don’t ask whether online learning works, but ask how does it work, how do we make it better, under what conditions, for whom, etc).
Recruiting the next generation of scholars into this work may endanger their potential to publish and get grants
We need to modify practitioner preparation to provide guidance about the challenges of collaboration with researchers (e.g., teach value/limitations of research)
Numerous colleagues are invested in this model (e.g., WT Grant, iterative design work, design-based implementation)
Progress is slow. Are there really exciting initiatives that would make a huge difference that might change both edu research and practice?
- Build the partnership model into the preparation of doctoral students by institutionalizing and acknowledging relationships and acknowledging relationships. Schools of education need to embrace this approach and faculty members need to be supported even if it doesn’t have immediate payoff in journal articles. A way for schools of education to become relevant and keeping us from the constant danger of becoming second rate departments of Arts and Sciences rather than 1st grade institutions of social change
- Promote accountability by developing reliable and feasible measures of classroom practice that might eventually take the place of student outcome measures
- Take the wisdom of practice seriously and develop a mechanism for systematizing and curating it. Journal review process is elegant and effective in maintaining standards for a certain kind of knowledge, but why do we dismiss anecdotes that teachers tell us? Because there is no epistemological structure for evaluating and curating that knowledge. What does that structure look like? Even though practitioners generate knowledge, it disappears. Developing a mechanism to capture and curate knowledge would be a hugely powerful mechanism. We acknowledge wisdom of practice but don’t take it seriously. We need to raise the level of the wisdom of practice to a more respectable level, by making it indeed more respectable. Urgent agenda because of internet sharing of uncurated practices. There currently isn’t a way to curate teacher knowledge ( GV side note: Curation of internet resources is prevalent, and a number of individuals are arguing for a system of publish then filter – lots of connections to openness, digital participation, and digital/media literacies re: curation)
As a field we need an exciting agenda of research that is cumulative, rigorous and realistic