Category: my research
One of the main arguments that we made in our recent paper on MOOCs, which is also the argument that I continue in this op ed piece published in Inside Higher Ed, is that the field needs to embrace diverse research methods to understand and improve digital learning. The following passage is from our paper, and given that the paper is quite long, I thought that posting it here might be helpful:
By capturing and analyzing digital data, the field of learning analytics promises great value and potential in understanding and improving learning and teaching. The focus on big data, log file analyses, and clickstream analytics in MOOCs is reflective of a broader societal trend towards big data analytics (Eynon, 2013; Selwyn, 2014) and toward greater accountability and measurement of student learning in higher education (Leahy, 2013; Moe, 2014). As technology becomes integrated in all aspects of education, the use of digital data and computational analysis techniques in education research will increase. However, an over-reliance on log file analyses and clickstream data to understand learning leaves many learner activities and experiences invisible to researchers.
While computational analyses are a powerful strategy for making a complex phenomenon tractable to human observation and interpretation, an overwhelming focus on any one methodology will fail to generate a complete understanding of individuals’ experiences, practices, and learning. The apparent over-reliance on MOOC platform clickstream data in the current literature poses a significant problem for understanding learning in and with MOOCs. Critics of big data in particular question what is missing from large data sets and what is privileged in the analyses of big data (e.g., boyd & Crawford, 2012). For instance, contextual factors such as economic forces, historical events, and politics are often excluded from clickstream data and analyses (Carr, 2014; Selwyn 2014). As a result, MOOC research frequently examines learning as an episodic and temporary event that is divorced from the context which surrounds it. While the observation of actions on digital learning environments allows researchers to report activities and behaviors, such reporting also needs an explanation as to why learners participate in MOOCs in the ways that they do. For example, in this research, participants reported that their participation in MOOCs varies according to the daily realities of their life and the context of the course. Learners’ descriptions of how these courses fit into their lives are a powerful reminder of the agency of each individual.
To gain a deeper and more diverse understanding of the MOOC phenomenon, researchers need to use multiple research methods. While clickstream data generates insights on observable behaviors, interpretive research approaches (e.g., ethnography, phenomenology, discourse analysis) add context to them. For example, Guo, Kim, and Rubin (2014), analyzed a large data set of MOOC video-watching behaviors, found that the median length of time spent watching a video is six minutes, and recommended that “instructors should segment videos into short chunks, ideally less than 6 minutes.” While dividing content into chunks aligns with psychological theories of learning (Miller, 1956), this finding does not explain why the median length of time learners spent watching videos is six minutes. Qualitative data and approaches can equip researchers to investigate the reasons why learners engage in video-watching behaviors in the ways that they do. For example, the median watching length of time might be associated with learner attention spans. On the other hand, multiple participants in this study noted that they were fitting the videos in-between other activities in their lives – thus shorter videos might be desirable for practical reasons: because they fit in individuals’ busy lives. Different reasons might be uncovered that explain why learners seem to engage with videos for six minutes, leading to different design inspirations and directions. Because the MOOC phenomenon, and its associated practices, are still at a nascent stage, interpretive approaches are valuable as they allow researchers to generate a refined understanding of meaning and scope of MOOCs. At the same time, it is significant to remember that a wholly interpretive approach to understanding learning in MOOCs will be equally deficient. Combining methods and pursuing an understanding of the MOOC phenomenon from multiple angles, while keeping in mind the strengths and weaknesses of each method, is the most productive avenue for future research.
A computational analysis and data science discourse is increasingly evident in educational technology research. This discourse posits that it is possible to tell a detailed and robust story about learning and teaching by relying on the depth and breadth of clickstream data. However, the findings in our research reveal meaningful learner activities and practices that evade data-capturing platforms and clickstream-based research. Off-platform experiences as described above (e.g., notetaking) call into question claims that can be made about learning that are limited to the activities that are observable on the MOOC platform. Further, the reasons that course content is consumed in the ways that it is exemplifies the opportunity to bring together multiple methodological approaches to researching online learning and participation.
What do learning experiences in MOOCs look like? Amy Collier, Emily Schneider and I have just published a paper that provides some in-depth answers to this question. Here is a copy of the paper in pdf. The paper is part of a special issue published by the British Journal of Educational Technology which can be found here (there are many excellent pieces in that issue, so be sure to read them).
In addition to trying to understand learner experiences, in the paper we describe that we did this study because “ease of access to large data sets from xMOOCs offered through an increasing number of centralized platforms has shifted the focus of MOOC research primarily to data science and computational methodologies, giving rise to a discourse suggesting that teaching and learning can be fully analyzed, understood and designed for by examining clickstream data”
Our abstract reads:
Researchers describe with increasing confidence what they observe participants doing in massive open online courses (MOOCs). However, our understanding of learner activities in open courses is limited by researchers’ extensive dependence on log file analyses and clickstream data to make inferences about learner behaviors. Further, the field lacks an empirical understanding of how people experience MOOCs andwhy they engage in particular activities in the ways that they do. In this paper, we report three findings derived by interviewing 13 individuals about their experiences in MOOCs. We report on learner interactions in social networks outside of MOOC platforms, notetaking, and the contexts that surround content consumption. The examination and analysis of these practices contribute to a greater understanding of the MOOC phenomenon and to the limitations of clickstream-based research methods. Based on these findings, we conclude by making pragmatic suggestions for pedagogical and technological refinements to enhance open teaching and learning.
We reported 3 main findings:
1. Interactions in social networks outside of the MOOC platform
A number of learners alluded to interactions they have had with individuals who are part of their social networks. These include digital connections with other participants in a MOOC, face-toface interactions with friends and family, and face-to-face interactions with new connections in a MOOC.
Despite the fact that none of the popular MOOC platforms support integrated notetaking at the time of writing this paper, nearly all interviewees reported taking notes while watching lecture videos. Only one interviewee never took notes. However, the tools used to take notes and the subsequent use of notes varied substantially by learner.
3. Consuming content
All individuals participating in this study discussed factors that shaped the ways they consumed MOOC content, shedding light on the context surrounding their participation. Scholars in the learning sciences have long highlighted the critical role of the environment, arguing that learning must be understood as a sociocultural phenomenon situated in context and culture (Brown, Collins & Duguid, 1989). Patterns of MOOC content consumption can be examined by clickstream data, but these contextual factors help explain why learners exhibit particular patterns of participation.
Veletsianos, G., Collier, A., & Schneider, E. (2015). Digging Deeper into Learners’ Experiences in MOOCs: Participation in social networks outside of MOOCs, Notetaking, and contexts surrounding content consumption. British Journal of Educational Technology 46(3), 570-587.
I am editing, revising, and re-writing various parts of my book, Networked Scholars. I still like the name, but I mentioned the other day on Twitter that I should rename the book to “Yes, but…” because of the complexities and intricacies inherent in the use of social media for scholarship (as in “yes scholars network, but privilige permeates networks”). Or because I now know that trying to synthesize research my colleagues and I did over the last 6 years isn’t an easy feat (as in “Yes, I’ll write this book, but I am looking forward to turning my attention to other activities”).
Today I was writing about crowdsourcing and networks as places of knowledge sharing, creation, and dissemination. Here’s a relevant piece:
While Tufecki (2014) convincingly argues that practices may differ from one social media platform to another, and big data analyses focusing on one platform may not transfer to others, one common element in the use of social media for knowledge production and dissemination is the concept of crowdsourcing. Crowdsourcing refers to the process of gathering contributions from large groups of individuals in order to solve a common problem or tackle a challenge. Though readers may be familiar with modern crowdsourcing examples that are mediated by technology (e.g., wikipedia as a content crowdsourcing platform), the practice has long existed before the rise of social media. For instance, the design of the Sydney Opera House was crowdsourced. It was based on a 1955 international design competition that received 233 entries. Crowdsourcing content and ideas characterizes social media use, and scholars have capitalized on this practice to gather readings for their syllabi, activities for their courses, resources for their research, and other input – including effort – intended to solve scholarly problems.[Not included in the book: A fun but could-have-held-my-iphone-more-horizontally picture of the lovely Sydney opera house I took a while back]
I am editing, revising, and re-writing various parts of my book, Networked Scholars. I can’t write any more today, so here’s a visual update:
Update (May 13): As a result of your amazing response to this invitation, we are not currently seeking to interview any more people. We are deeply humbled by everyone’s desire to contribute and will be sharing our results in due course. Thank you!
We are inviting PhD students/candidates and academics to participate in a research study that we are conducting entitled “Academics’ use of social media: care and vulnerability.”
While the research community has studied the use of social media for teaching/research, we don’t know much about how social media are used by academics to share the challenges they face, express their vulnerabilities, and experience care online.
If you have disclosed a professional challenge that you have faced on social media (e.g. blogged eponymously or anonymously about: being denied tenure, a dissertation committee conflict, or underemployment or adjunct challenges), we invite you to participate in this study.
If you know of any colleagues who have disclosed such challenges on social media, please feel free to share this call with them.
We believe that these experiences are significant to share and discuss and we would love the opportunity to interview you to learn and write about your experiences.
If you are interested in participating in this study, please visit the following page to read the consent form that provides more details about this project: http://survey.royalroads.ca/index.php?sid=44151
We understand that this topic is very personal and discussing it with us may be difficult. If you have any questions or concerns regarding this study, please don’t hesitate to contact us. We would love to talk to you more about it.
George & Bonnie
Dr. George Veletsianos
Canada Research Chair and Associate Professor
Royal Roads University
Dr. Bonnie Stewart
Royal Roads University/University of Prince Edward Island
If it wasn’t abundantly clear by now, George Siemens and Stephen Downes are two individuals that are making significant contributions to the field. I respect them both and I enjoy engaging with their work. They have been having a conversation regarding the research and academic diversity in MOOCs, (here and here and here) as a result of a report that George and colleagues released on the history and current state of blended, distance, and online education.
I am writing to add to that conversation because my colleagues and I analyzed some parts of the literature published on MOOCs, and have some results that are relevant and interesting. The paper is under review but the editor gave me permission to share our findings.
We studied the disciplinary distribution of the authors who published MOOC research in 2013-2015 and compared it to the submissions to the MOOC Research Initiative (MRI) reported in Gašević et al., (2014). Our tests showed that the MOOC literature published in 2013-2015 differed significantly from the MRI submissions: our corpus had a greater representation of authors from Computer Science and the Gašević et al., corpus had a greater representation of authors from Education and Industry. In other words, our corpus was less dominated by authors from the field of education than were the MRI submissions. One of Downes criticisms is the following: “the studies are conducted by people without a background in education.” This finding lends some support to his claim, though a lot of the research on MOOCs is from people affiliated with education, but to support that claim further one could examine the content of this papers and identify whether an educational theory is guiding their investigations.
We also compared author affiliation information in our papers with the papers used in Liyanagunawardena et al.’s (2013) review of the 2008-2012 MOOC literature. We found that the two samples differed significantly. For example, the Liyanagunawardena et al. corpus was relatively over-represented in the Independent Researcher category. This result suggests that the bodies of literature published in 2008-2012 and 2013-2015 differ in significant ways. This may or may not hold true for the writing that has occurred in blogs, unpublished reports etc. We don’t know and we haven’t studied that.
Finally, and most significantly, we found that the disciplinary makeup of the literature is changing over time: there’s greater interdisciplinary activity in MOOC research now than in the past. This result is very interesting and its implications are worth examining. Suffice to say that this provides opportunities (can we capitalize on the expertise of one another to improve education?) and challenges (are newcomers to the field capitalizing on what we now about the science of learning?). The move to greater inter- and cross- disciplinarity in the field is evident in other initiatives. See for example, the Digital Learning Research Network.
Keep in mind that this research faces some of the same limitations raised by Downes (i.e. like Siemens, our inclusion criteria mean that some research is included while other is excluded). However, it also addresses some of those criticisms. For example, we tried to verify whether some of our results could have arisen by chance by running 10,000 computer simulations on the samples. The computer is confident that they could not have arisen by chance.
I’m hoping this paper will be out of peer-review soon so that I can share, but I’m thankful to the editor that allowed us to share our findings.
The couches of strangers, and three perspectives on the relationship between social media and scholarship
The thought of spending a night on a stranger’s couch many elicit apprehension and concern. The thought of spending time online may elicit many trepidation for scholars. Scholars are worried about the time commitment of such activities when universities may not value them. Scholars may also be concerned about personal-professional boundaries. Both couchsurfing and networked scholarship offer opportunities for growth as well: couchsurfing may allow people from different cultures to get to know one another; networked scholarship might allow scholars from disparate disciplines to meet and collaborate. Alternatively, both activities may have relatively mundane outcomes: sleeping on a stranger’s couch does not necessarily mean that one will have a life-changing experience, in the same way that going online does not mean that one will find a welcoming and supportive scholarly community. And engagement with couchsurfing or networked scholarship may require certain literacies for successful participation.
The practice of networked scholarship isn’t without perils. While advocacy for open, social, and digital scholarship features prominently in the literature (Kimmons, 2014), the reality on the ground is that scholars’ activities on social media are both exceptional and mundane, and their experiences are inspiring and harrowing – but above all, such experiences are neither universal nor pre-determined.
Siemens and Matheos (201X) argued that educational institutions reflect the societies which house them: as societies change, so do their educational institutions and the scholarly practices that they support and encourage. As social media and openness become increasingly popular, sharing economies gain hold, and online networks permeate every aspect of life, the scholarly enterprise and the work that educators and researchers do is experiencing social, cultural, and technological tensions to change. However, we should be careful in our attribution of causality. Academics may have always wanted to share more freely, connect in better ways, and social media simply supported that desire.
At the same time, we should be weary of the perspective that technologies are neutral tools that merely respond to the needs of users. Technologies have assumptions and worldviews embedded in their design that shape the experiences and behaviours of their users. The algorithms used by Facebook to deliver tailored timelines and the recommender systems used by Amazon are representative examples of the ways that technologies are influenced by their developers worldviews.
Thus, the relationship between academic practices and technologies is negotiated and complex. It can be seen via three perspectives.
The first perspective suggests that social media (and their design and affordances) shape scholarship and participation. This is the technological deterministic perspective that is often revealed in narratives pertaining to social media having an impact on scholarship. Institutional encouragement to use social media to increase scholarly reach and citations falls under this perspective.
The second suggests that teaching and scholarship (and the structures, rewards, practices of academia) shape how social media are used. This perspective reflects a social shaping of technology approach. This perspective recognizes that networked scholarly practices are shaped by social, cultural, economic, and political factors, rejecting the notion that technologies (and practices) are deterministic.
The third perspective is an extension of the second and anticipate that academics adapt and appropriate social media to fulfill personal and professional desires and values. This perspective holds that, with adequate information and evidence, learners, instructors, and researchers have the agency to accept or reject any particular technology or to find alternative uses for it that will better serve their needs. Such agency is recognized in scholars’ strategic uses of technology in scholarship broadly, and in teaching and research in particular.
These three perspectives are often unstated, but permeate the literature and conversation pertaining to social media use in education and scholarship.
I am editing, revising, and re-writing various parts of my book, Networked Scholars. The current draft of the introduction is as follows:
Introduction: What does couchsurfing have to do with networked scholarship?
It’s 2005. I’m a graduate student at the University of Minnesota in Minneapolis.
Though I don’t always enjoy driving, I am particularly fond of road trips and the opportunities they provide for learning about the world. That summer, my girlfriend and I decided to take a road trip from Minnesota to Tennessee following the roads that run parallel to the Mississippi River.
A few months prior to this trip I discovered Couchsurfing.com and this site provided one of my early experiences with modern social media
“Couchsurfing” refers to spending a night or two on the couch of a stranger. The website facilitated interactions between individuals who were interested in hosting others and individuals who were interested in spending nights on someone else’s couch.
Given my mild disdain of driving long distances, we divided our trip in such a way so as to limit daily driving to 4 to 6 hours. We arranged for accommodations driving South toward Tennessee because we wanted to follow the Great River Road, but we decided to return back to Minneapolis via a different route so as to visit more states.
Using couchsurfing.com, we found two individuals who were willing to host us on their couch. One in Diamond City, Arkansas and one in Kansas City, Kansas.
An elderly couple was willing to let us sleep on their couch in Diamond City, Arkansas. We knew nothing about Diamond City, but once we arrived we learned that Diamond City is a small retirement town, seemingly a destination only for those visiting their parents who retire there.
We arrived in Diamond City around 5pm. We stopped at one of the two local diners prior to joining our hosts so as to have an early dinner and pre-empt being a burden to our hosts. Our hosts were welcoming and friendly, and we spend the next few hours in their living room getting to know each other.
In getting to know each other, we learned that our hosts were as unsure as we were of this arrangement. We were the first people who reached out to them for hosting, and, while they were apprehensive at first, they decided to host us, as we seemed to be “just a couple of kids from the Midwest.”
Eventually we made our way to the porch. We continued our conversation, enjoying a beautiful August night, when out of nowhere, 10 or so people joined us there. They brought their chairs and musical instruments. We quickly learned that these were our hosts friends who were preparing a “hootenanny” for us. A hootenanny – an informal gathering involving folk music and dancing- was as foreign to me as I was to them. The spent many hours at the hootenanny. We danced, we learned about life in retirement towns, and discussed the demise of small-town America.
Our hosts generosity and hospitality stayed with me. These individuals were not only willing to make themselves vulnerable and share their house with us for the night, but they also went out of their way to organize a hootenanny and share this aspect of their life with us. The next morning, a group of our hosts’ friends prepared breakfast, and we left for our next couchsurfing destination: Kansas City, Kansas.
Dave, our gracious Kansas City host, invited his friends to have dinner and drinks with us, gave us a tour of Kansas City, and offered us his bed instead of the couch: He had painted his living room the night before and the smell of paint was still lingering. “Guests don’t need to be exposed to toxic fumes” he argued.
When I share these two stories with others, I often face puzzling looks and questions regarding my sanity. “What if your host was an axe murderer?” someone asked me once. “And you really don’t know these people?” is another question I am asked often. While the idea of sleeping at a stranger’s couch may not be appealing to everyone, and may sound a bit too trusting, all my limbs are still intact.
But more importantly, these stories demonstrate the power and potential of networks and openness. Supported by the Web as a social platform that allows individuals to consume, produce, remix, and contribute content without the need for specialized technical know-how, networks and openness question scholars’ foundational activities and assumption. This is one of the main ideas underpinning this book.
Social media have become part of the fabric of contemporary societies and our educational systems. Worldwide experiences with social media suggest that these technologies are actively transforming various aspects of common culture in both intentional and unexpected ways. For instance, as news agencies adopt social media to engage audiences and increase revenues, they are simultaneously reporting on instances where the use of social networking sites has led to harm. Social media have also penetrated the higher education sector, and have influenced not only the ways students connect with each other, but also the ways scholarship is organized, delivered, enacted, and experienced (Weller, 2011). Recent reports note that YouTube and Facebook are popular sites for academics with “over 90% of [2,000 faculty surveyed were] using social media in courses they’re teaching or for their professional careers outside of the classroom” (Moran, Seaman, Tinti-Kane, 2011). Open practices in a variety of educational and scholarly settings have also gained wide interest and attention in recent years (Wiley, 2006; Wiley & Hilton, 2009). Proponents of openness claim that open practices may “broaden access to education and knowledge, reduce costs, enhance the impact and reach of scholarship and education, and foster the development of more equitable, effective, efficient, and transparent scholarly and educational processes” (Veletsianos & Kimmons, 2012).
This book differs from other books focusing on social media in education and digital scholarship in a number of significant ways.
First, this is not a how-to book. You will find no advice in this book about how to use social media to share your research, engage students, or increase your citations.
Second, this book does not advocate for networked scholarship. My goal is to understand and problematize the concept of networked scholarship and its implications.
This book focuses upon scholars’ experiences and activities online because the ways that social media are used and experienced by academics are not well understood and the evidence describing the experiences of scholars engaged in scholarly practices online is both limited and fragmented.
A number of high-profile news items illustrate the tensions and diverse opinions that exist around this topic, as well as the strong emotions attached to professors’ online participation:
- “If you’re a professor in Kansas, better stay off the Internet,” proclaims the byline on a December 2013 magazine article, following the Kansas Board of Regents decision to adopt guidelines regarding “improper use of social media” that followed a professor’s Twitter update.
- Elsevier is cracking down on professors who share their research online in violation of the copyright agreements that they signed, and professors are striking back, promoting boycotts of Elsevier’s academic journals.
- While some university administrators, like the President of the University of British Columbia might “despise” social media and consider Twitter to be “one of the worst things” created in his lifetime, some academics have found community and solidarity in social media spaces.
Although the statistics regarding scholar participation on social media are worthwhile to examine, the stories behind the statistics are much more interesting and meaningful. Even in the cases where news articles present stories regarding academics’ social media participation, these fall short of the intricate tensions that arise in networked practice. For example, in addition to promoting boycotts of Elsevier numerous academics are refusing to review for publishers who do not provide open access to their work, and, more interestingly, are using networked technologies to share copyrighted materials with each other. And even though social networking sites are disparaged by some, academics have used these places to develop legitimate online learning communities, launch research projects, and share intimate details of their lives (e.g., their struggles with debilitating diseases) – converting places like Twitter into networks of care and bonding. Alternatively, even though social media allow widespread access, issues like power and influence may mediate participation, calling into doubt the democratizing potential of these technologies.
I used Couchsurfing.com once more when I was living in the UK in 2009. I was going to spend a few days in Faro, Portugal and had sent the following note to my potential host:
I am writing in the hopes that you will be interested in hosting me on your couch for 2 nights. I can see from your profile that it’s your birthday on May 2nd, and I would completely understand if you have something else planned.
A little bit about me: I live in the UK and moved here to teach at the University of Manchester. I lived in the United States (Minneapolis) for 8 years, but I am originally from Cyprus. I speak Greek and English and love to explore and travel, learn about different cultures, photography, music, beer, and good company. I am very friendly and considerate and am looking forward to visiting Portugal for the first time!
Please look at my profile and if there’s anything that you would like to know, please feel free to ask!
Within a few hours, Eric responded and noted that he had birthday plans and I should join. Specifically, he and his friends were having a party on an island off the coast of Faro and I should join them. The party was starting on the 1st and ending on the 3rd and long as I didn’t mind “sleeping on the floor or even on the beach” Eric was happy to host me.
Next… tying it all together.