Professor & Canada Research Chair in Innovative Learning and Technology at Royal Roads University

Category: sharing

Using an instructional design perspective to analyze MOOC materials

Posted on December 13th, by George Veletsianos in courses, moocs, open, scholarship, sharing. 19 comments

A facebook conversation from yesterday encouraged me to share one of the assignments that I developed for my instructional design course. The goal of the class is for the students to understand, experience, and apply instructional design in a variety of educational contexts.

One of the assignments I developed for asked students to enroll in a Massive Open Online Course (MOOC) and analyze the instructional materials within the course using one of the rubrics provided by Dick and Carey (the instructional design book we use in class). It was a lot of fun and the students appreciated the exercise. Given the lack of presence and voice by instructional designers in MOOC happenings, the lack of valid, reliable, and serious research that exists on the topic (though Rita Kop’s work on cMOOCs is admirable), and my desire to engage students in contemporary events, I came up with this assignment to embed MOOC analysis in my course. The assignment is available for download on https://dl.dropbox.com/u/2533962/instr-materials-veletsianos.doc and posted below for those who just want to skim it without downloading it. Enjoy and feel free to use it:

Instructional Material analysis assignment

Individually, you will examine and report on the instructional materials of one popular digital learning initiative. An analysis matrix will be provided to you, and you will use that to matrix to evaluate these initiatives.

Length: Minimum 500 words.

Criteria Levels of Attainment Points
Written analysis (evaluation)
  • Evaluation adheres to the matrix, is thoughtful, and presents evidence of original thought
  • Evaluation does not adhere to the matrix or is superficial on various levels
87-0
Rubric completion
  • Learner completes and submits the rubric for evaluating instructional materials (p. 250-251) for his/her selected initiative.
2

 

This task requires a few hours of research before you can actually complete it. Even though this is an individual task, if you would like to discuss the assignment with any of your colleagues, please feel free to do so.

Mechanics

First read the chapter and the rest of the materials for this week. Without reading those, I can assure you that your understanding of the issues presented will be superficial.

Second, examine the rubric provided by Dick & Carey for evaluating instructional materials (p. 250-251 – see below for the rubric). You will be completing this rubric for a digital environment, and it’s a good idea to understand what it encompasses before you proceed.

Third, select one course provided on one of the following platforms to examine:

  • A course on Coursera (select a course that is occurring right now or has been completed. DO NOT select a course that has not started yet): https://www.coursera.org/courses
  • A course on EdX (select a course that is occurring right now. DO NOT select a course that has not started yet): https://www.edx.org/courses

You can also choose to examine DS106: http://ds106.us/ I am including DS106 on its own because it is a course as opposed to the above (Coursera, EdX, and Udemy) which are platforms. If you pick any of these three (Coursera, EdX, or Udemy), then you should also pick a course (e.g., Within Coursera a possible course is https://www.coursera.org/course/friendsmoneybytes).

Assignment

Once you have made your selection, it’s time to research your course. Spend time looking around, examining and evaluating the instructional materials provided. You will use the rubric to keep track of the criteria that need to be assessed, and then using this rubric you will write a report assessing the instructional material for the course.

You should start your report by stating the course and its provider. A link would also be helpful. For example, using the example above, I would start my report by stating the following:

“I am examining the course entitled Networks: Friends, Money and Bytes (https://www.coursera.org/course/friendsmoneybytes). This course if offered through Coursera and is taught by Mung Chiang who is a Professor or Electrical Engineering at Princeton University. The course is an introduction to the topic of X and its objectives are XYZ.”

Your report should be specific and detailed in its evaluation of instructional material, and should be guided by the five criteria families discussed by DC: Goal-centered, learner-centered, learning-centered, context-centered, technical criteria. I would like to see that you understand each criterion and that you are capable of applying it to evaluating your course. For example, at the very least, I would expect to see statements such as the following:

Instructional designers use five criteria families to evaluate instructional materials. Learner-centered criteria focus on XYZ and refer to X. The instructional materials for this course appear to be adequate for this criterion because <provide list of reasons here>. The course could be improved in this domain by <list of additions/revisions here>. However, because item X was not disclosed in the course, I am not able to evaluate Y.

Let me reiterate that to complete this assignment you will need to do background research on the course and the platform. For example, your background research on Coursera will reveal that some of these courses have more than 80,000 students from around the world. This fact alone will impact your evaluation!

Instructional Material Evaluation Rubric

Rubric is copyright of: Dick, W., Carey, L. & Carey, J. (2008). Systematic Design of Instruction, (7th ed.) Upper Saddle River, NJ: Pearson.

A. Goal-centered Criteria:
Are the instructional materials:

Yes No Some
1. Congruent with the terminal and performance objectives?
2. Adequate in content coverage and completeness?
3. Authoritative?
4. Accurate?
5. Current?
6. Objective in presentations (lack of content bias)?

 

Learner-centered Criteria:
Are the instructional materials appropriate for learners’:
Yes No Some
1. Vocabulary and language?
2. Development level?
3. Background, experience, environment?
4. Experiences with testing formats and equipment?
5. Motivation and interest?
6. Cultural, racial, gender needs (lack bias)?

 

Learning-centered criteria
Do the material include:
Yes No Some
1. Pre-instructional material?
2. Appropriate content sequencing?
3. Presentations that are complete, current and tailored for learners?
4. Practice exercises that are congruent with the goal?
5. Adequate and supportive feedback?
6. Appropriate assessment?
7. Appropriate sequence and chunk size?

 

Context-centered Criteria
Are/do the instructional materials:
Yes No Some
1. Authentic for the learning and performance sites?
2. Feasible for the learning and performance sites?
3. Require additional equipment/tools?
4. Have congruent technical qualities for planned site (facilities/delivery system)?
5. Have adequate resources (time, budget, personal availability and skills)?

 

Technical criteria
Do the instructional materials have appropriate:
Yes No Some
1. Delivery system and media for the nature of objectives?
2. Packaging?
3. Graphic design and typography?
4. Durability?
5. Legibility?
6. Audio and video quality?
7. Interface design?
8. Navigation?
9. Functionality?

Papers and quotes relating to emerging technology, universities, and reform

Posted on December 12th, by George Veletsianos in sharing. 13 comments

As someone who deeply values teaching and learning, I have been reading a number of articles and books that explore the role, values, responses, and potential of universities in the face of calls for reform (both market reform and otherwise). The role of emerging technologies is a key area of interest for me in this discussion. Below are interesting articles and quotes that are worthy of reflection and further investigation:

Blum, D., & Ullman, C. (2012). The globalization and corporatization of education: the limits and liminality of the market mantra. International Journal of Qualitative Studies in Education, 25(4), 367–373.

Burka, P. (2012). Storming the Ivory Tower. The Texas Monthly.  Retrieved on November 12, 2012 from http://www.texasmonthly.com/preview/2012-10-01/feature

Schwier, R. (2012). The corrosive influence of competition, growth, and accountability on institutions of higher education. Journal of Computing in Higher Education, 24(2), 96–103.

The Burka article is a fantastic summary of what transpired in Texas during 2011-2012 with regards to reform initiatives, with a focus on the University of Texas at Austin. Blum and Ullman (2012) introduce a special journal issue focused on neoliberalism and education. Schwier (2012) explores how competition, growth, and accountability might interact with universities, and what relationship they might have with instructional design and technology.

Quotes:

“What do the forces of competition, growth, and accountability have to do with instructional design and technology? I think we are complicit in perpetuating these agendas in higher education. We have long been associated with improved teaching and learning efficiency, and of course distance learning is seen as a tool for competing successfully with other institutions…And distance learning supports an accountability agenda because it is, in many cases, seen as a product… Consequently we help our institutions treat education as a commodity—as credit units sold for dollar amounts. Is it any wonder we hear complaints about students who act like demanding customers?” (Schwier, 2012, pp. 101)

“And instructional design and technology has more to offer on the quality side of the ledger than on the efficiency side. We understand more today about what it means to build exciting and successful learning environments than we ever have before, and we understand how to take advantages of the affordances of emerging technologies. But our designs do not always result in greater efficiency; nor should they.” (Schwier, 2012, pp. 102).

“Instructional design and technology professionals need to be able to argue for vibrant teaching and learning environments, to promote strong cultures of teaching and learning in our institutions, and not act as content hacks or live in fear of failing to measure up to externally defined efficiency criteria.” (Schwier, 2012, pp. 102)

“In the neoliberal state, healthcare and education have been transformed through the market mantra, into products that individuals can buy and sell.” (Blum & Ullman, 2012, pp. 368)

‘While the liberal state only slightly lessened the role of ascribed status (the idea that people succeed both as individuals and as groups, through the status ascribed to them through their race, gender, or class status), the reality is that the twentieth century has been character- ized in the Global North by a combination of achievement and ascription, making success about both “who you know and where you are from” as well as “pulling yourself up by the bootstraps”.’ (Blum & Ullman, 2012, pp. 368)

“Schools have accepted the bulk of the blame for our economic problems despite the reality that educators have been virtually disempowered at every level. The truth is that our worldwide economic problems have little to do with the school-based preparation of human capital, but instead are deeply tied to the limits of market capitalism.” (Blum & Ullman, 2012, pp. 372)

Digital Lifestyles event at Hill Country Middle School

Posted on December 3rd, by George Veletsianos in sharing. 3 comments

I love spending time with others and participating in conversations that help others understand and explore the role of technology in education, society, and 21st century living. On Thursday (Dec 6th), I get to do that at a local middle school as part of the Digital Parent learning series, an event hosted by Carl Hooker who is the Director for Instructional Technology in the Eanes Independent School District. I will be talking about participation in online spaces and networked cultures and why and how parents can support and foster their children’s online participation. I anticipate that I will have to defend the notion of real and virtual life being inextricably intertwined and, to a large degree, inseparable (which is an argument that resembles one that I made in 2008 and one that Nathan Jurgenson recently described quite eloquently). I also look forward to hearing and learning from my co-panelists’ experiences!

If you are in Austin, please join us for this free event.

Educational Technology and Related Education Conferences January-June 2013

Posted on November 26th, by George Veletsianos in sharing. 7 comments

The last version of Clayton Wright’s list of educational technology conferences that I posted on this blog was for January to June 2011.  Clayton has once again provided the community with an updated (extensive) list of educational technology conferences for the upcoming 6 months (Jan-Jun 2013): Clayton Wright Educational Technology and Education Conferences January to June 2013 (.doc)

MOOCs, credit, accreditation, and narratives

Posted on November 20th, by George Veletsianos in moocs, online learning, sharing. 22 comments

I’m working through my thoughts with this blog entry, as I’ve been trying to use this space to think out loud about my work and what I see happening in online education and higher ed.

A lot has been written about MOOCs and accreditation, and a lot more will be forthcoming. For example, see Terry Anderson’s post on this.

Today, I run across this quote in an article at Time Magazine:

…if Liu passes the graduate-level Harvard course she is taking for free through edX — one of the leading providers of massive open online courses, or MOOCs — she will be granted 7.5 credit hours, which her school district has agreed to accept as a form of professional development that can help her earn a higher salary. Liu might be among the first students nationwide to turn free online coursework into tangible college credit, but that number may soon grow exponentially.
Critical educators have done a good job on exposing systems of oppression and unequal distribution of power that impoverish learning experiences. I believe that such a lens is increasingly important in the work of any researcher and educator thinking about the future of education. To illustrate, the description above is not just a narrative of the success of open education. It’s also a narrative of moocs  “carving new markets” rather than innovating the way higher education functions for the masses of people that could not have attained a degree in the first place. I think that we need to keep an open mind with regards to the potential, as well as the aims and pitfalls, of such initiatives. To explore a different perspective, I suggest that you read Richard Hall’s analysis on how the profit motive is threatening higher education.
Contrast this with the TechCrunch perspective that  “the school system, as we know it, is on the verge of extinction”as “it’s inevitable that online courses will in one way or another replace schools.”  The question to ask here is not whether this prophecy will come true. We know that it won’t because universities are valued social institutions that are embedded in the culture of their times, and even though they may change, they won’t disappear. An analysis of educational technology predictions of the past also shows that hype is rarely realized (pdf). What is important to ask however is this: Who benefits from the narrative of “extinct schools?” Is it the student? The edtech startups? The investors?

 

Assumptions and Challenges of Open Scholarship

Posted on November 1st, by George Veletsianos in emerging technologies, moocs, my research, NPS, open, papers, scholarship, sharing. 15 comments

What is the value of a critique?

The value of critique is to help us see a phenomenon through a different lens, to help us make sense of something in a different way, and to spark a conversation. This is the purpose, and value, of a paper we recently published with IRRODL on the topic of open scholarship.

The paper identifies the assumptions and challenges of openness and open scholarship and attempts to put forward suggestions for addressing those. A summary of our paper, appears below:

Many scholars hope and anticipate that open practices will broaden access to education and knowledge, reduce costs, enhance the impact and reach of scholarship and education, and foster the development of more equitable, effective, efficient, and transparent scholarly and educational processes. Wiley and Green (2012, pp. 88) note that “only time will tell” whether practices of open scholarship will transform education or whether the movement “will go down in the history books as just another fad that couldn’t live up to its press.” Given the emerging nature of such practices, educators are finding themselves in a position in which they can shape and/or be shaped by openness (Veletsianos, 2010). The intention of this paper is (a) to identify the assumptions of the open scholarship movement and (b) to highlight challenges associated with the movement’s aspirations of broadening access to education and knowledge. The goal of this paper is not to frame open scholarship as a problematic alternative to the status quo. Instead, as we see individuals, institutions, and organizations embrace openness, we have observed a parallel lack of critique of open educational practices. We find that such critiques are largely absent from the educational technology field, as members of the field tend to focus on the promises of educational technologies, rarely pausing to critique its assumptions. Selwyn (2011b, pp. 713) even charges that our field’s inherent positivity “limits the validity and credibility of the field as a site of serious academic endeavour.” Our intention is to spark a conversation with the hopes of creating a more equitable and effective future for digital education and scholarship. To this end, this paper is divided into three major sections. First, we review related literature to introduce the reader to the notion of open scholarship. Next, we discuss the assumptions of openness and open scholarship. We then identify the challenges of open scholarship and discuss how these may limit or problematize its outcomes.

Common assumptions and challenges are summarized as follows:

Common themes and assumptions Challenges
Open scholarship has a strong ideological basis rooted in an ethical pursuit for democratization, fundamental human rights, equality, and justice. Are these ideals essential components of the open scholarship movement or are merely incidental to those who are pioneering the field?
Open scholarship emphasizes the importance of digital participation for enhanced scholarly outcomes Scholars need to develop an understanding of participatory cultures and social/digital literacies in order to take full advantage of open scholarship.Need to redesign university curricula to prepare future scholars to account for the changing nature of scholarship.

 

Open scholarship is treated as an emergent scholarly phenomenon that is co-evolutionary with technological advancements in the larger culture Technology both shapes and is shaped by practice.Technology is not neutral, and its embedded values may advance tensions and compromises (e.g., flat relationships, homophily, filter bubbles).
Open scholarship is seen as a practical and effective means for achieving scholarly aims that are socially valuable Open scholarship introduces new dilemmas and needs (e.g., personal information management challenges; Social stratification and exclusion).

Given the topic, the best home for this paper was the International Review Of Research In Open And Distance Learning, through which you can download the paper for free in an open access manner:

Veletsianos, G. & Kimmons, R. (2012). Assumptions and Challenges of Open Scholarship. The International Review Of Research In Open And Distance Learning,13(4), 166-189. [HTML access or PDF access]

 

Learning Designer job posting for edX

Posted on October 26th, by George Veletsianos in sharing. 5 comments
My colleague Dan Surry shared this job posting on Twitter, and I thought I’d repost it here. A positive outcome of the booming interest in online education and educational technology are the jobs that will become available in the industry. I am happy to see this job posting because the lack of educators and learning/instructional designers in these initiatives is unfortunate. At the very least, input from instructional designers can help these initiatives avoid the pitfalls of the past and assist educational technology innovators think beyond efficiency and consider effectiveness, engagement, and social justice.
Title: Learning Designer
Req Number: mit-00009144
Department: edX
FT/PT: Full Time
Employment / Payroll Category: Administrative

LEARNING DESIGNER, edX, to plan, develop, and deliver highly-engaging and media rich online courses as part of the content and development team. Will determine and apply sound pedagogical strategies to unique situations and a diverse set of academic disciplines. Responsibilities include working with producers, product developers, and course staff on implementing instructional design approaches in the development of media and other course materials; articulating learning objectives and aligning them to content design strategy and assessments; writing effective instructional text and audio and video scripts; coordinating workflows with video and content development team; identifying and sharing best practices; creating course communication style guides; developing use case guides; serving as liaison to instructional design teams located at X universities; designing peer review processes; applying game-based learning theory and design to selected courses; using learning analytics and metrics to inform course design and revision process; working closely with the content research director on articulating best practices for MOOC teaching and learning and course design; and assisting in the development of pilot courses.

REQUIREMENTS: a master’s in educational technology, instructional design, or related field; excellent interpersonal, verbal and written communication, project management, problem solving, and time management skills; flexibility; ability to work on multiple projects/courses, meet deadlines, and manage expectations; capacity to develop new and relevant technology skills; experience using game theory design and learning analytics to inform instructional design decisions and strategy; and experience with video and screencasting, LMS platform, XML, HTML, CSS, Adobe Design Suite, Camtasia or Captivate, and web 2.0 collaboration tools. Experience in higher education and in a start-up or research environment desirable. MIT-00009144-P

Application Information

Contact: Massachusetts Institute of Technology
Online App. Form: http://sh.webhire.com/servlet/av/jd?ai=631&sn=I&ji=2648861

A list of educational technology academic journals

Posted on October 10th, by George Veletsianos in sharing. 46 comments

My colleague Jon Becker posted the following question on Twitter and brought it to my attention:

I find that the decision of where to publish one’s work requires a lot of thought. Issues to consider include open access, readership, reputation, audience, institutional norms/expectations, and perceived fit. For those of you interested in only open access journals, you can visit the list of open access educational technology journals that I put together and have been crowdsourcing since 2009.

Given the wide range of factors that one needs to consider, this list is incomplete (which is why Jon clarified that he is looking for more than the journals in the open access list). I personally have published in the following discipline-specific journals (at times more than once), and consider them as worthwhile outlets:

Computers in Human Behavior
Computers & Education
Australasian Journal of Educational Technology
The International Review Of Research In Open And Distance Learning
Journal of Computer Assisted Learning
Contemporary Issues in Technology and Teacher Education
The Internet and Higher Education
British Journal of Educational Technology
Quarterly Review of Distance Education
Distance Education
Journal of Research on Technology in Education
Journal of Computing in Teacher Education
Interacting with Computers
Educational Technology
Journal of Interactive Learning Research
Journal of Educational Computing Research

Again, this list is incomplete. For instance, I want to publish a practitioner-oriented piece with Tech Trends, and haven’t yet got around to do that.

And if one needs more options, my colleagues and I at the Learning Technologies program recently compiled a list of journals to help us navigate this process. That list consists of the following:

Educational Technology Research & Development
Journal Of Educational Computing Research
Journal Of Research On Technology In Education
Computers & Education
British Journal Of Educational Technology
Computers In Human Behavior
The Internet And Higher Education
Journal Of Computer Assisted Learning
Learning Media And Technology
Australasian Journal Of Educational Technology
Educational Technology & Society
Interactive Learning Environments
Research In Learning Technology
Journal Of Interactive Learning Research
Journal Of Educational Technology Systems
Computers In The Schools
American Journal Of Distance Education
Interdisciplinary Journal Of Problem-Based Learning
Journal Of Technology And Teacher Education
International Review Of Research In Open And Distance Research.
Distance Education
Journal Of Distance Education
Journal Of Educational Multimedia And Hypermedia
International Journal On E-Learning
Contemporary Issues In Technology & Teacher Education (CITE)