A facebook conversation from yesterday encouraged me to share one of the assignments that I developed for my instructional design course. The goal of the class is for the students to understand, experience, and apply instructional design in a variety of educational contexts.

One of the assignments I developed for asked students to enroll in a Massive Open Online Course (MOOC) and analyze the instructional materials within the course using one of the rubrics provided by Dick and Carey (the instructional design book we use in class). It was a lot of fun and the students appreciated the exercise. Given the lack of presence and voice by instructional designers in MOOC happenings, the lack of valid, reliable, and serious research that exists on the topic (though Rita Kop’s work on cMOOCs is admirable), and my desire to engage students in contemporary events, I came up with this assignment to embed MOOC analysis in my course. The assignment is available for download on https://dl.dropbox.com/u/2533962/instr-materials-veletsianos.doc and posted below for those who just want to skim it without downloading it. Enjoy and feel free to use it:

Instructional Material analysis assignment

Individually, you will examine and report on the instructional materials of one popular digital learning initiative. An analysis matrix will be provided to you, and you will use that to matrix to evaluate these initiatives.

Length: Minimum 500 words.

This task requires a few hours of research before you can actually complete it. Even though this is an individual task, if you would like to discuss the assignment with any of your colleagues, please feel free to do so.

Mechanics

First read the chapter and the rest of the materials for this week. Without reading those, I can assure you that your understanding of the issues presented will be superficial.

Second, examine the rubric provided by Dick & Carey for evaluating instructional materials (p. 250-251 – see below for the rubric). You will be completing this rubric for a digital environment, and it’s a good idea to understand what it encompasses before you proceed.

Third, select one course provided on one of the following platforms to examine:

• A course on Coursera (select a course that is occurring right now or has been completed. DO NOT select a course that has not started yet): https://www.coursera.org/courses

• A course on EdX (select a course that is occurring right now. DO NOT select a course that has not started yet): https://www.edx.org/courses

• A free course on Udemy (select a course that includes at least 5 “lessons/lectures”): http://www.udemy.com/courses

You can also choose to examine DS106: http://ds106.us/ I am including DS106 on its own because it is a course as opposed to the above (Coursera, EdX, and Udemy) which are platforms. If you pick any of these three (Coursera, EdX, or Udemy), then you should also pick a course (e.g., Within Coursera a possible course is https://www.coursera.org/course/friendsmoneybytes).

Assignment

Once you have made your selection, it’s time to research your course. Spend time looking around, examining and evaluating the instructional materials provided. You will use the rubric to keep track of the criteria that need to be assessed, and then using this rubric you will write a report assessing the instructional material for the course.

You should start your report by stating the course and its provider. A link would also be helpful. For example, using the example above, I would start my report by stating the following:

“I am examining the course entitled Networks: Friends, Money and Bytes (https://www.coursera.org/course/friendsmoneybytes). This course if offered through Coursera and is taught by Mung Chiang who is a Professor or Electrical Engineering at Princeton University. The course is an introduction to the topic of X and its objectives are XYZ.”

Your report should be specific and detailed in its evaluation of instructional material, and should be guided by the five criteria families discussed by DC: Goal-centered, learner-centered, learning-centered, context-centered, technical criteria. I would like to see that you understand each criterion and that you are capable of applying it to evaluating your course. For example, at the very least, I would expect to see statements such as the following:

Instructional designers use five criteria families to evaluate instructional materials. Learner-centered criteria focus on XYZ and refer to X. The instructional materials for this course appear to be adequate for this criterion because <provide list of reasons here>. The course could be improved in this domain by <list of additions/revisions here>. However, because item X was not disclosed in the course, I am not able to evaluate Y.

Let me reiterate that to complete this assignment you will need to do background research on the course and the platform. For example, your background research on Coursera will reveal that some of these courses have more than 80,000 students from around the world. This fact alone will impact your evaluation!

Instructional Material Evaluation Rubric

Rubric is copyright of: Dick, W., Carey, L. & Carey, J. (2008). Systematic Design of Instruction, (7th ed.) Upper Saddle River, NJ: Pearson.

A. Goal-centered Criteria:
Are the instructional materials:

Reddit is one of the communities that I follow for professional and personal purposes. For professional purposes specifically, it serves as a site for my online ethnography on networked participatory scholarship and digital scholarship. As part of that work, I am trying to make sense of the meaning of open digital participation for learning, teaching, scholarship, and education. One of the most informative and enjoyable aspects of Reddit is the IAmA subreddit in which individuals with interesting stories answer user questions. For example, one individual shared intimate details of his/her work and experiences with for-profit education, and another discusses teaching high school science and the misconceptions surrounding the teaching profession. The other day, Salman Khan (of Khan Academy) answered questions on the site, and I thought some of you might be interested in reading the Q&A, not just for Khan’s answers but also for the types of questions that were being asked. Though my vision of education differs from Khan’s vision of education, I appreciate that numerous students and teachers have found value in his efforts and I welcome any initiative that opens up conversations about what the future of education should look like. In any event, here is the Q&A with Salman Khan.

* Reddit logo courtesy of Wikipedia: https://en.wikipedia.org/wiki/File:Reddit_logo.svg

One thing that I don’t usually post on this blog is information related to my research on pedagogical agents and virtual characters, which is one of the research strands that I’ve followed for the past 4 years. I am breaking away from that mold by posting this note : )

Specifically, my colleagues (Aaron Doering and Charles Miller) and I developed a research and design framework to guide smooth, natural, and effective communication between learners and pedagogical agents. Our reasons for developing this framework were varied, but after four years of research and design in the field, I became convinced that to push the field forward, we needed guidance. I use the word “guidance” as opposed to the words “rules” or “laws” because we “anticipate that designers, researchers, and instructors will adapt and sculpt the guidelines of the EnALI framework into their unique instructional contexts, ultimately kindling future research and design that will expand the framework foundations.”

The framework (called Enhancing Agent Learner Interactions or EnALI) is grounded on three major theories: socio-cultural notions of learning, cooperative learning, and conflict theory. In this, we have tried to bring a humanist perspective and encourage designers and researchers to move beyond the use of pedagogical agents as dispassionate tools delivering pre-recorded lectures… but I’ll save that information for a different post. The paper is to appear in the Journal of Educational Computing Research as: Veletsianos, G., Miller, C., & Doering, A. (2009). EnALI: A Research and Design Framework for Virtual Characters and Pedagogical Agents. Journal of Educational Computing Research, 41(2), 171-194 [email me for a preprint].

The framework is posted below, but if you want a full explanation of the guidelines, please refer to the paper. As always questions, comments, and critique are appreciated:

1. Pedagogical Agents should be attentive and sensitive to the learner’s needs and wants by:

• Being responsive and reactive to requests for additional and/or expanded information.
• Being redundant.
• Asking for formative and summative feedback.
• Maintaining an appropriate balance between on- and off-task communications.

2. Pedagogical Agents should consider intricacies of the message they send to learners by:

• Making the message appropriate to the receiver’s abilities, experiences, and frame of reference.
• Using congruent verbal and nonverbal messages.
• Clearly owning the message.
• Making messages complete and specific.
• Using descriptive, non-evaluative comments.
• Describing feelings by name, action, or figure of speech.

3. Pedagogical Agents should display socially appropriate demeanor, posture, and representation by:

• Establishing credibility and trustworthiness
• Establishing role and relationship to user/task.
• Being polite and positive (e.g., encouraging, motivating)
• Being expressive (e.g. exhibiting verbal cues in speech).
• Using a visual representation appropriate to content.