Category: teacherbots

Playing with prompt engineering in response to Hickey & Luo prompt engineering analysis

In an worthwhile and expansive post Daniel Hickey and Qianxu Morgan Luo explore knowledge-rich prompts and the ways in which ChatGPT and Bard differ in the results they return. I thought that

  1. their exploration was interesting,
  2. that their findings were interesting (e.g., “this initial analysis suggests that ChatGPT was much more capable of interpreting and applying the article using an engineered prompt that included the abstract. But ChatGPT did not add much knowledge about cyber ranges In contrast, Bard appeared to take away almost nothing from the abstract but knew more about cyber ranges”)
  3. but, I was convinced that the prompts could have been engineered in more nuanced ways to improve the outcome.

I also recently learned that thinking of prompts in an incremental fashion may provide more nuanced result, and so I went about to recreate Hickey and Luo’s prompt in ChatGPT to see if I could generate a better outcome.

What did Hickey and Luo do?

Their knowledge-rich prompt was the following:

Use the following article by Sinatra, Heddy, & Lombardi (2015) to motivate engagement by cybersecurity learners who are using a digital range and use APA citations: Engagement is one of the hottest research topics in the field of educational psychology. Research shows that multifarious benefits occur when students are engaged in their own learning, including increased motivation and achievement. However, there is little agreement on a concrete definition and effective measurement of engagement. This special issue serves to discuss and work toward addressing conceptual and instrumentation issues related to engagement, with particular interest in engagement in the domain of science learning. We start by describing the dimensional perspective of engagement (behavioral, cognitive, emotional, agentic) and suggest a complementary approach that places engagement instrumentation on a continuum. Specifically, we recommend that instrumentation be considered on a “grain-size” continuum that ranges from a person-centered to a context-centered orientation to clarify measurement issues. We then provide a synopsis of the articles included in this special issue and conclude with suggestions for future research.

ChatGPT’s output was:

Research in educational psychology has consistently shown that engagement is a key factor in successful learning outcomes (Sinatra, Heddy, & Lombardi, 2015). For cybersecurity learners who are using a digital range, engagement is particularly important as it can enhance their motivation and achievement in this field. As Sinatra et al. (2015) note, engagement can be viewed from a dimensional perspective, encompassing behavioral, cognitive, emotional, and agentic dimensions. These dimensions can be used to design effective learning experiences that tap into students’ interests and motivations, such as through gamification or interactive simulations.

However, there is still a need for a concrete definition and effective measurement of engagement, especially in the context of science learning (Sinatra et al., 2015). To address this issue, educators in the field of cybersecurity can consider using a complementary approach that places engagement instrumentation on a continuum, ranging from a person-centered to a context-centered orientation (Sinatra et al., 2015). This approach can help clarify measurement issues and enable educators to tailor their teaching strategies to the individual needs and interests of their students.

In conclusion, engagement is a crucial factor in the success of cybersecurity learners using a digital range. By taking a dimensional perspective and using a continuum-based approach to measure engagement, educators can design effective learning experiences that promote student motivation and achievement in this field.

Reference:

Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges and opportunities of engaging learners in the scientific practices of explanation and argumentation. Journal of Educational Psychology, 107(2), 380–397. https://doi.org/10.1037/edu0000004

What did I do?

I divided the prompt in chunks, incrementally moving from the field (cybersecurity), to the topic within the field (cyber ranges) to the application of motivation to the topic (the Sinatra et al citation), to probing for specifics. I also used the term “cyber range” rather than “digital range” which Hickey and Luo used. This approach generated specific and actionable recommendations, especially compared to the generic reply above. My prompts and ChatGPT responses are below.

Prompt: I’d like to learn more about cybersecurity. Could you help me with this topic?

Sure! I’d be happy to help you learn more about cybersecurity. What specific areas of cybersecurity are you interested in? Some possible topics within cybersecurity include:

  1. Network security
  2. Information security
  3. Application security
  4. Cloud security
  5. Identity and access management
  6. Incident response
  7. Risk management
  8. Cybersecurity laws and regulations

Let me know what you’re interested in, and I can give you some pointers on where to start.

Prompt: What do people mean when they mention “cyber range” in the context of cybersecurity?

A “cyber range” is a simulated environment used for training and testing cybersecurity skills and technologies. It is essentially a virtual training ground where individuals and teams can practice identifying, preventing, and responding to cyber threats.

Cyber ranges can be set up to simulate a variety of scenarios, such as network intrusions, phishing attacks, malware infections, and denial-of-service attacks. They may also incorporate realistic components such as simulated user behavior, fake websites, and decoy systems.

By providing a safe and controlled environment for cybersecurity training and testing, cyber ranges can help organizations and individuals develop their skills and test their defenses without risking their actual systems or data. They can also be used to evaluate and compare different cybersecurity tools and techniques.

Some cyber ranges are open to the public, while others are only available to specific organizations or groups. Some are physical facilities that require specialized equipment and infrastructure, while others are entirely virtual and can be accessed from anywhere with an internet connection.

Overall, cyber ranges are an increasingly important tool for improving cybersecurity readiness and resilience in the face of an ever-evolving threat landscape.

Prompt: I would like to motivate engagement by cybersecurity learners who are using a cyber range. Using the ideas from Sinatra, Heddy, & Lombardi (2015) that are in quotes, write how to motivate engagement and use APA citations: “Engagement is one of the hottest research topics in the field of educational psychology. Research shows that multifarious benefits occur when students are engaged in their own learning, including increased motivation and achievement. However, there is little agreement on a concrete definition and effective measurement of engagement. This special issue serves to discuss and work toward addressing conceptual and instrumentation issues related to engagement, with particular interest in engagement in the domain of science learning. We start by describing the dimensional perspective of engagement (behavioral, cognitive, emotional, agentic) and suggest a complementary approach that places engagement instrumentation on a continuum. Specifically, we recommend that instrumentation be considered on a ‘grain-size’ continuum that ranges from a person-centered to a context-centered orientation to clarify measurement issues. We then provide a synopsis of the articles included in this special issue and conclude with suggestions for future research.”

To motivate engagement in cybersecurity learners who are using a cyber range, it is important to create a learning environment that is both challenging and supportive. According to Sinatra, Heddy, & Lombardi (2015), engagement is influenced by a variety of factors, including behavioral, cognitive, emotional, and agentic dimensions. To address these dimensions and promote engagement in the cyber range, the following strategies can be implemented:

  1. Provide hands-on learning experiences: learners should be given the opportunity to actively engage with the cyber range and practice their skills in a realistic and challenging environment. This can help to increase their sense of control and agency, which is important for promoting engagement (Sinatra, Heddy, & Lombardi, 2015).
  2. Create a supportive learning environment: learners should feel supported and encouraged throughout the learning process. Providing feedback, encouragement, and support can help to foster positive emotional and cognitive engagement (Sinatra, Heddy, & Lombardi, 2015).
  3. Use a variety of instructional strategies: learners have different learning styles and preferences, and it is important to use a variety of instructional strategies to cater to these differences. For example, using videos, interactive simulations, and quizzes can help to increase cognitive engagement (Sinatra, Heddy, & Lombardi, 2015). [Sidenote from George: This is worthy of a post around AI dispensing bad advice and perpetuating myths]
  4. Set clear learning goals: providing learners with clear learning goals and objectives can help to increase their motivation and focus. Goals should be challenging but achievable, and learners should be given regular feedback on their progress (Sinatra, Heddy, & Lombardi, 2015).

Overall, creating a supportive and challenging learning environment that promotes engagement can help to increase motivation and achievement in cybersecurity learners who are using a cyber range.

Reference: Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1-13. doi: 10.1080/00461520.2014.1002924

Prompt: I want to know more about #2. How do I create a supportive learning environment in a cyber range?

Creating a supportive learning environment in a cyber range involves providing learners with feedback, encouragement, and support throughout their learning process. Here are some specific strategies you can use to promote a supportive learning environment:

  1. Provide timely feedback: learners need feedback on their progress to understand what they are doing well and what they need to improve. Provide feedback as soon as possible after a learning activity and be specific about what the learner did well and what they need to work on.
  2. Encourage learners: it is important to encourage learners and recognize their efforts. Encouragement can come in many forms, such as praise, positive reinforcement, and recognition of achievements. This can help learners build confidence and feel more motivated to learn.
  3. Build a sense of community: building a sense of community among learners can help to create a supportive learning environment. Encourage learners to work together, share their experiences, and collaborate on learning activities. This can create a sense of belonging and increase engagement.
  4. Provide support resources: learners may need additional support to be successful in a cyber range. Provide resources such as online help, tutorials, and support forums to help learners overcome challenges and get the assistance they need.
  5. Use effective communication: effective communication is key to creating a supportive learning environment. Communicate clearly and frequently with learners, provide clear instructions, and be available to answer questions and provide support.

By using these strategies, you can create a supportive learning environment that promotes engagement and success in the cyber range.

Prompt: Can you give me an example of an encouraging message I can send to learners who are in a cyber range?

Sure! Here is an example of an encouraging message you can send to learners who are in a cyber range:

“Great work on completing that cyber range activity! Your hard work and dedication to learning in this field are truly commendable. Remember, every challenge you overcome is an opportunity to grow and become a better cybersecurity professional. Keep up the good work, and don’t hesitate to reach out if you need any support or assistance. You’ve got this!”

This message acknowledges the learner’s efforts, encourages them to continue their learning journey, and offers support if needed.

Faculty curiosities about AI tools and ChatGPT

I led an online workshop/conversation on AI for ~200 faculty at three colleges/universities who came together today to learn about the topic. It centered on the following questions. I am sharing them here for the benefit of others, but also to ask: Are there other curiosities that you are seeing locally? (Yes, I know that the most recent EDUCAUSE poll highlights cheating as a top concern, though I’m not certain it ought to be)

  • How can (should??) I use AI for the benefit of my students’ learning?
  • Is ChatGPT really the disruptor it seems to be?
  • ChatGPT (AI) and authentic assessment – can these co-exist?
  • Neither I nor my students are as tech-savvy as it is often assumed we are. How do we keep up with innovations like ChatGPT, whether they be ‘good’ or ‘bad’, and how do we learn when to embrace them or ignore them?
  • Is ChatGPT (or other AI) a blessing or a curse for higher education?

But how do you really feel about being a chatbot?

As some of you know, I’m writing a book on teacherbots, chatbots and artificial intelligence in education. Which means that I have conversations with chatbots every now and then. Like this one:

Bots, AI, & Education update #3

Today’s rough set of notes that focus on teacherbots and artificial intelligence in education

  • Chatbots: One of the technologies that’s mesmerized silicon valley
  • Humans have long promised future lives enhanced by machines
  • Many proponents highlight the qualities of bots vis-a-vis teachers
    • personal
    • personalized
    • monitoring & nudging
    • can give reliable feedback
    • don’t get tired
    • etc etc
  • Knewton: Algos to complement and support teacher (sidenote: as if anyone will be forthright about aiming to replace teachers… except perhaps this book that playfully states that “coaches (once called teachers)” will cooperate with AI)
  • Genetics with Jean: bots with affect-sensing functionality, ie software that detects students’ affective states and responds accordingly
  • Driveleress Ed-Tech: Robots aren’t going to march in for jobs; it’s the corporations and the systems that support them that enable that to happen.

Bots, AI, & Education update #2

Yesterday’s rough set of notes that focus on teacherbots and artificial intelligence in education

  • Notable critiques of Big Data, data analytics, and algorithmic culture (e.g., boyd & Crawford, 2012; Tufecki, 2014 & recent critiques of YouTube’s recommendation algorithm as well as Caulfield’s demonstration of polarization on Pinterest). These rarely show up in discussions around bots and AI in education, critiques of learning analytics and big data (e.g., Selwyn 2014; Williamson, 2015) are generally applicable to the technologies that enable bots to do what they do (e.g., Watters, 2015).
  • Complexity of machine learning algorithms means that even their developers are at times unsure as to how said algorithms arrive at particular conclusions
  • Ethics are rarely an area of focus in instructional design and technology (Gray & Boling, 2016)  – and related edtech-focused areas. In designing bots where should we turn for moral guidance? Who are such systems benefiting? Whose interests are served? If we can’t accurately predict how bots may make decisions when interacting with students (see bullet point above), how will we ensure that moral values are embedded in the design of such algorithms? Whose moral values in a tech industry that’s mired with biases, lacks broad representation, and rarely heeds user feedback (e.g., women repeatedly highlighting the harassment they experience on Twitter for the past 5 or so years, with Twitter taking few, if any, steps to curtail it)?

Bots, AI, & Education update #1

A rough set of notes from today that focus on teacherbots and artificial intelligence in education

  • Bots in education bring together many technologies & ideas including, but not limited to artificial intelligence, data analytics, speech-recognition technologies, personalized learning, algorithms, recommendation engines, learning design, and human-computer interaction.
    • They seek to serve many roles (content curation, advising, assessment, etc)
  • Many note the potential that exists in developing better algorithms for personalized learning. Such algos are endemic in the design of AI and bots
    • Concerns: Black box algorithms, data do not fully capture learning & may lead to biased outcomes & processes
  • Downes sees the crux of the matter as What AI can currently do vs. What AI will be able to do
    • This is an issue with every new technology and the promises of its creators
    • Anticipated future impact features prominently in claims surrounding impact of tech in edu
  • Maha Bali argues that AI work misunderstands what teachers do in the classroom
    • Yet, in a number of projects we see classroom observations as being used to inform the design of AI systems
  • “AI can free time for teachers to do X” is an oft-repeated claim of AI/bot proponents. This claim often notes that AI will free teachers from mundane tasks and enable them to focus on those that matter. We see this in Jill Watson, in talks from IBM regarding Watson applications to education, but also in earlier attempts to integrate AI, bots, and pedagogical agents in education (e.g., 1960s, 1980s). Donald Clark reiterates this when he argues that teachers should “welcome something that takes away all the admin and pain.” See update* below.
  • Another oft-repeated claim is that AI & bots will work with teachers, not replace them
  • At times this argument is convincing. At other times, it seems dubious (e.g., when made in instances where proponents ask readers/audience to imagine a future where every child could have instant access to [insert amazing instructor here])
  • Predictions regarding the impact of bots and AI abound (of course). There’s too many to list here, but here’s one example
  • Why a robot-filled education future may not be as scary as you think” argues that concerns around robots in education are to be expected. The article claims that people are “hard-wired” to perceive “newness as danger” as it seeks to explain away concerns by noting that education, broadly speaking, avoids change. There’s no recognition anywhere in the article that (a) education is, and has always been, in a constant state of change, and (b) edtech has always been an optimistic endeavour, so much so that its blind orthodoxy has been detrimental to its goal of improving education.

 

Update:

From Meet the mind-reading robo tutor in the sky:

And underpaid, time-stressed teachers don’t necessarily have the time to personalize every lesson or drill deep into what each child is struggling with.

Enter the omniscient, cloud-based robo tutor.

“We think of it like a robot tutor in the sky that can semi-read your mind and figure out what your strengths and weaknesses are, down to the percentile,” says Jose Ferreira, the founder and CEO of ed-tech company Knewton.”

Powered by WordPress & Theme by Anders Norén