Author: professor

Multiple-choice questions are an attractive assessment method for online learning developers, principally because they can be marked automatically. In most professional development scenarios, they need to assess more than factual recall and instead present the learner with a scenario and ask them what they would do. They are used within this learning programme for both formative and summative assessments.

Formative assessments

The purpose of formative assessment is to give learners feedback, so it is necessary not only to create the scenarios, a range of responses and identify the best response but also to explain why. These are presented in this programme as slide presentations with answers and explanations following the scenarios and the possible responses.

Summative assessments

The purpose of summative assessment is to assess learners at the end of the programme, so it is necessary not to create the scenarios, a range of responses and identify the best response and then compare the learners’ responses with the best response. It is necessary to build a question bank from the individual questions and then present the learner with a random selection, to facilitate re-assessments and discourage learners from sharing the answers. This functionality is provided by many learning or content management systems, which may also generate certificates automatically for learners who correctly answer sufficient questions. In this case, you will need to generate the questions as a spreadsheet or comma comma-separated values text file.

Using AI to help you generate your scenarios and multiple-choice questions

AI-based tools can help considerably in generating questions for both summative and formative assessments.

At its most basic, you can write a prompt for your large language model asking to generate x questions about the topic you wish to be assessed.

However, considerable refinement and experimentation is required to create effective, meaningful and in-depth scenarios and questions.

From experience, ChatGPT has two common failings:

• The best response is nearly always the longest. This is a giveaway to human learners. Make sure you include an instruction to make the responses of equal length, but check the output and manually edit as necessary
• It often puts the best response in the same place, often response c, if the responses are labelled a to e. Again, make sure you include an instruction to distribute the best responses equally, but check the output and manually edit as necessary.

My experience is that Google Gemini is slightly better at this task, and if you first upload your content to LM Notebook, and then use the chat function within Notebook, it is easier to get more meaningful scenarios and detailed explanations. You can also specify the output within your prompt, asking for a table of scenarios with column headings tailored to create a comma-separated values file to import directly into a question bank for a summative assessment for example. For formative assessments, I copy and paste the material into a PowerPoint presentation and then export the images to create the slide presentations that you can see throughout this programme.

Again, the best thing to do is to experiment for yourself. Try the following:

Create an LLM prompt specifying your requirements, either manually or with the help of your LLM.
Upload your material to be assessed into a new Notebook in Notebook LM.
Within the Chat function, use the prompt to generate your questions.
Review your questions. They are unlikely to be totally satisfactory at the first attempt!
Either manually edit the questions or refine your prompt to get closer and regenerate before final manual editing.

How was the process? How were the results?
Record your thoughts in your learning diary before you move on….

We have seen previously that dialogues can be more engaging than single talking heads.

AI-based tools can help us create engaging dialogues.

Automated notetaking has become a relatively common use of Large Language Models. Notebook LM is Google’s summarising tool. One of its most useful functions is the audio overview. This can be used to create a dialogue and is the tool used to generate the dialogues found within this learning programme, where you see the following image:

Notebook LM has developed rapidly and now offers significant options for tailoring its output:

You can tailor the length, the type and crucially by giving specific instructions about the sources, scope, purpose and audience. The best way to investigate the possibilities is to experiment with the tool.

First, gather your sources. A good starting point is between 3 and 5 articles as pdf files.
Next go to https://notebooklm.google.com/
Create a new notebook and upload your sources.
Click on the pencil on the Audio overview button to customise your output and then select Generate.
Listen to the output.
Repeat with different customisations
What did you think of the output?
What were the best customisation options?
What were the limitations?

Record your thoughts in your learning diary before moving on…..

Now that you have completed this learning programme, you have the opportunity to submit your design for an online programme for expert feedback.

Step 1: Complete your design by specifying how you will implement, evaluate and maintain your programme

Download a template below and then complete the final section of your plan.

Download your template here

Step 2: Collate the three sections of your plan into a single document

Once you have completed this section of your plan, collate your whole plan by creating a single document from your completed template from the first module, the template from the second module that you updated after the third module, and the completed template from this module.

Use what you have learnt to decide what you would do in each of the following situations:

Implementation Challenge 1

Implementation Response 1

Implementation Challenge 2

Implementation Response 2

Implementation Challenge 3

Implementation Response 3

Implementation Challenge 4

Implementation Response 4

Implementation Challenge 5

Implementation Response 5

Implementation Challenge 6

Implementation Response 6

Implementation Challenge 7

Implementation Response 7

Implementation Challenge 8

Implementation Response 8

Implementation Challenge 9

Implementation Response 9

Implementation Challenge 10

Implementation Response 10

After each scenario, the best way forward is explained. Use the dots to move through the slides

What have you learnt from these scenarios, especially the ones where you weren’t sure what to do?

Record your thoughts in your learning diary before moving on….

This discussion has been generated by a combination of AI tools (under the close human supervision of the course author).

The goal was to create a discussion about the content of this final module.

Please read our policy statement on the use of AI in our courses

What are the key points that you found interesting in this discussion?
How did the discussion help develop your understanding of the issues?

Record your thoughts in your learning diary before moving on….

For online learning developers, translating a theoretical understanding of professional recognition into practical, impactful programmes requires strategic foresight and meticulous execution in programme design and delivery.

To ensure online professional development programmes are genuinely impactful within the UK, developers must meticulously align their content and learning outcomes with identified industry needs and prevailing employer demand. This necessitates proactive engagement with employers to gain a nuanced understanding of specific skill shortages, such as those in critical digital, analytical, leadership, and communication competencies.³ The explicit call for “employer-led course design” underscores the paramount importance of co-creation or close, continuous consultation with industry stakeholders throughout the development process. Programmes should be designed to provide “precision learning” that meets “just-in-time” skill requirements, potentially leveraging the inherent rapid deployment capabilities of micro-credentials. The focus must remain steadfast on developing practical, job-ready skills that employers demonstrably value, as this directly leads to increased employability and higher earning potential for learners.

The persistent skills shortages in the UK, despite significant investment in training, suggest a misalignment between the supply and demand of professional development offerings. This indicates that programme developers must transition from a solely content-centric approach to a demand-driven, outcomes-focused model, actively collaborating with industry to ensure the relevance and efficacy of their programmes. The solution lies in developing programmes that are explicitly designed with direct employer input and are precisely targeted at identified skill gaps, rather than offering generic content. This necessitates a shift from a supply-side perspective, focusing on what can be taught, to a demand-side perspective, concentrating on what employers genuinely need.

Addressing Challenges in Online Delivery

While online learning offers unparalleled flexibility and accessibility, developers must proactively address its inherent challenges to ensure the delivery of high-quality professional development experiences. Mitigating the perceived “limited human connection” and “online fatigue” requires fostering a strong sense of community through intentionally designed interactive elements, such as dynamic discussion boards, collaborative projects, and regular live virtual sessions. To effectively counter challenges related to “self-discipline and time management”, clear accountability mechanisms, including structured milestones and intuitive progress tracking, are crucial components of programme design. Ensuring that “technology dependence and accessibility issues” are minimised involves providing robust technical support, designing user-friendly learning platforms, and rigorously adhering to national accessibility regulations. Furthermore, providing timely, constructive, and personalised feedback is essential to compensate for the reduced opportunities for immediate feedback often found in online environments.

The persistent challenges inherent in online learning, such as isolation and self-discipline, are not merely logistical hurdles but fundamental pedagogical considerations. Successfully addressing these requires intentional design choices that either mimic or effectively compensate for the benefits typically associated with traditional face-to-face interaction, thereby enhancing the overall efficacy and learner experience. This demonstrates that effective online learning design is not a simple transfer of content but a sophisticated pedagogical process. It requires a deep understanding of the unique psychological and social aspects of remote learning and the strategic design of interventions to ensure sustained engagement, motivation, and effective knowledge transfer.

Looking to The Future

The landscape of online professional development in the United Kingdom is undeniably dynamic, propelled by the imperative for continuous upskilling and reskilling in a rapidly evolving job market. Developers of online learning programmes must strategically navigate the distinct yet interconnected realms of certification, credentials, and micro-credentialing, understanding their specific purposes and the varied pathways to formal recognition. Embracing rigorous quality assurance through proactive engagement with regulatory bodies such as Ofqual and the QAA, alongside relevant professional associations, is paramount for establishing programme credibility and ensuring widespread employer acceptance. Furthermore, effective programme design must proactively address the unique challenges inherent in online delivery, fostering deep engagement, promoting learner accountability, and cultivating a supportive virtual learning environment.

The increasing employer demand for verifiable, skill-specific qualifications, particularly micro-credentials, underscores a fundamental shift towards agile, outcomes-focused learning. By integrating cutting-edge technologies such as digital badges and, where appropriate, blockchain, developers can ensure the authenticity, portability, and widespread recognition of professional achievements, thereby significantly enhancing learner employability and accelerating career progression. The overarching narrative reveals a systemic shift in the UK from traditional, lengthy qualifications to a more modular, flexible, and verifiable ecosystem of professional recognition. This indicates a future where a professional’s identity and capabilities will increasingly be built upon a dynamic “portfolio of competencies,” evidenced by a combination of credentials, certifications, and stackable micro-credentials, rather than relying solely on a single, traditional degree. The future of online professional development in the UK, therefore, lies in the creation and delivery of responsive, high-quality, and formally recognised learning experiences that empower individuals to thrive and effectively address the evolving needs of the national workforce.

How will you ensure that your objectives are aligned with those of your target sector?

Record your thoughts in your learning diary before you move on…