What to do

Check the learning package to see if learning resources are provided for all weeks / topics of the course.

Where to focus

Learning resources are usually one of the main content areas  and hence should be easily identifiable as a stand-alone section with headings and subheadings.

What to look out for, specifically

Examples of learning resources may be readings materials like journal articles, textbooks, videos, class slides, etc.

Tips

You may want to see if learning resources are presented consistently e.g. a video followed by a few readings for each topic and there is not a big difference in the number of materials assigned per week.

Show manaakitanga by ensuring all required learning resources are easy to access, reliable, and available when needed. Consistent access supports equity and allows learners to engage without unnecessary barriers.

Example

Course resources are embedded directly within Canvas, with downloadable versions available. Links open reliably, and no additional logins are required for essential materials.

Further Info

• A comprehensive guide to Universal Design for Learning (UDL) – Ako Aotearoa
• Canvas Baseline Practices (CBP)

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Check the size of the materials students need to download or view on their own devices to ensure they are not too large and do not cause storage or playback issues while loading.

Where to focus

Videos or audio files are usually much larger in size than reading materials in Word and PDF format, hence a closer look at them would be great.

What to look out for, specifically

Book chapters and journal articles are usually quite small in size so you may focus more on the textbook, audio recordings/videos uploaded or software/apps that students are expected to install on their devices.

Tips

Play videos/audio files to see if they load quickly and play smoothly. Download files that look big e.g. the whole book, not a chapter, or a long video, to see how large the file is. Students may have storage issue if they have to download files that are hundreds of Mbs to a mobile/portable device.

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Open reading materials, play audio-visual files and check other resources e.g. learning objects, software and apps to see if they work properly on a range of different devices.

Where to focus

All the learning resources that are available in the learning package, however you do not need to engage with each of them for too long, just a quick check is sufficient.

What to look out for, specifically

Check if the materials can be viewed/watched online, downloaded and saved to a local hard drive for student engagement at their convenience.

Tips

You may want to use a desktop computer, a laptop, tablet and mobile phone to check if all the learning resources are functional. It would be best if the devices are not the latest models since students may use old devices that are not the most powerful.

What to do

Find out whether students can control how they view learning resources, e.g. expand/collapse/annotate readings, or fast-forward/rewind audio/video files.

Where to focus

All the different formats of learning resources in the learning package to check for learner control in multi-modal forms, e.g. written texts and/or videos.

What to look out for, specifically

How students can control their learning process and engagement with learning resources.

Tips

Students may be given different levels of learner control dependent on the type of learning resources so you may want to pay attention to whether students can have the same kind of learner control over materials of the same type.

Some information and examples for learner control of learning resources are available here: E-Learning Essentials 2020 – Chapter 24 Instructional Design Principles

Support rangatiratanga by providing resources that allow learners to engage in ways that suit their needs. Look for features that enable flexibility, such as playback control, downloading, or annotation.

Example

Videos include playback controls and downloadable transcripts, while readings are provided as text-selectable PDFs for annotation.

Further Info

• A comprehensive guide to Universal Design for Learning (UDL) – Ako Aotearoa
• Canvas Baseline Practices (CBP)
• Ngā Hau e Whā o Tāwhirimātea: Culturally Responsive Teaching and Learning for the Tertiary Sector

What to do

Check if learning resources serve the purpose they are supposed to.

Where to focus

The content area for learning resources, learning goals and schedules.

What to look out for, specifically

The content, use and intended purposes of the learning resources.

Tips

You may put on both the hat of a learning designer and student to see if the materials provided are fit for their intended purpose as described in the learning package. You may wish to consider the discipline in which the learning package is situated and whether the choice of learning resources are fit for the purpose of this discipline. An example might be in relation to the use of AI.

Demonstrate manaakitanga by ensuring resources are usable, accessible, and function as intended. Look for materials that meet accessibility standards and avoid common barriers such as poor formatting or broken links.

Example

Resources include captioned videos, accessible PowerPoint slides with strong colour contrast, and editable documents rather than scanned PDFs.

Further Info

• NZ Government Web Accessibility Standards
• A comprehensive guide to Universal Design for Learning (UDL) – Ako Aotearoa
• Canvas Baseline Practices (CBP)

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Skim the learning package quickly to check that copyright regulations are followed, e.g. whether the developers have permission from copyright holders or are entitled to use external resources such as images, videos, and readings.

Where to focus

The learning resources that are not created by the teaching team, including images, audio-visual materials, graphs and readings.

What to look out for, specifically

Whether the materials included in the learning package are copyright-free, or copyright holders have provided permission for the materials to be used.

Tips

Pay attention not only to copyright issues but also to consistent compliance with relevant regulations. For example, if most resources are copyright-free but a few images are not, then a Yes But is a good option.

Further information regarding copyright can be found here: Copyright guidance (University of Wollongong)

What to do

Skim the learning package to ensure learning sources are properly attributed and referenced.

Where to focus

The learning resources not created by the teaching team themselves, a list of references or the copyright statement provided by the developers. These include an appropriate acknowledgement of the use of AI.

What to look out for, specifically

Examples may be Creative Common licenses, links to original sources or signals that permission to use has been granted by copyright holders.

Tips

You should make sure all learning resources from external sources are properly referenced and attributed. For example, at the end of the slide decks there may be a list of references and a table of attribution provided.

Reflect kotahitanga by modelling ethical and respectful use of resources. Look for clear attribution of all materials, including acknowledgement of cultural origins where relevant.

Example

Images include Creative Commons licences and source attribution, and cultural artefacts are accompanied by acknowledgement of iwi or community origins where appropriate.

Further Info

• A comprehensive guide to Universal Design for Learning (UDL) – Ako Aotearoa
• Ako Explored (Ako Aotearoa)
• Canvas Baseline Practices (CBP)

What to do

Check whether learning resources are inclusive of diverse author/student cohorts and culturally appropriate, without being offensive to any parties involved.

Where to focus

Check the readings, images and audio-visual materials in the topics and content sections.

What to look out for, specifically

Whether learning resources include content that is specifically about Indigenous culture or sensitive topics like religion, and whether there are inappropriate materials or language that display biased opinions or stereotypes.

Tips

Have a quick look through the learning package and identify potential issues regarding cultural considerations rather than read all the information provided. An overview evaluation may be quicker and more useful than a thorough appreciation of each individual learning resource.

For more information, you may refer to the Learning and Teaching Across Cultures Good Practice Principles and Quick Guides by the Australian Government available here: Learning and Teaching Across Cultures (IEAA)

Embed manaakitanga by ensuring learning resources are culturally safe and respectful. Look for accurate use of te reo Māori, appropriate framing of sensitive content, and guidance where material may impact learners.

Example

Before discussing colonisation, a note advises learners of potentially distressing content and offers support options. Māori terms are used correctly with macrons.

Further Info

• Huakina mai: Doorways toward culturally responsive education
• Tataiako
• Ngā Hau e Whā o Tāwhirimātea: Culturally Responsive Teaching and Learning for the Tertiary Sector
• Card, A., Merry, R., & Carroll-Lind, J. (Eds). (2022). Bicultural principles of teaching and learning online | Ngā mātāpono kākano rua o te mahi ako tuihono

What to do

Skim the learning resources provided to see if they are created by authors from various cultural, gender and demographic backgrounds and hence students from culturally and linguistically diverse origins can relate to them without feeling excluded.

Where to focus

The authors, content and audiences of learning materials.

What to look out for, specifically

A salient example of diversity in learning resources may be that they are written by various authors of different genders, cultural origins from different regions with diverse perspectives presented. Also be mindful of the ability of AI generated resources to have an inherent bias.

Promote kotahitanga by ensuring learning materials represent a range of perspectives and experiences relevant to Aotearoa. Representation should feel authentic rather than tokenistic.

Example

Case studies include Māori, Pasifika, Asian, and rural perspectives, reflecting a range of lived experiences and contexts.

Further Info

• Ngā Hau e Whā o Tāwhirimātea: Culturally Responsive Teaching and Learning for the Tertiary Sector
• Card, A., Merry, R., & Carroll-Lind, J. (Eds). (2022). Bicultural principles of teaching and learning online | Ngā mātāpono kākano rua o te mahi ako tuihono

What to do

Skim to see if learning resources do not focus only on a particular region e.g. Australasia.

Where to focus

The list of references, library reading catalogue or abstracts/summary of resources provided.

What to look out for, specifically

Check the context and intended audience of the learning resources to see where they apply to, a particular country or region or world-wide.

Tips

You may want to check the course profile or introduction to see whether the learning package is intended for students in a particular context, e.g. only domestic Australian students, or for a wider audience to evaluate the suitability of the contextualised materials.

Encourage ako by positioning Aotearoa perspectives alongside global contexts. Look for learning materials that connect local knowledge with international frameworks while maintaining Indigenous visibility.

Example

A sustainability course compares Māori kaitiakitanga with Indigenous and international environmental approaches.

Further Info

• Tertiary Education Strategy – Summary
• Indigenous higher education Māori experience in New Zealand

What to do

Record the reason or evidence why a Yes / Yes But / No But / No was given for each success indicator. Start with the number, e.g. 7.1.1. and put the comment for each indicator in a new paragraph.

Where to keep these comments

There is no separate box to provide comments for each success indicator, so all comments go in the same section under all the success indicators in a standard.

Why it is necessary to have these comments

Reviewers need to meet to discuss the Combined Review, and by that time they may have forgotten why they made that decision or where specific information was found for specific success indicators as some time may have passed. Having such explanations and evidence handy also helps reviewers reach a decision where there are differences in opinion.

Tips

When a Yes was given, you may want to note where you found that piece of information as the other reviewer may have not seen it. For a Yes But, No But, or a No, it is important to note what is missing and how the issue can be addressed. These comments will be made available to the developers and staff later and your comments will be helpful for them in improving their learning package.