What to do

Skim the entire learning package quickly to see if the language used is culturally appropriate and respectful of the diversity of Australia’s people. You may want to focus on the following areas while checking if inclusive language is used appropriately when referring to:

• Aboriginal and Torres Strait Islander peoples
• Age diversity
• Cultural and linguistic diversity
• Gender and sexual diversity
• People with disability

Where to focus

The following sections of the learning package are more likely to provide evidence of appropriate and inclusive language: Announcements, Discussion Forum, Introduction.

What to look out for, specifically

Examples of appropriate and inappropriate language use for each of the above areas can be found in the Style Manual by Australian Government

Tips

You may recognise that examples of inappropriate and non-inclusive language often stand out and hence can be identified quite easily without careful reading of the site. While skimming the learning package, if anything catches your eye, stop, and leave a note in the Comment section at the end of Standard 2. You may have difficulty finding it again later as supporting evidence of your evaluation in the Combined Review.

The language should be welcoming, respectful, and accessible to all learners. In Aotearoa contexts, inclusive language may include appropriate use of te reo Māori and acknowledgement of diverse cultural perspectives, reflecting manaakitanga. Look for whether the course use inclusive language that welcomes a diverse learner cohort.

Example

A Canvas module opens with a bilingual greeting (“Nau mai, haere mai – Welcome!”) and clear weekly instructions written in the second person.

Further Info

• Tataiako  (pp. 10, 14)

What to do

Check if diverse perspectives are represented and discussed in the learning resources, teaching activities and assessment tasks of the learning package. Diversity can be in the form of:

• Authors: Materials are written by different people, not an individual or single group, of a particular cultural background, gender, or age range only. If content has been AI generated (if known), consider if it includes a diverse and inclusive voice.
•  
• Theories: Various theories and viewpoints are presented and discussed e.g. in the slides and readings, images, content and videos.
•  
• Chronological order: Resources may cover different developmental phases of a phenomenon or artefact. If the learning package focuses on a particular period of time, e.g. contemporary TEL practices, readings may be published across a number of years within that period.
•  
• Geographical settings: Resources should focus on more than one geographical area, e.g. not Australia or NZ only (depending on where the learning package is situated).
•  
• Genre: Resources provided in the learning package should be a combination of different types, e.g. journal articles, book chapters, reports by government and organisations, etc.

Where to focus

The best places to look for such evidence of diverse perspectives are in the Readings/Resources section, also in the slide decks, schedule, learning objectives, and assessment task description.

What to look out for, specifically

See if you can find readings by different authors and of different genres, including literature review papers; content on various theories and developmental phases in the weekly schedule; a reading list with different years of publications, and titles/abstracts that mention different geographical regions.

Tips

Make sure you leave a comment in the comment box explaining why you give that success indicator a Yes / Yes But / No But / No. It will be immensely helpful when you discuss the different evaluation between reviewers in the Combined Review, especially when you cannot remember what made you tick that box a while ago.

Learning environments should recognise the diversity of learners and communities in Aotearoa. Including Māori, Pasifika, and other perspectives supports whanaungatanga and helps learners see their identities and experiences reflected in the curriculum. Look for whether the learning package materials acknowledge diverse cultural perspectives relevant to Aotearoa.

Example

A sustainability module presents both mātauranga Māori perspectives and Western scientific approaches to environmental management.

Further Info

• Huakina mai: Doorways toward culturally responsive education
• Card, A., Merry, R., & Carroll-Lind, J. (Eds). (2022). Bicultural principles of teaching and learning online | Ngā mātāpono kākano rua o te mahi ako tuihono

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Check if the online learning package is displayed properly on a range of electronic devices from a desktop computer, a laptop, a tablet, and a smartphone with Internet connection. The more devices, the better as a learning package may look fine on a tablet but not on a phone screen or vice versa.

Where to focus

No specific areas. Use a device to log in to the learning package, flick through the menu and try all tabs without paying attention to the content. Once having checked the whole site, log out and try signing in again on another device.

What to look out for, specifically

Images, videos, and texts, and how they are displayed on the screen of each device. Pay close attention to where there are overlapped texts and images/videos, or where audio-visual materials are not displayed appropriately. Headings, subheadings, and bullet points or font sizes and zooming are also things to consider as they may be arranged incorrectly in small screens.

Tips

Try using devices that are not the latest in the range, since many students may use basic or old devices, like an old laptop with low specifications. Where possible, try using devices of different makes and brands.

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Check if the learning package can be accessible on different platforms e.g. Microsoft, Android, Macintosh etc., and operating systems, e.g. Windows, iOS, Linux, etc.

Where to focus

Check the menus and pages to see if the learning package works well and is displayed properly on a platform or operating system. There is no need to read the information given in each section.

What to look out for, specifically

Check the menu, pages, sections, quizzes, etc. to see if they are functional and accessible.

Tips

The same learning package may look different on different platforms. It may not be necessary to check the learning package compatibility with all platforms as a reviewer may not have a device running Linux operating system, so it is fine to try only the two most popular ones (Windows and iOS).

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Access the online learning package on different browsers to see if there are any problems loading or viewing it. You may try using as many browsers as possible, making sure you try at least two of the most popular ones being used by Universities.

Where to focus

More attention can be paid to videos, images, and content areas with large amounts of text, however there is no need to spend too much time on each of them. Simply click through the whole site and see if any issues arise.

What to look out for, specifically

When you log in to the learning package in a browser, check how long it takes to load the website. While you click on a menu, tab, or link, pay attention to whether the site responds quickly and things move smoothly. Also, it may be useful to see if videos and images are loaded quickly and video or audio files play without delay or lag.

Tips

Try accessing the online package when you are opening multiple tabs on each browser to see if the loading speed is sluggish. Also, you may use different browsers at the same time, at times with peak Internet traffic, such as in the evening, as students may have unstable connections.

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Check if the course materials meet the seven principles for universal design listed below:

• Equitable Use
• Flexibility in Use
• Simple and Intuitive Use
• Perceptible Information
• Tolerance for Error
• Low Physical Effort
• Size and Space for Approach and Use

Where to focus

Check the website, the content areas and activities presented in the learning package. More attention can be paid to the learning materials, weekly activities and assessment tasks as the students will focus on these areas hence accessibility is important there.

What to look out for, specifically

It may be easier to focus on things that may arise as an accessibility issue for a certain group of users, for example those with disabilities. If there are any concerns or potential issues with accessibility, make sure you leave a comment to explain your opinion on the success indicator. Typical examples of accessibility issues can be found here: https://www.washington.edu/accesscomputing/AU/v2/problems.html You can also use the WAVE browser plugin or similar to do these checks for you.

Tips

Have a quick look at the seven principles for universal design listed in the link above, then quickly check the content of the learning package. Looking for potential issues and evidence for why they can be an accessibility issue. If there is none, then a Yes can be selected.

More information regarding the seven principles for universal design is available here: https://universaldesign.ie/what-is-universal-design/the-7-principles/. Additionally, UDL 3.0 has been released, which also focuses on creating a learning environment that supports the learner’s journey. It focuses on the principles of engagement, representation and action and expression. More information is available here: https://www.adcet.edu.au/resource/11762/cast-udl-guidelines-3-0-released

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Check if external tools and applications adhere to accessibility standards using seven principles for universal design provided in Section 2.3.1. Please note that you may not be provided access to all these external tools and applications by the learning package developer. If that is the case, please focus only on the tools and applications you can access and leave a comment about those that you cannot check.

Where to focus

External tools and applications may be provided in class slides, announcements, assessments, or learning resources.

What to look out for, specifically

Like 2.3.1, it may be more timesaving to look for accessibility issues as they may be easier to identify rather than check all of the learning package methodically. Typical examples of accessibility issues can be found here: Accessible University: List of Accessibility Problems You can also check the tool specs on their respective website and see if they meet the latest accessibility standards such as WW3 WCAG 2.1 and to what extent (AA⁺ etc)

Tips

Information regarding the seven principles for universal design is available here for you to review if necessary: Seven Principles of UDL

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Check if the site is optimised for screen readers. The following link contains examples of problems that may make it hard for screen readers to work efficiently: https://www.washington.edu/accesscomputing/AU/v2/problems.html Also, it is important to see if files are consistently named and organised by type and size.

Where to focus

Have a quick look around the whole site to check compatibility with screen readers. For file names and sizes, you may want to check the Learning Resources section, Assessment and Announcements as they would usually be the places where files are uploaded.

What to look out for, specifically

Pay attention to headings, subheadings, text, images with text, and the naming protocol for uploaded files. Look for potential issues that may interfere with screen readers, or file names that look out of place, or files that are too large and take too long to download.

Admin Reviewer – This is an admin reviewer’s task so the first and second reviewers do not have to engage with it as you will see the evaluation result and comment by the admin review when you log in to the system.

What to do

Check whether multimedia content is available in different formats to ensure options that meet users’ needs and preferences. For example, whether photos have an accompanying text description, and audio or video recordings have subtitles and transcripts in case students did not or could not listen or watch.

Where to focus

Multimedia content is usually provided in Announcements, Learning Resources, and other content areas of the site, such as the Introduction or at the beginning of a page/section.

What to look out for, specifically

Images, graphic content, audio and video, and other ways to present the content of these types of multimedia in case they are inaccessible, or not displayed properly. They can be in various alternate formats, for example a PowerPoint slideshow for a video, or the transcripts of an audio file can be automatically generated or composed and embedded in the video or provided as an attachment.

Tips

If videos are not produced by the University but are taken from external sources, e.g. YouTube, they may not have an accompanying transcript, but students may choose to turn on subtitles, which is a function available on the YouTube video player window.

Here is a video from YouTube support on how to set up automatic subtitles on YouTube videos

What to do

Check if students can provide immediate feedback after they engage with the learning materials. For example, if there is a checklist for them to mark a task as completed, a thumbs up/down button to show their opinion on a reading, or a flagging option for them to mark content of importance or interest. It may also be instructions for students to leave a reflection on the Discussion Forum, or comment using videos in VoiceThread, etc.

Where to focus

Students are usually asked to provide feedback in the Discussion Forum, Announcements, Introduction or Learning Resources sections.

What to look out for, specifically

Any opportunities for students to provide their feedback, in any form, e.g. a survey, a reflection journal, or a sentence in an announcement saying students should write to the course convener should they have any questions or feedback regarding the content. Also, be aware that the LMS being used may provide feedback functionality, but it may not have been enabled by the developer. If this is evident, please make a note of that in the comments.

Tips

Students may have the opportunity to provide feedback directly to the teaching team in face-to-face sessions or video conferencing like synchronous meetings/tutorials. However, these communications are out of the scope of the review so reviewers may only provide an opinion based on what is available on the site. A comment may be useful to remind the design team that they can provide explicit instructions in the site on ways students can provide immediate feedback to teaching staff. This would help ensure that students know what to do to provide feedback if oral instructions are missed.

Providing simple ways for learners to respond to activities supports ongoing dialogue between teachers and learners. This reflects ako, where feedback informs teaching and learning in both directions. Look for whether the learners able to give quick feedback on learning activities or resources.

Example

Each module page includes a quick feedback option allowing learners to indicate whether the activity was helpful.

Further Info

• Student voice in tertiary education settings

What to do

Check if there are any instructions or encouragement for students so that they can provide their feedback to teaching staff at different points of time, i.e. in different weeks at the beginning, middle and end of the course/unit. The feedback can be of any form, e.g. orally or in writing, and various formats, e.g. via email, in Discussion Forum, feedback survey, etc.

Where to focus

Such instructions for students to provide feedback may be found in the Announcements, Introduction, Discussion Forum or even in Assessment section.

What to look out for, specifically

This may sometimes be a short sentence somewhere on the site, introducing the learning package to students or inviting feedback from them in the sections mentioned above. You may also want to look for links to background surveys, reminders to complete evaluations, discussion threads, reflection journals or even a quick feedback page in the weekly lecture/tutorial slides.

Tips

Pay attention to critical points of time like the first and last weeks, and probably around the mid-semester break. If you can find instructions requiring students to provide feedback at about 2-3 different points of time or more during the whole course/unit, then it probably deserves a good Yes.

Regular feedback opportunities allow teaching teams to respond during the course and improve the learning experience. This supports whanaungatanga by strengthening dialogue between learners and educators. Look for whether the learners able to provide feedback during the course rather than only at the end.

Example

A short mid-course survey allows learners to comment on pacing, workload, and clarity of materials.

Further Info

• Student voice in tertiary education settings

What to do

Check if there are the instructions provided to students on how their feedback should be submitted and what it would be used for. This could include relevant information about how the data will be stored, or if AI would be used to process student feedback.

Where to focus

Like in 2.4.2, this may be found in the Announcements, Introduction, Discussion Forum or  Assessment section

What to look out for, specifically

The instructions on feedback collection and usage can be given in texts, or orally. You may want to look for the description of purpose in a feedback or evaluation survey, or the introduction of a completion checklist for learning materials, a discussion thread opening, or task description of a reflection journal, etc.

Tips

Such instructions may be explicit or implicit, for example students may be asked to click on a star rating, place a heart or leave a thumbs up in Padlet without clear explanation why it should be done. Alternatively, students may see a thumbs up / thumbs down button next to a reading or smiley emoticons after an activity without instructions but assume that they are expected to click on one of them to show their opinion and attitudes. Also, students may be informed of how they should provide feedback but not of how it would be used. In such cases, a Yes But or No But would be a more feasible outcome than a Yes.

Transparency about feedback processes encourages meaningful participation and builds trust. Sharing how feedback informs course improvements supports rangatiratanga by recognising learner voice and agency. Look for whether the learning package has anywhere that explains how learner feedback will be used to improve the course.

Example

An introductory video explains when surveys occur and how previous student feedback influenced course design.

Further Info

• Student voice in tertiary education settings
• Striving for excellence – A guide for tertiary teachers. Volume 2 (p. 7)

What to do

Record the reason or evidence why a Yes / Yes But / No But / No was given for each success indicator. Start with the number, e.g. 2.1.1. and put the comment for each indicator in a new paragraph.

Where to keep these comments

There is no separate box to provide comments for each success indicator, so all comments go in the same section under all the success indicators in a standard.

Why it is necessary to have these comments

Reviewers need to meet to discuss the Combined Review, and by that time they may have forgotten why they made that decision or where specific information was found for specific success indicators as some time may have passed. Having such explanations and evidence handy also helps reviewers reach a decision where there are differences in opinion.

Tips

When a Yes was given, you may want to note where you found that piece of information as the other reviewer may have not seen it. For a Yes But, No But, or a No, it is important to note what is missing and how the issue can be addressed. These comments will be made available to the developers and staff later and your comments will be helpful for them in improving their learning package.