Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-84806

AI actions must inform assistive technology users about the results of the actions' processing

XMLWordPrintable

    • MOODLE_405_STABLE, MOODLE_500_STABLE
    • MOODLE_405_STABLE, MOODLE_500_STABLE
    • MDL-84806-405
    • MDL-84806-500
    • MDL-84806-main
    • Hide

      Setup

      1. Log in as admin
      2. Go to Site admin -> General -> AI -> AI providers
      3. Create an AI instance with either of the available providers. Reach out if you need credentials or install the DummyAI plugin, which doesn't need credentials (for Moodle 4.5 the MOODLE_405_STABLE branch should be used): 

        git clone https://github.com/moodlemobile/moodle-aiprovider_dummyai ai/provider/dummyai

      4. Enable the new provider instance.
      5. Go to Site admin -> General -> AI -> AI placements
      6. Enable both 'Course assistance placement' and 'Text editor placement'
      7. Ensure all actions are enabled: Explain text, Summarise text, Generate text, Generate image
      8. Have a screen reader of some sort available for testing (Recommended to test with NVDA, JAWS, or equivalent). 

      Test - Explain text and Summarise text

      1. Log in as admin
      2. Create a course
      3. Create a 'Page' activity
      4. In the 'Page content' field, enter in a paragraph of any text
      5. Click 'Save and display'
      6. Turn on your screen reader
      7. In your browser's 'Network' tab, enable throttling (slow the connection)
      8. Using the 'AI features' dropdown, choose 'Explain'
      9. CONFIRM that you hear the following:
        • 'Generating your response'
        • 'Cancel'
        • The actual read out response of the explain action
      10. Using the 'AI features' dropdown, choose 'Summarise'
      11. CONFIRM that you hear the following:
        • 'Generating your response'
        • 'Cancel'
        • The actual read out response of the summarise action
      12. Turn off throttling
      13. Go back to Site admin -> General -> AI -> AI providers
      14. Edit the provider and alter the API key (we want it to fail)
      15. Go back to the page resource
      16. Using the 'AI features' dropdown, choose 'Explain'
      17. CONFIRM that you hear the following:
        • 'Something went wrong'
        • 'Try again'

      Test -  Generate text and Generate image

      1. Log in as admin
      2. Ensure the API key for the AI provider is correct (reset from last test)
      3. Go to your edit profile page and locate the text editor
      4. Choose 'AI generate text' from the AI dropdown
      5. Type in anything you want to generate a response
      6. Turn on your screen reader
      7. Click on 'Generate text'
      8. CONFIRM that you hear the following:
        • The actual read out response of the generate text action
      9. Click on 'Cancel'
      10. Choose 'AI generate image' from the AI dropdown
      11. Type in anything you want to generate a response
      12. Click on 'Generate image'
      13. CONFIRM that you hear the following:
        • The looping loading messages -> 'Processing your request..., Almost done...' etc.
        • The prompt text you typed in
      14. Go back to Site admin -> General -> AI -> AI providers
      15. Edit the provider and alter the API key (We want it to fail. If you were using DummyAI, change it to one of the other providers and just put in any config.)
      16. Go back and generate some text again
      17. CONFIRM that you hear the following:
        • 'Something went wrong'
        • 'Try again later'
      Show
      Setup Log in as admin Go to Site admin -> General -> AI -> AI providers Create an AI instance with either of the available providers. Reach out if you need credentials or install the DummyAI plugin, which doesn't need credentials (for Moodle 4.5 the MOODLE_405_STABLE branch should be used): git clone https: //github.com/moodlemobile/moodle-aiprovider_dummyai ai/provider/dummyai Enable the new provider instance. Go to Site admin -> General -> AI -> AI placements Enable both 'Course assistance placement' and 'Text editor placement' Ensure all actions are enabled: Explain text, Summarise text, Generate text, Generate image Have a screen reader of some sort available for testing (Recommended to test with NVDA, JAWS, or equivalent).  Test - Explain text and Summarise text Log in as admin Create a course Create a 'Page' activity In the 'Page content' field, enter in a paragraph of any text Click 'Save and display' Turn on your screen reader In your browser's 'Network' tab, enable throttling (slow the connection) Using the 'AI features' dropdown, choose 'Explain' CONFIRM that you hear the following: 'Generating your response' 'Cancel' The actual read out response of the explain action Using the 'AI features' dropdown, choose 'Summarise' CONFIRM that you hear the following: 'Generating your response' 'Cancel' The actual read out response of the summarise action Turn off throttling Go back to  Site admin -> General -> AI -> AI providers Edit the provider and alter the API key (we want it to fail) Go back to the page resource Using the 'AI features' dropdown, choose 'Explain' CONFIRM that you hear the following: 'Something went wrong' 'Try again' Test -  Generate text and Generate image Log in as admin Ensure the API key for the AI provider is correct (reset from last test) Go to your edit profile page and locate the text editor Choose 'AI generate text' from the AI dropdown Type in anything you want to generate a response Turn on your screen reader Click on 'Generate text' CONFIRM that you hear the following: The actual read out response of the generate text action Click on 'Cancel' Choose 'AI generate image' from the AI dropdown Type in anything you want to generate a response Click on 'Generate image' CONFIRM that you hear the following: The looping loading messages -> 'Processing your request..., Almost done...' etc. The prompt text you typed in Go back to  Site admin -> General -> AI -> AI providers Edit the provider and alter the API key (We want it to fail. If you were using DummyAI, change it to one of the other providers and just put in any config.) Go back and generate some text again CONFIRM that you hear the following: 'Something went wrong' 'Try again later'
    • Hide

      Code verified against automated checks with warnings.

      Checked MDL-84806 using repository: https://github.com/davewoloszyn/moodle.git

      Should these errors be fixed?

      Built on: Fri May 16 07:29:23 UTC 2025

      Show
      Code verified against automated checks with warnings. Checked MDL-84806 using repository: https://github.com/davewoloszyn/moodle.git MOODLE_405_STABLE (0 errors / 1 warnings) [branch: MDL-84806-405 | CI Job ] overview (0/1) , phplint (0/0) , phpcs (0/0) , js (0/0) , css (0/0) , phpdoc (0/0) , commit (0/0) , savepoint (0/0) , thirdparty (0/0) , externalbackup (0/0) , grunt (0/0) , shifter (0/0) , mustache (0/0) , gherkin (0/0) , Warn: The MDL-84806 -405 branch at https://github.com/davewoloszyn/moodle.git has not been rebased recently (>20 days ago). MOODLE_500_STABLE (0 errors / 0 warnings) [branch: MDL-84806-500 | CI Job ] main (0 errors / 0 warnings) [branch: MDL-84806-main | CI Job ] Should these errors be fixed? Built on: Fri May 16 07:29:23 UTC 2025
    • Hide

      Launching automatic jobs for branch MDL-84806-405

      Launching automatic jobs for branch MDL-84806-500

      Launching automatic jobs for branch MDL-84806-main

      Built on: Tue May 27 07:26:53 UTC 2025

      Show
      Launching automatic jobs for branch MDL-84806 -405 https://ci.moodle.org/view/Testing/job/DEV.02%20-%20Developer-requested%20PHPUnit/19465/ PHPUnit (sqlsrv / complete) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66595/ Behat (NonJS - boost and classic / --tags @editor_tiny) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66596/ Behat (Firefox - boost / --tags @editor_tiny) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66597/ Behat (Firefox - classic / --tags @editor_tiny) Launching automatic jobs for branch MDL-84806 -500 https://ci.moodle.org/view/Testing/job/DEV.02%20-%20Developer-requested%20PHPUnit/19466/ PHPUnit (sqlsrv / complete) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66598/ Behat (NonJS - boost and classic / --tags @editor_tiny) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66599/ Behat (Firefox - boost / --tags @editor_tiny) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66600/ Behat (Firefox - classic / --tags @editor_tiny) Launching automatic jobs for branch MDL-84806 -main https://ci.moodle.org/view/Testing/job/DEV.02%20-%20Developer-requested%20PHPUnit/19467/ PHPUnit (sqlsrv / complete) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66601/ Behat (NonJS - boost and classic / --tags @editor_tiny) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66602/ Behat (Firefox - boost / --tags @editor_tiny) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66603/ Behat (Firefox - classic / --tags @editor_tiny) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/66604/ App tests (stable app version) / --tags @editor_tiny) Built on: Tue May 27 07:26:53 UTC 2025
    • 2
    • Team Hedgehogs 2025 Sprint 1.3, Team Hedgehogs 2025 Sprint 1.4, Team Hedgehogs 2025 Sprint 2.0, Team Hedgehogs 2025 Sprint 2.1

      To meet the WCAG Level AA Success Criterion 4.1.3—Status Messages, we must ensure that we inform users of the processing result whenever an AI action has generated results.

      From the auditors:

      When the site returns results generated by AI, the result is displayed on the screen, but assistive technology does not inform the user that the process has been completed.

      This applies to both Generate text and Generate image AI actions. This may also apply to the Explain (5.0 only) and Summarise actions.

        1. (1) 11 Passed -- (Main)MDL-84806.png
          95 kB
          Kim Jared Lucas
        2. (1) 17 Passed -- (Main)MDL-84806.png
          212 kB
          Kim Jared Lucas
        3. (1) 9 Passed -- (Main)MDL-84806.png
          239 kB
          Kim Jared Lucas
        4. (2) 13 Passed -- (Main)MDL-84806.png
          591 kB
          Kim Jared Lucas
        5. (2) 17 Passed -- (Main)MDL-84806.png
          161 kB
          Kim Jared Lucas
        6. (2) 8 Passed -- (Main)MDL-84806.png
          251 kB
          Kim Jared Lucas

            david.woloszyn@moodle.com David Woloszyn
            jpataleta Jun Pataleta
            Raquel Ortega Raquel Ortega
            Jun Pataleta Jun Pataleta
            Kim Jared Lucas Kim Jared Lucas
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 week, 1 day, 7 hours, 12 minutes
                1w 1d 7h 12m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.