It is recommended that you perform these steps in the order they are presented as they have been sequenced in a way that attempts to reduce toggling settings on and off as much as possible.

Once all the tests have been covered, it is encouraged that you explore different uses (eg in different courses/activities) and fiddle with the settings in an effort to identify any issues not covered here.

Prerequisites

1. This test should be completed by Moodle HQ.
2. You will require a valid OpenAI key and organisation ID. Reach out to the Platform team for credentials.
3. It is recommended to use a fresh install of Moodle without any AI set up yet.
4. Restore the course using the file attached to this issue (you may have to run cron to complete the restore)

Tests

AI providers and actions

1. Log in as admin.
2. Go to Site admin -> General -> AI -> Manage settings for AI providers.
3. CONFIRM that the 'OpenAI API Provider' is disabled by default.
4. Enable 'OpenAI API Provider'.
5. Click on the 'Settings' link in the 'OpenAI API Provider' row.
6. Enter a valid 'OpenAI API key' and 'Organization ID' in the appropriate fields.
7. Save changes.
8. CONFIRM there are three provider actions under the 'Provider actions' heading and that they are all enabled by default.
9. Disable 'Generate text' and 'Generate image'.
10. Go to Site admin -> General -> AI -> Manage settings for AI placements.
11. CONFIRM there are two placements called 'Course assistance placement' and 'HTML Text editor placement' and that they are both disabled by default.
12. Enable 'HTML Text Editor Placement'.
13. Click the 'Settings' link for 'HTML Text Editor Placement'.
14. CONFIRM there are error messages displayed for both 'Generate text' and 'Generate image'.
15. Go back to "OpenAI API Provider" settings page and re-enable 'Generate text' and 'Generate image'.
16. Go back to the 'Settings' for 'HTML Text Editor Placement'.
17. CONFIRM there are no errors there now.

HTML editor placement test

1. Go to the restored course (see prerequisites).
2. Add a new 'Page' resource.
3. Scroll to one of the editors.
4. CONFIRM there is an icon called 'Generate AI content' icon (sparkles) in the toolbar.
5. Click on the the 'Generate AI content' dropdown icon.
6. CONFIRM there are two options called 'AI generate text' and 'AI generate image'.
7. Click on the editor's 'Insert' menu.
8. CONFIRM there are two options called 'AI generate text' and 'AI generate image'.
9. Go to Site admin -> General -> AI -> Manage settings for AI placements.
10. Click the 'Settings' link for 'HTML Text Editor Placement'.
11. Disable 'Generate text'.
12. Return to your 'Page' resource editor.
13. CONFIRM that there is no longer a dropdown menu for AI generate content and you can only see the 'AI generate image' icon on the toolbar.
14. Click the 'Insert' menu and confirm you can only see 'AI generate image' ('AI generate text' should not be there).
15. Repeat from Step 9, but swap the disabling of text for image.
16. Finally, disable both text and image and confirm you cannot see either in the editor.
17. Re-enable both text and image and return to the 'Page' resource editor.

Privacy policy test

1. From the editor, click 'AI generate text'.
2. CONFIRM that a modal window appears with the AI usage policy.
3. Click 'Decline'.
4. CONFIRM the AI modal closes.
5. Click 'AI generate text' again.
6. CONFIRM that a modal window appears with the AI usage policy again.
7. Click 'Accept and continue'.
8. CONFIRM you can now access the 'Generate text using AI' window.

Generate text test (1)

1. In the text area, type in something like 'Write a short poem'.
2. Click 'Generate text'.
3. CONFIRM you see the expected results.
4. Click 'Regenerate'.
5. CONFIRM you see a variation of your original request.
6. CONFIRM that you cannot edit the generated text in the modal.
7. Click on 'Insert'.
8. CONFIRM that the text as been added to the editor and is now editable.
9. Click 'Save and display'.
10. CONFIRM that the generated text has been inserted correctly.

Generate text test (2)

1. Go to Site admin -> General -> AI -> Manage settings for AI providers.
2. Click on the 'Settings' link in the 'OpenAI API Provider' row.
3. Under the 'Provider actions' heading, locate the 'Generate text' row and click 'Settings'.
4. Edit the 'System instruction' text by adding '3. Add the words TEST TEST TEST' to the start of the results'.
5. Save changes.
6. Return to the 'Page' resource again.
7. Generate some text again.
8. CONFIRM that your result has prepended the words 'TEST TEST TEST' (LLMs can get things wrong, repeat this if the results are not what you would expect).
9. CONFIRM that you cannot edit the generated text in the modal.
10. Click on 'Insert'.
11. CONFIRM that the text as been added to the editor and is now editable.
12. Click 'Save and display'.
13. CONFIRM that the generated text has been inserted correctly.

Generate image test (1)

1. From the editor, click 'AI generate image'.
2. Enter a prompt to generate an image (e.g. 'Cat on a mat wearing a hat').
3. Click on 'Generate image'.
4. CONFIRM you see the loading wheel and messages as the image is generated.
5. CONFIRM that the resulting image is placed in the right-hand-side of the modal.
6. CONFIRM there is 'Generated by AI' text added to the bottom of the image and that it is legible.
7. Click on 'Regenerate'.
8. CONFIRM the results are a variation on the original prompt.
9. Click 'Next'.
10. CONFIRM that the modal shows the generated image with the prompt set as the image description (e.g. 'Cat on a mat wearing a hat - Generated by AI').
11. Click 'Save'.
12. CONFIRM the image has been added to the editor.
13. Click on the image and select the image icon to edit the image.
14. CONFIRM that the 'Image details' modal appears with the image and description.
15. Adjust the size of the image and press 'Save'
16. CONFIRM that the size is updated.
17. Click 'Save and display'.
18. CONFIRM that the generated image has been inserted correctly.
19. Repeat this test using different orientations (e.g. square, landscape, portrait) - to save some time, no need to re-test "regenerate" with these since that feature has been tested in the first run through.

Generate image test (2)

1. From the editor, click 'AI generate image'.
2. Enter a prompt to generate an image that is very long (150+ characters)
3. Click on 'Generate image'.
4. After generation, click 'Next'.
5. CONFIRM that the lengthy prompt text has truncated and has a '...' at the end. 

Course assistance - summarise text

1. Go to the restored course.
2. View the 'Grandma's Kimchi' page.
3. CONFIRM there is no 'Summarise' button displayed above the main page content.
4. Go to Site admin -> General -> AI -> Manage settings for AI placements.
5. Enable 'Course assistance placement'.
6. Go back to the 'Grandma's Kimchi' page.
7. CONFIRM there is now a 'Summarise' button above the text/main page content.
8. Click the 'Summarise' button.
9. CONFIRM that the right-hand-side drawer opens with the 'AI summary' heading and generated text.
10. CONFIRM that the generated text is actually summarising the page contents.
11. CONFIRM below the summary states 'Generated by AI'.
12. Click on the 'Regenerate' button.
13. CONFIRM there is a variation to the original results displayed.
14. Note the content of the results.
15. Close the right-hand-side drawer.
16. Click the 'Summarise' button again.
17. CONFIRM that the draw reopens with the same text that you previously noted (and not a new result).
18. Refresh the page and click the 'Summarise' button again.
19. CONFIRM there is a newly generated summary (and not the one you previously noted).
20. With the drawer open, attempt to click the original 'Summarise' button.
21. CONFIRM you cannot click it with the drawer open (ie trying to click it has no effect).
22. Click the 'Copy' button below the generated text.
23. CONFIRM the text has been copied by pasting into your favourite text editor and verifying that the contents exactly match those produced by in the AI summary.

API key error

1. Go to Site admin -> General -> AI -> Manage settings for AI providers.
2. Click on the 'Settings' link in the 'OpenAI API Provider' row.
3. Change the 'OpenAI API key' value to an invalid key.
4. Save changes.
5. Return to the 'Grandma's Kimchi' page.
6. Click on 'Summarise'.
7. CONFIRM the drawer opens with the message 'Something went wrong'.
8. Open your browser console.
9. CONFIRM there is a message about the incorrect API key.
10. Return to a text editor and try to generate an image and some text.
11. CONFIRM that in both cases you receive the 'Something went wrong' message.