In a course with visible groups enabled, loading the grades page will load a pluginfile for a group image for every group. Courses with thousands of groups will load thousands of pluginfiles, which can cause rate limiting and session locks. These are also reloaded on every page/filter.
The vast majority of courses won't have enough groups for this to matter, but when you get a course with thousands of groups the performance is horrible.
Ideally the logic should be improved to not load so many pluginfiles at once, or worst case add a setting to limit the number.
These are all loaded from inside the groups selector, if that selector has not been opened then it should not load any of these plugin files at all. They should all be lazy loaded as you scroll through the list of groups: