On our busy system, the roledefs cache is purged approximately every 3 seconds. This is bad for performance and would be extremely bad if using memcache as it will cause all caches to be purged that are stored on the same memcache.

It has not crashed the OU's system, but only because we have a hacked version of memcache which ignores most of the purge requests.

(As a workaround on any system affected by this, the roledefs cache could be configured to use a different memcache, or not use memcache.)

This was caused by these lines from MDL-49398 (where the roledefs cache was introduced):
https://github.com/moodle/moodle/blob/master/lib/accesslib.php#L224-L225

Everything else done by accesslib_clear_all_caches function clears only private global/static variables and per-request/-session caches - except this added part, which clears an application cache.

The usual path for triggering this cache purge is the context::reload_if_dirty() function which spots a context that has been modified since the user's permissions were loaded, and causes all capabilities to be reloaded. This behaviour is correct, but it is only trying to reload the user's session cache: it probably should not clear the application cache that lists role definitions. With the current code, if a context is modified in any way, and if a number of users who are logged into the system continue to view their next page, each time the system will reload their (individual) capabilities, and clear the (application-wide) MUC cache.

I am not sure what is the correct solution to this bug. There are a couple of possibilities:

1. Change it to call accesslib_clear_role_cache which deletes the specific keys instead of purging (much less bad but still makes this individual cache less effective on a heavily-loaded system).
2. Change it so accesslib_clear_all_caches does not clear this application cache - and change any other code that might be relying on that behaviour to explicitly call accesslib_clear_role_cache.

I think the second option is better because it seems to me that accesslib_clear_all_caches was really intended to clear caches within the specific user session, not system-wide caches (presuambly roledefs cache should only be cleared when somebody edits a role!)