Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-23754 Performance improvements META
  3. MDL-27120

Backup / Restore / Import performance optimisations

XMLWordPrintable

    • MOODLE_20_STABLE, MOODLE_23_STABLE
    • MOODLE_23_STABLE
    • wip-mdl-27120
    • Hide

      Test 1:

      1. Backup a course (Go to course and click backup), no error should be produced and course should be backed up.
      2. Restore same course and make sure course is restored properly.

      Test2:

      1. Backup multiple course. Open multiple/different browsers and backup different courses at the same time
      2. restore these courses at the same time in different browsers and make sure they get restored properly.

      Test 3:

      1. Set automated backup (Settings -> Site administrator -> Courses -> Backups -> Automated backup setup)
      2. Run cron (/admin/cron.php), no error should be produced.

      Test 4:
      Run phpunit and make sure you don't get any error (especially backup/converter/moodle1/tests/lib_test.php)

      Show
      Test 1: Backup a course (Go to course and click backup), no error should be produced and course should be backed up. Restore same course and make sure course is restored properly. Test2: Backup multiple course. Open multiple/different browsers and backup different courses at the same time restore these courses at the same time in different browsers and make sure they get restored properly. Test 3: Set automated backup (Settings -> Site administrator -> Courses -> Backups -> Automated backup setup) Run cron (/admin/cron.php), no error should be produced. Test 4: Run phpunit and make sure you don't get any error (especially backup/converter/moodle1/tests/lib_test.php)

      Backup / restore is performing slowly which is causing many imports to fail for large courses.

      In addition to this, an interactive backup/restore/import does not increase max_execution_time, so max_execution_time will remain at PHP's default (which could be something like 60 secs). In Moodle 1.9, both backup and restore would set max execution time to 3000 (50 minutes).

      I have also identified 3 major bottlenecks so far in the backup code (see attached profiles for further evidence):
      1) base_plan::get_setting() function does a sequential scan lookup to find settings, instead of hash-based, which was very slow and called $setting->get_name() over 1 million times and taking ~30s (see get_setting.png)
      2) The various backup_ui_stage classes in backup/util/ui/backup_ui_stage.class.php each contain an initialise_stage_form() method which is using QuickForms in an inefficient way. It calls $form->add_setting() which calls to moodle_form->setDefault which calls QuickForm->setDefaults which fires an "updateValue" even to all quickform elements. So it has n*n complexity (see quickforms1.png and quickforms2.png)
      3) The backup_xml_transformer's constructor calls register_links_encoders() which causes an expensive call to get_plugins_list(). The problem is, the class can be constructed 1500+ times during a backup, so it needs a cache! (see getplugins1.png and getplugins2.png)

      I have patches for the above 3 issues which I will put on GitHub and link here.

        1. quickforms2.png
          quickforms2.png
          59 kB
        2. quickforms1.png
          quickforms1.png
          64 kB
        3. getplugins2.png
          getplugins2.png
          24 kB
        4. getplugins1.png
          getplugins1.png
          24 kB
        5. get_setting.png
          get_setting.png
          36 kB

            rajeshtaneja Rajesh Taneja
            ashleyholman Ashley Holman (Inactive)
            Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
            Aparup Banerjee Aparup Banerjee
            Aparup Banerjee Aparup Banerjee
            Votes:
            10 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.