Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-83119

search_solr: Implement check on connectivity, space usage

XMLWordPrintable

    • MOODLE_405_STABLE
    • MOODLE_500_STABLE
    • MDL-83119-main
    • Hide

      Testing this feature requires a working Solr server.

      1. Visit Site administration / General / Advanced features and turn on Enable global search if necessary.
      2. Visit Site administration / Plugins / Search / Manage global search and select the Simple search search engine (not Solr) under Select search engine.
      3. Visit the status checks page (Site administration / Reports / System status).
        • EXPECTED: There should be no 'Solr' check.
      4. Visit Site administration / Advanced features and turn off Enable global search.
      5. Visit Site administration / Plugins / Search / Manage global search, select the Solr search engine under Select search engine, and set Index when disabled to no (default).
      6. Visit the status checks page again.
        • EXPECTED: There should still be no 'Solr' check.
      7. Visit Site administration / Advanced features and turn back on Enable global search.
      8. If necessary, set the Solr configuration (Site administration / Plugins / Search / Solr) so that it is correct and points to a working Solr server.
      9. Visit the status checks page again.
        • EXPECTED: There is a 'Solr' check, and it shows green (OK) along with the time taken to get status (e.g. 0.02s).
      10. Click the More info link next to the Solr check.
        • EXPECTED: In addition to the information shown on the front page, it also shows the size of the search index e.g. The index is using 50.1 KB on the Solr server (this could be very small for a newly-created one, or larger if there is some data).
      11. Go to the Solr configuration screen (which you can now get to by clicking Settings next to the Solr check) and change the Solr settings so that it does not connect. The easiest way to do this is to change the hostname by adding some incorrect character to the start. Save changes.
      12. Visit the status checks page again.
        • EXPECTED: The Solr check is now showing red (Error) with a suitable message e,g, Cannot connect to Solr server. (This may add up to 5 second delay to the pages.)
      13. Click the More info link.
        • EXPECTED: In more info, it shows you the full exception for the failed connection attempt.
      14. Go to the Solr configuration screen (Settings next to Solr check) and fix the settings back to correct values (correct the hostname); also, under Index size limit set a value for the index size which is very slightly higher than than the real size for the index you saw in step N. Save changes.
      15. Visit the status checks page.
        • EXPECTED: The Solr check is now showing yellow (Warning) and the index size message e.g. The index is using 50.1 KB on the Solr server.
      16. Go to the Solr configuration screen (Settings next to Solr check) and set Index size limit less than the real size for the index you saw in step N. Save changes.
      17. Visit the status checks page.
        • EXPECTED: The Solr check is now showing red (Error) and Index larger than specified size.
      Show
      Testing this feature requires a working Solr server. Visit Site administration / General / Advanced features and turn on Enable global search if necessary. Visit Site administration / Plugins / Search / Manage global search and select the Simple search search engine (not Solr) under Select search engine . Visit the status checks page ( Site administration / Reports / System status ). EXPECTED: There should be no 'Solr' check. Visit Site administration / Advanced features and turn off Enable global search . Visit Site administration / Plugins / Search / Manage global search , select the Solr search engine under Select search engine , and set Index when disabled to no (default). Visit the status checks page again. EXPECTED: There should still be no 'Solr' check. Visit Site administration / Advanced features and turn back on Enable global search . If necessary, set the Solr configuration ( Site administration / Plugins / Search / Solr ) so that it is correct and points to a working Solr server. Visit the status checks page again. EXPECTED: There is a 'Solr' check, and it shows green ( OK ) along with the time taken to get status (e.g. 0.02s ). Click the More info link next to the Solr check. EXPECTED: In addition to the information shown on the front page, it also shows the size of the search index e.g. The index is using 50.1 KB on the Solr server (this could be very small for a newly-created one, or larger if there is some data). Go to the Solr configuration screen (which you can now get to by clicking Settings next to the Solr check) and change the Solr settings so that it does not connect. The easiest way to do this is to change the hostname by adding some incorrect character to the start. Save changes. Visit the status checks page again. EXPECTED: The Solr check is now showing red ( Error ) with a suitable message e,g, Cannot connect to Solr server . (This may add up to 5 second delay to the pages.) Click the More info link. EXPECTED: In more info, it shows you the full exception for the failed connection attempt. Go to the Solr configuration screen ( Settings next to Solr check) and fix the settings back to correct values (correct the hostname); also, under Index size limit set a value for the index size which is very slightly higher than than the real size for the index you saw in step N. Save changes. Visit the status checks page. EXPECTED: The Solr check is now showing yellow ( Warning ) and the index size message e.g. The index is using 50.1 KB on the Solr server . Go to the Solr configuration screen ( Settings next to Solr check) and set Index size limit less than the real size for the index you saw in step N. Save changes. Visit the status checks page. EXPECTED: The Solr check is now showing red ( Error ) and Index larger than specified size .
    • Hide

      Code verified against automated checks.

      Checked MDL-83119 using repository: https://github.com/sammarshallou/moodle.git

      More information about this report

      Built on: Mon 21 Oct 2024 08:36:24 AM UTC

      Show
      Code verified against automated checks. Checked MDL-83119 using repository: https://github.com/sammarshallou/moodle.git main (0 errors / 0 warnings) [branch: MDL-83119-main | CI Job ] More information about this report Built on: Mon 21 Oct 2024 08:36:24 AM UTC
    • Show
      Launching automatic jobs for branch MDL-83119 -main https://ci.moodle.org/view/Testing/job/DEV.02%20-%20Developer-requested%20PHPUnit/17054/ PHPUnit (sqlsrv) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/58056/ Behat (NonJS - boost and classic) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/58057/ Behat (Firefox - boost) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/58058/ Behat (Firefox - classic) https://ci.moodle.org/view/Testing/job/DEV.01%20-%20Developer-requested%20Behat/58059/ App tests (stable app version) Built on: Mon Oct 21 06:25:27 AM UTC 2024

      When using the Solr search engine, there is currently no 'status check' (via the Check API) that the search engine is up/responding.

      Additionally we have some issues in our installation with the size used by Solr index. Solr has stringent disk space requirements and when using 3rd party hosting where you pay for the exact storage space, it's desirable to monitor the size so that you notice before the hosting provider if it's getting close to the limits.

      I propose adding a new status check. The check will only be enabled if Solr is configured. It reports:

      • OK - If it can connect to Solr and the index size is comfortably less than specified threshold.
      • Warning - If it can connect to Solr but the index size exceeds 90% of threshold
      • Error - If it can't connect to Solr, or if it can but the index size exceeds threshold.

      Status showing OK:

      Status showing index too large:

      Status showing a connection failure (host address is wrong in this case):

      To make the warning state work, there will also be a new admin setting search_solr/indexsizelimit. By default this will be set to 0 (off, no limit).

        1. screenshot-1.png
          16 kB
          Sam Marshall
        2. screenshot-ok.png
          14 kB
          Sam Marshall
        3. screenshot-2.png
          70 kB
          Sam Marshall
        4. screenshot-adminsetting.png
          9 kB
          Sam Marshall
        5. 83119 test - no solr on simple search.png
          140 kB
          Andrew Gosali
        6. 83119 test - no solr when solr selected but global search disabled.png
          146 kB
          Andrew Gosali
        7. 83119 test - solr appear.png
          154 kB
          Andrew Gosali
        8. 83119 test - solr index size info.png
          51 kB
          Andrew Gosali
        9. 83119 test - solr error when connection fail.png
          59 kB
          Andrew Gosali
        10. 83119 test - solr error have detailed info.png
          296 kB
          Andrew Gosali
        11. 83119 test - solr able to show size error.png
          51 kB
          Andrew Gosali
        12. 83119 test - solr able to show size warning.png
          55 kB
          Andrew Gosali

            quen Sam Marshall
            quen Sam Marshall
            Katie Ransom Katie Ransom
            Huong Nguyen Huong Nguyen
            Andrew Gosali Andrew Gosali
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 7 hours
                7h

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.