To test this:
At present it is not possible to synchronise course categories from an external database in moodle.
After applying this patch, create an external database. The following assumes that you're using postgres, and have a user 'moodle'.:
–
-
- Creating our test dataset
sudo su - postgres
createdb -O moodle -EUTF8 externaldb
cat <<EOF | psql externaldb
CREATE TABLE categories (
id integer NOT NULL,
title character varying(255) NOT NULL,
description text,
parent integer
);
ALTER TABLE public.categories OWNER TO moodle;
EOF
-
- Create some data in the categories
cat <<EOF | php5 | psql externaldb
<?php
echo "INSERT INTO categories (id, title, description, parent) VALUES (1, 'Top Level', 'Description for Top Level', null);\n";
\$i = 2;
while (\$i <= 6) {
echo "INSERT INTO categories (id, title, description, parent) VALUES (\$i, 'Faculty \$i', 'Description for Faculty \$i', 1);\n";
\$j = 1;
while (\$j <= 5) {
echo "INSERT INTO categories (id, title, description, parent) VALUES ({\$i}00{\$j}, 'Department {\$j}', 'Description for Department {\$i}', {\$i});\n";
\$j++;
}
\$i++;
}
EOF
–
-
- Iniial Configuration
Now configure the enrolment/database module as follows:
Site administration
-> Plugins
-> Enrolments
-> Manage enrol plugins
Enable "External database"
Navigate to Settings page for External database
Set database driver, host, user, password, and name as you've set them for your database
Under "Creation of new Course Categories"
Set "External course categories table" to:
categories
Set "Course Category id field" to:
id
Set "Course Category name field" to:
title
Leave the parent, and description fields blank for the moment as we'll use these to test updates, and re-parenting.
Now run a cli sync:
—
-
- Test Initial sync
Open a shell, and switch to your web server user. Under debian I do:
sudo su - www-data
bash
cd ~/git/moodle
php enrol/database/cli/sync.php
In Moodle, you should now have lots of top level categories.
Open one of the categories in the category editor and verify that it has:
- the correct name;
- a category id number as appears in the database (e.g. Faculty 2 has idnumber 2; Faculty 2 -> Department 3 has idnumber 2003); and
- no description.
—
-
- Test Setting a Category Parent
Navigate back to the plugin's settings page and
Set "Course Category parent field" to:
parent
Run the cli sync again
In Moodle, the categories should now be re-parented. You should now have a top level category, with five sub-categories, each having five further sub categories
—
-
- Test Setting a Category Description
Navigate back to the plugin's settings page and
Set "Course Category description field" to:
description
Set the "Format of Course Category description field" to "Plain text format"
Run the cli sync again
Open the same category again and verify that it now has the correct description and format.
—
-
- Testing Deletion
echo "DELETE FROM categories;" | psql externaldb
Run the cli sync again
All categories should be deleted with the exception of the "Top Level" category. It is not possible to delete this using the standard moodle category_delete_move
All courses should have been re-parented to the "Top Level" category.
—
-
- Testing a stupdily large dataset
Please note that the Moodle interface doesn't actually work at all well with this many. This test merely proves that the category sync doesn't hog memory
Ensure that you have the following set in your moodle config:
define('MDL_PERF' , true);
define('MDL_PERFDB' , true);
define('MDL_PERFTOLOG' , true);
define('MDL_PERFTOFOOT', true);
I stress, moodle cannot handle this number of course categories. It cannot render the Edit Courses page with this many. This is a purely a test to demonstrate that this patch is really pretty memory efficient
cat <<EOF | php5 | psql externaldb
<?php
echo "INSERT INTO categories (id, title, description, parent) VALUES (1, 'Top Level', 'Description for Top Level', null);\n";
\$i = 2;
while (\$i <= 99) {
echo "INSERT INTO categories (id, title, description, parent) VALUES (\$i, 'Faculty \$i', 'Description for Faculty \$i', 1);\n";
\$j = 1;
while (\$j <= 99) {
echo "INSERT INTO categories (id, title, description, parent) VALUES ({\$i}00{\$j}, 'Department {\$j}', 'Description for Department {\$i}', {\$i});\n";
\$j++;
}
\$i++;
}
EOF
echo "SELECT COUNT
FROM categories;" | psql externaldb
There should be 9801 course categories
Run the sync again – it may take some time and it does generate output.
After creating all of the categories it may appear to hang – It's just commiting the transaction
In my test setup (exactly as above) with a fresh Moodle installation on master, I see the following perf data:
PERF: time: 87.208004s memory_total: 38125852B (36.4MB) memory_growth: 37229628B (35.5MB) memory_peak: 81938716B (78.1MB) includecount: 75 contextswithfilters: 0 filterscreated: 0 textsfiltered: 0 stringsfiltered: 0 langcountgetstring: 9807 langcountmemcache: 9804
Memory peaks at 78MB which is relatively low for such a large dataset.
The number of reads/writes is regrettable, but is only high for the initial sync
Try performing a second sync and you'll see far lower reads/writes and memory usage:
PERF: time: 2.722942s memory_total: 39761160B (37.9MB) memory_growth: 38864936B (37.1MB) memory_peak: 57756268B (55.1MB) includecount: 75 contextswithfilters: 0 filterscreated: 0 textsfiltered: 0 stringsfiltered: 0 langcountgetstring: 7 langcountmemcache: 4 langco