Recently I discovered our Moodle gradebook was playing up and not correctly recording marks from assignments as well as been unable to assign categories from quizes.

I tracked the problem down to the Moodle database losing all its default entries for its tables. The cause is unknown but is quite likely the UTF conversion utility I ran previously to make the system compatible with Asian languages for LOTE.

To fix this problem I needed to recreate the Moodle DB structure and then reimport the data.

Backup

First make a backup. I made a mistake the first time I tried repairing this and the backup I made saved me.

mysqldump -u backupacc -p -Q --opt moodle > moodleBackup.sql

I created a duplicate site with a clean DB (copy the moodle folder and create a new DB) and went through the install. This needs to have all your modules and blocks to ensure all tables are created.

I then dumped the structure from this new DB and the old DB and compared them using Meld (or any other diffing program will work).

 mysqldump -u backupacc -p --no-data --skip-add-drop-table moodletest2 > moodleGoodStruc.sql
mysqldump -u backupacc -p -d --skip-add-drop-table moodle > moodleOldStruc.sql

Using Meld determine if there are any extra tables or fields in your current system that don’t exist in the clean one.

Then you need to determine if it is safe to delete those tables / field in your main DB or if they need to be added to the clean system. e.g. grade_items_history.decimals and grade_items_history.display were in the main DB but not the new one. After searching for it discovered they can be removed (MDL-15985).

After getting them to match you need to dump your Moodle data from your main site.

You can speed up the next steps by reducing the size of backup_log if you wish. Just do a

Delete
FROM `mdl_backup_log`
WHERE `time` <1242777600

Where the time code is calculated as a unix time code from about a month before todays date. This can reduce alot of space of your DB, and improve import/export times.

mysqldump -u backupacc -p  --no-create-info --extended-insert --complete-insert moodle > moodleData200906171559.sql

You need the –complete-insert option to ensure each insert is labelled with field name, in case field order is different. I didn’t do this the first time and needed to restore the backup.

Testing Merge

Now you need to test it. Create a new empty DB and import the Structure then import your data.

mysql -u root -p moodleTest3 < moodleStruc.sql
mysql -u root -p moodleTest3 < moodleData200906171559.sql

Check that no errors occurred during import, this is when I discovered most of my problems with extra non existent field.

Prep for Applying

Put your site in Maintenance mode so no users can login while you are testing this. (i.e. when you restore data site will already be in maintenance mode)

When everything is OK do another backup, as after this you are going to delete it all. (I recommend taking your site completely offline for this, not just maintenance mode. Edit your config.php and temporarily change either the username, password or db name or your database settings.)

Now re-export the Data again (assuming your site has been running while you have been doing tests)

mysqldump -u backupacc -p  --no-create-info --extended-insert --complete-insert moodle > moodleData200906171559.sql

Applying to Site

WARNING: HAVE A GOOD RELIABLE BACKUP BEFORE THIS STEP.  THIS WILL DESTROY YOUR MOODLE SITE.

Now use phpMyAdmin to drop every table in the Moodle main DB. Just use Check All at the bottom and select Drop.

Now you need to redo your test restore above but to your main DB.

Once complete reconnect your config.php file and check everything on your site looks the same. Open some courses, check some gradebooks, have a look around.

When its OK take out of Maintenance mode and you should be good to go.

Find Empty Moodle Courses

If you use an auto-enrolment module you will often find you have lots of courses created that are never used. We have used the LDAP and Database enrolment before and both have created large numbers of courses. Some are not used by teachers,others are just used as child courses for year wide meta courses. Either way at the end of the year you may wish to identify any class that hasn’t been used. (Perhaps to delete using the Bulk Deletion options).

No Content

Following SQL code I’ve written will generate a list containing a sum of most of the resources and activities a course has, allowing you to quickly identify those with little to no content. It bases this count on having no labels, resources, assignments, etc.

SELECT m.id, m.`shortname` , m.fullname, cCount.totalcount
FROM mdl_course m
LEFT JOIN (

SELECT courseCount.course, sum( courseCount.subcount ) AS totalcount
FROM (

SELECT course, count( * ) AS subcount
FROM mdl_resource
GROUP BY course
UNION ALL SELECT course, count( * ) AS subcount
FROM mdl_quiz
GROUP BY course
UNION ALL SELECT course, count( * ) AS subcount
FROM mdl_assignment
GROUP BY course
UNION ALL SELECT course, count( * ) AS subcount
FROM mdl_survey
GROUP BY course
UNION ALL SELECT course, count( * ) AS subcount
FROM mdl_label
GROUP BY course
UNION ALL SELECT course, count( * ) AS subcount
FROM mdl_glossary
GROUP BY course
UNION ALL SELECT course, count( * ) AS subcount
FROM mdl_homework
GROUP BY course
UNION ALL SELECT course, count( * ) AS subcount
FROM mdl_wiki
GROUP BY course
) AS courseCount
GROUP BY courseCount.course
) AS cCount ON cCount.course = m.id

You may need to edit above code if you do not have an activity installed (eg homework).

No Users

The following code generates a list of all your courses together with how many students are enrolled in each. Useful to find out if you have any courses with no one enrolled.

SELECT cr.shortname, cr.fullname, count( ra.id ) AS enrolled
FROM `mdl_course` cr
JOIN `mdl_context` ct ON ( ct.instanceid = cr.id )
LEFT JOIN `mdl_role_assignments` ra ON ( ra.contextid = ct.id )
WHERE ct.contextlevel =50
GROUP BY cr.shortname, cr.fullname
ORDER BY `enrolled` ASC

You can export either of the above queries into Excel and manipulate it from there. Order by the counts then copy the short names’ of courses to delete, paste into a text file and upload to the Bulk Course Deletion addon. Goodbye excess courses.

Data Cleanup: Roles without Users

Sometimes when a user gets deleted Moodle doesn’t clean up after it self as well as it should. The following code will list all the rows in your role assignments table that no longer match to a user:

SELECT *
FROM `mdl_role_assignments`
WHERE `userid` NOT
IN (
SELECT id
FROM mdl_user
)

If you wish to the delete all of these just run following. As always make sure you have a good backup before deleting anything from DB.

DELETE
FROM `mdl_role_assignments`
WHERE `userid` NOT
IN (
SELECT id
FROM mdl_user
)