Partitioning of Moodle's log Table

Author: Antonio Piedras Morente (apiedras@ub.edu) November 2010

This work is protected by a Creative Commons license (http://creativecommons.org)

Attribution-NonCommercial-ShareAlike 2.5 (http://creativecommons.org/licenses/bync-sa/2.5/)

The partition of Moodle's log table (usually mdl_log) is a solution for its growth in large
sites.
Recall that in this table, Moodle includes a lot of user activities, like login or logout, file
upload, course view, consult to course's resources, and so on.
This partitioning facilitates the maintenance of this table without affecting performance
of the database or force us to modify any script of Moodle source code.
To do possible this system it is essential that the database management system, allow
make tables "daughters" from a table "mother."
Here's an example with the database manager postgres (http://www.postgresql.org/).

Steps
1. Create tables "daughters" from the table "mother" mdl_log, by "time" field, a
table for each month. In each child table we create the same constraints and
indexes (under another name) that has the table "mother".
The intervals of field "time" by months as:
1 of the month and year from 00:00:00 until the last day of the month and year of
23:59:59.
Remember that time is saved in Moodle in timestamp with gmt's date and time
format.
NOTE: these values of "time" should not be overlap from a "daugther" table to
another because it will create problems if one interval is included in two different
tables.
For example, to create the "daugther" table for January 2010 would:

CREATE TABLE mdl_log_2010_01 (

CHECK ("time"> = 1262304000 AND "time" <= 1264982399)
) inherits (mdl_log);
ALTER TABLE mdl_log_2010_01
ADD CONSTRAINT mdl_log_2010_01_id_pk PRIMARY KEY (id);
CREATE INDEX mdl_log_2010_01_act_ix
WHERE mdl_log_2010_01
USING btree
(action);
CREATE INDEX mdl_log_2010_01_cmi_ix
WHERE mdl_log_2010_01
USING btree
(cmid);
CREATE INDEX mdl_log_2010_01_coumodact_ix
WHERE mdl_log_2010_01
USING btree
(course, module, action);
CREATE INDEX mdl_log_2010_01_tim_ix
WHERE mdl_log_2010_01
USING btree
("time");
CREATE INDEX mdl_log_2010_01_usecou_ix
WHERE mdl_log_2010_01
USING btree
(userid, course);

2. Then create the function that will distribute the records to insert into the
"mother" table to the corresponding "daugther" table depending "time" value.
NOTE: Is very important that values of the field "time" that we have in the CHECK
statement when creating "daughter" table correspond exactly in function.
For example, to create a function that distributes the records of the first 5 months
of 2010 the function would be:

CREATE OR REPLACE FUNCTION mdl_log_funcio_insert_trigger()

RETURNS TRIGGER AS $$
BEGIN
IF (TG_OP = 'INSERT') THEN
IF NEW."time" >= 1262304000 AND NEW."time" <= 1264982399 THEN
INSERT INTO mdl_log_2010_01
(id,"time",userid,ip,course,module,cmid,"action",url,info)
VALUES
(NEW.id,NEW."time",NEW.userid,NEW.ip,NEW.course,NEW.module,NEW.cmid,NEW."action",NEW.url,
NEW.info);
ELSIF NEW."time" >= 1264982400 AND NEW."time" <= 1267401599 THEN
INSERT INTO mdl_log_2010_02
(id,"time",userid,ip,course,module,cmid,"action",url,info)
VALUES
(NEW.id,NEW."time",NEW.userid,NEW.ip,NEW.course,NEW.module,NEW.cmid,NEW."action",NEW.url,
NEW.info);
ELSIF NEW."time" >= 1267401600 AND NEW."time" <= 1270079999 THEN
INSERT INTO mdl_log_2010_03
(id,"time",userid,ip,course,module,cmid,"action",url,info)
VALUES
(NEW.id,NEW."time",NEW.userid,NEW.ip,NEW.course,NEW.module,NEW.cmid,NEW."action",NEW.url,
NEW.info);
ELSIF NEW."time" >= 1270080000 AND NEW."time" <= 1272671999 THEN
INSERT INTO mdl_log_2010_04
(id,"time",userid,ip,course,module,cmid,"action",url,info)
VALUES
(NEW.id,NEW."time",NEW.userid,NEW.ip,NEW.course,NEW.module,NEW.cmid,NEW."action",NEW.url,
NEW.info);
ELSIF NEW."time" >= 1272672000 AND NEW."time" <= 1275350399 THEN
INSERT INTO mdl_log_2010_05
(id,"time",userid,ip,course,module,cmid,"action",url,info)
VALUES
(NEW.id,NEW."time",NEW.userid,NEW.ip,NEW.course,NEW.module,NEW.cmid,NEW."action",NEW.url,
NEW.info);
ELSE
RAISE EXCEPTION 'No trobada la taula pel valor de time: %', NEW."time";
END IF;
END IF; -- Fi (TG_OP = 'INSERT')
RETURN NULL;
END;
$$
LANGUAGE plpgsql;

3. Now we have to create the trigger associated with the "mother" table mdl_log.
The trigger will call the previous function that distributes the records that are
destined for the "mother" table into "daughters" tables and indicates that not
save any record in the "mother" table (executing the RETURN NULL statement).
This means that the "mother" table will have always 0 entries.

If not we put the RETURN NULL statement "would get" the opposite effect we want
with the partitioning. that means to double the size of the mdl_log table.
The trigger must be set to run before (BEFORE order) insert record in "mother"
table and for each record (FOR EACH ROW order).
NOTE: To view the records in "mother" table should execute SELECT * FROM
mdl_log that returns all rows in "daugther" tables. To verify that "mother" table
has no records should do SELECT * FROM ONLY mdl_log and will show an
empty table.

CREATE TRIGGER trigger_mdl_log_insert

BEFORE INSERT ON mdl_log
FOR EACH ROW EXECUTE PROCEDURE mdl_log_funcio_insert_trigger();

4. Finally, we will set the constraint_exclusion parameter to ON to avoid seeking

consultations on all "daughters" tables when the WHERE clause uses the "time"
field to get records (to do it you must be database administrator).
5. Now we can work normally with mdl_log table, selecting, adding and deleting
records, being the database itself which is responsible for control in which
"daugther" table has to execute the SQL statement.
NOTE: If for any reason we would roll back the partitioning procedure should only
disable the trigger, and database will return to insert records into "mother" table
mdl_log. Bearing in mind that if you do this, the records that have inserted into
"daughters" tables should insert into "mother" table manually.