15 August 2012, 9:38

We have a site running 1.50dev connected to a Moodle site. For months it has been working as advertised.

Recently, new users on the Moodle site have been unable to connect to the Mahara site via the MNET connection. When clicked, browsers signal that the page is loading, but never does. New accounts are no longer automatically created on Mahara. Previously existing users can still access Mahara however via the Moodle MNET connection.

The only errors logged show (IP's and site disguised):

[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##] [WAR] b2 (lib/pieforms/pieform.php:1406) Pieform element institution_background wants to add headdata to the page, but it's too late. Build your form before generating the page., referer: http://mahara.university.edu/admin/users/institutions.php
[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##] Call stack (most recent first):, referer: http://mahara.university.edu/admin/users/institutions.php
[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##]   * log_message("Pieform element institution_background wants to ad...", 8, true, true) at /var/www/lib/errors.php:109, referer: http://mahara.university.edu/admin/users/institutions.php
[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##]   * log_warn("Pieform element institution_background wants to ad...") at /var/www/lib/pieforms/pieform.php:1406, referer: http://mahara.university.edu/admin/users/institutions.php
[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##]   * Pieform->build_element_html(array(size 9)) at /var/www/lib/pieforms/pieform.php:652, referer: http://mahara.university.edu/admin/users/institutions.php
[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##]   * Pieform->build() at /var/www/lib/pieforms/pieform.php:162, referer: http://mahara.university.edu/admin/users/institutions.php
[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##]   * Pieform::process(array(size 5)) at /var/www/lib/pieforms/pieform.php:71, referer: http://mahara.university.edu/admin/users/institutions.php
[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##]   * pieform(array(size 5)) at /var/www/admin/users/institutions.php:491, referer: http://mahara.university.edu/admin/users/institutions.php
[Wed Aug 15 08:21:15 2012] [error] [client ##.##.##.##] , referer: http://mahara.university.edu/admin/users/institutions.php

These errors are repeated. There is no PostgreSQL database error reported and no PHP error reported.

We have deleted the MNET connection and reestablished it with new public keys to no avail. (Had to go to the databases of both Moodle & Mahara to delete the keys as no path within Mahara admin screens and Moodle admin function has a bug preventing deletion of a peer.)

Any guidance?

Thanks!

15 August 2012, 10:42

More info:

We have also noted that an administrator on the Mahara site cannot now add a user to the institution as after clicking the Add button, again, the browser (several tried) indicates it's trying to connect but never does.

Further, unless/until the user deletes the Mahara cookie (at least in that browser session), he cannot navigate to any other Mahara page. Deleting the cookie, of course, forces the user to re-login

A test instance of Mahara, same server, same codebase, same Moodle instance to connect to, new database, works as it should.

Ideas?

Thanks!

15 August 2012, 20:51

Hi Worth,

Have you changed your environment at all recently? URL's, upgraded code (in moodle or mahara), upgraded apache/php, etc?

Is this the same setup as your other site, if so can you provide the config.php (stripped of the correct wwwroot etc).

Cheers,

Hugh

16 August 2012, 8:18

That was the first thing I thought of and there had been a recent update to PHP - now running PHP Version 5.3.6-13ubuntu3.8 - on the Mahara server.

However, we set up a second instance of the same exact Mahara code base on this same server as a test (fresh, empty database) and it behaved normally. This, plus the absence of PHP errors in the logs, seemed to indicate that the PHP version was not the issue.

Double checked the database directly to ensure that the auto-create users switch was still set correctly, institution settings normal, Mahara and Moodle keys corresponded.

This is an entirely different server, operating system, network, etc from my other post (re:  failure of 1.5.2 install - to be addressed separately) so that setup would not be relevant. Nevertheless, the config.php file for this server is (less comments, blank lines, names genericized):

<?php
$cfg = new StdClass;
$cfg->dbtype   = 'postgres8';
$cfg->dbhost   = 'localhost';
$cfg->dbport   = null;
$cfg->dbname   = 'dbname';
$cfg->dbuser   = 'dbuser';
$cfg->dbpass   = 'dbpassword';
$cfg->dbprefix = 'pre_';
$cfg->wwwroot = 'http://mahara.school.edu/';
$cfg->dataroot = '/srv/mahara_data';
$cfg->emailcontact = '[email protected]';
$cfg->renamecopies= false;
// closing php tag intentionally omitted to prevent whitespace issues

Can you think of any other database table or entry where corruption or a garble could have prevented the abiltiy of Mahara to auto create new users connecting from Moodle?

Thanks!

20 August 2012, 12:40

Any other ideas?

Thanks!

24 August 2012, 13:55

We have not been able to correct this show-stopping error - would appreciate any help, thoughts, speculations.

A couple of new inputs - we chased up a random error that showed up in the PHP log, indicating that other people had had trouble with the e-mail function due to a backlog of messages in the Mahara admin account. We looked and found we had something over 3400 - deleted these error messages, but no help.

We also periodically get these messages:

[Fri Aug 24 14:31:46 2012] [error] [client ##.##.##.##] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 8192 bytes) in /var/www/lib/user.php on line 564, referer: http://mahara.school.edu/

The script user.php shows the folloing on the relevant line:

 561
 562         return true;
 563     }
 564     throw new EmailException("Couldn't send email to $usertoname with subject $subject. "
 565                         . "Error from phpmailer was: " . $mail->ErrorInfo );
 566 }

Any ideas?

Thanks!

24 August 2012, 14:00

One other thing - if we try to connect via MNET from the Moodle site to Mahara with a Moodle user who has not yet connected to Mahara, the connection is never made - browser just spins.

If we then try to type the URL of the Mahara site in the browser address window (Firefox, IE, whatever), again, the browser connection indicator just spins. It becomes necessary to delete the Mahara server cookie in order to return to the Mahara site.

Might this be some sort of "sessions" problem?

Note that we created a second instance of Mahara on the same server and find that we can connect and create new user accounts normally in that instance from the same Moodle server.

Quite frustrating...

Thanks again.

24 August 2012, 16:38

So taking a step back, I'm thinking there has got to be something in the database that is preventing the auto creation of new users. Here' s the logic:

-  Two different Mahara instances are running on the same server. Both are connected to the same remote Moodle instance. Both are running in the same exact environment: Ubuntu operating system, PHP version, Apache version, Mahara code-base, PostreSQL server are all the same.  Different databases.  The old instance will no longer auto create new users; the new instance will.

-  Both instances of Mahara use XMLRPC to connect to Moodle. Previously existing users from Moodle can connect to both Mahara instances, which seems to validate the MNET connection & settings, keys, etc.

-  New users can connect to the new instance of Mahara from Moodle and have accounts created. New users cannot connect to the old instance of Mahara from Moodle at all.

So by process of elimination, it would seem to be some sort of database glitch in the old instance database that is preventing the new users from Moodle trying to access the old instance of Mahara. So, how to fix?

Autovacuum has been faithfully running in PostgreSQL. Aside from what I've reported above, there are no database error messages (which surprises me!) We've manually vacuumed the old instance database table by table, with no protests from the system.

Would it make sense to try to re-index some tables, run CLUSTER on them? 

Any other ideas?

Thanks...

26 August 2012, 18:40

Hi Worth,

It may be nothing, but the fact that you are getting out of memory errors on smallish tasks could have something to do with it.

It looks like you have a limit of 256M. If you are able to edit your php.ini file and set this temporarily to 512M or even 1024M, restart apache (or php-fpm), and see if that makes a difference.

Check back in if it still doesn't help, and I'll have another think :D

Cheers,

Hugh