28 October 2009, 12:43

Hello all!

I am working on implementaton of "walled garden" concept in our Mahara deployment, which will be used as basement for controlling interation between members of different insitutions. Below I outline the brief specification of "walled garden" concept. Any comments and suggestions would be highly appreciated.
 
Implementation
 
Whether institution is in "walled garden" or not is controlled by additional setting in "Administer Institution" interface. If "Walled Garden" concept is implemented, users are allowed to communicate within instituion only, unless trust realtionships with other institution has been set. Trust relationships are controlled by "Institution Administrator" and "Site Administrator". On the the "Institution Administrator" level trust relaitionships are controlled on request/approve basis: instiution administrator is able to list existent institutions and request trust relationships from each of them, once request is sent and approved by aimed institution's admin, resiprocal trust relationship between institutions is established. On the the "Site Administrator" level  relaitionships are controlled through "Institution Trust Relationsips" interface that allows administrator to maintain resiprocal trust relationships between any institutions directly.
 
Therefore, members of "Walled Garden" institution are able to communicate with members of their institution and members of institution with which trust relationships were set (by Institutional or Site Administrator). By default institutions have "Walled Garden" set to "No", i.e. members of the particular institution can interact to any users in the system except members of "Walled garden" institutions with no interinstitutional trust relationships. To illustrate what is said above, say member of institution is finding friends using Groups->Find friends menu. Based on "Walled garden" option setting for the institution, the following users will be listed:
 
Members of "Walled garden" institution (walledgarden = 1) are able to find:
    * users in all institutions with trust relationship
    * users in own institution
Members of "No walled garden"(default) institution (walledgarden  = 0) are able to find:
    * users in all default institutions (which includes own institution)
    * users in all institutions with trust relationships
Members of 'no institution' are able to find:
    * users in all default institutions (which includes own institution)
 
Technical Details
 
"Walled garden" implemented as an option in "Administer Institution" interface which changes the corresponding flag in  'institution' table. Insitution trust relationship establishing mechanism for "Insitution Admin" level could be pretty similar to friend relationship establishing method: administrator can list and search institutions, then she can send trust request, which is refelcted in 'institution_trust_request' table, after approval the pair of trusted institutions is added to 'institution_trust' mapping table. On "Site Admin" level, however, pairs of trusted instituions are added directly to 'institution_trust'. "Institution Trust Relationsips" interface could be similar to "Institution Staff/Member/Admin" form where first controlled insitution is choosen from dropdown menu, then trusted institutions are added or removed using pair of boxes, one if which is filled with awailable institions, another with institutions that controlled institution already trusts.
 
I have tried to implement "Walled garden" in friend searching (search_user) method, the following adding to the searching SQL code seems to do its job:
 
 $wgsql = 'LEFT OUTER JOIN {usr_institution} member ON u.id = member.usr
                 WHERE (member.institution IN (
                    SELECT DISTINCT i.name FROM {institution} i, {institution_trust} itr
                    WHERE (i.name = itr.inst1 AND itr.inst2 IN (\'' . join('\',\'', $data['institutions']) . '\'))
                        OR (i.name = itr.inst2 and itr.inst1 IN (\'' . join('\',\'', $data['institutions']) . '\'))
                        OR i.name IN (\'' . join('\',\'', $data['institutions']) . '\')';
        if (!$USER->get('walledgarden')) {
            $wgsql .= "OR i.walledgarden = 0)
                       OR member.institution IS NULL)";
        }else {
            $wgsql .= "))
                      ";
        }
 
where $USER->get('walledgarden') initiated at user login and contains 0 if at least one institution user in is not in walled garden or if user is not a member of any institution (otherwice 1), and $data['institutions'] is an array containing the list of intitutions user in.
 
'institution_trust' mapping table structure:
 Column |          Type          | Modifiers
--------+------------------------+-----------
 inst1  | character varying(255) | not null
 inst2  | character varying(255) | not null
 
Other issues
 
We should think what will happen if some interation between users occured (e.g. they became friends, published wall and forum posts) while trust relationsips between institutions were in place, and then one or both institution adminst decided to break trust relationship between institutions. In my opinion, thouse who already friends, should be left as thouse with ability to use all tools in friend's user area (wall, blog comments, etc.)

I am pretty sure that more issues will come out when we will be closer to applying "walled garden" to various intraction tools, so fare I mention only the general one.

29 October 2009, 17:21

Hi Ruslan - thanks for posting this, and sorry for the delay in replying.

This sounds good, in principle. I like how institutional admins can set up walled gardens by request while site admins can just do it.

Technically, the proposal sounds workable. Note that you might have to add checks all over the place for users being in the right walled garden, and you might want to put some thought into how to make this easy for future coders. E.g. the user profile page (user/view.php) will have to respect this permission, as will the view access page (can't see users in search results outside of walled garden) and of course the view access check itself, possibly.

You're right about bringing up the possibility of problems when walled gardens are "torn down". This might be done for a reason, and I would almost suggest that if a walled garden is removed, all inter-institution friendships are removed from the system too. This is not only more like what the admins would expect to happen, it's also easier to implement.

You might want to have a quick look into the performance of any query adjustments or new methods you add, just to see how you think having walled gardens will affect the performance of things like user search. It's not a biggie, but it's worth looking into.

I'd like to see some mockups of the UIs you're planning - from reading what you've described, I can think of at least one more UI for trust relationships for admins/ a grid with institutions along the top and down the side, and checkboxes if they have a trust relationship. Not saying that's a better UI, just that it's worth exploring the possibilities 

Hope that's enough feedback to get you rolling  

30 October 2009, 6:42

Hi Nigel,

Thanks for the feedback! 

>You're right about bringing up the possibility of problems when walled gardens are "torn
>down". This might be done for a reason, and I would almost suggest that if a walled
>garden is removed, all inter-institution friendships are removed from the system too.
>This is not only more like what the admins would expect to happen, it's also easier
>to implement.

I was, in fact, thinking not to break up trust relationships, because trsut relashinships are reciprocal. If the walled garden for the institution torns down, members of  this institution will be able to communicate with members of "no walled garden" institutions, members of "no institution" as well as thouse institutions with whom trust relatinships are establised (while "walled garden" institutions are able to communicate with institutions in trust list only). Trust control will be awailable for institutional admin irrespective to the fact whether they are in walled garden. In other words, admin of walled garden institution  can request tfriendship from non walled garden institution, as well as another way around.

Regarding UI,  so far I am working on UI for Institutional Admin which uses the similar logic like one implemented in user/myfriends.php and user/find.php with only difference that any of institutional admins (if more than one) get email notifications and can approve or decline requests. For site admin, it will probaby be two-step way, first it will be needed to choose insititution to administer, second - choose a number of institutions to match with choosen institution. Not sure, but it might be implemented as a part of Administer Insititution interface (admin/user/institutions.php) below the "locked fields". I will think a while and come up with ideas later.

Meanwhile, another upcoming concern is that we will have about 700 insititutions in our deployment of Mahara, so Administer Insititution interface (list of institutuons) will, probabaly, require pagination and searching, and current interface for managing Admins/Staff/Members with dropdown list of institutions will not be easy to use.

01 November 2009, 17:15

Regarding the friendships - I thought the point of walled gardens was that people in one couldn't see those people outside it? To my mind, that would include friendships too. Here's a use case:

Worthing Girl's school starts on a nationally hosted system, in their own walled garden. So they can all only see each other.

Then they strike up an agreement with Brighton Girl's school, to be part of the same walled garden. Friendships start being made.

Then, at the annual football match, a Brighton girl does a horrible tackle that brings the entire school into disrepute. Worthing won't have anything to do with Brighton any more. So the Worthing admin dissolves the relationship.

At this point, you would expect all friendships to end between the schools, yes?

Or were you just talking about institution admins? To my mind, even their friendships would end.

The UI - yeah, might be best on a different screen. Would be good to see some screenshots so we can get a feel for how it's looking! 

Yeah, we have 150 institutions on MyPortfolio, and it's quite a long list. Was even worse when that list wasn't sorted! Any ideas you have for better add/remove interfaces are welcome, I think we copied that idea from Moodle inititally, and haven't really felt so much pain to have to work on those screens since. 

01 November 2009, 19:54

Hi Nigel,

>I thought the point of walled gardens was that people in one couldn't see those people outside it?

Right, but if trust relathipships (or friendship) between institutions are established, people in one walled garden will be able to see everyone in trusted institutions as well.

>Then they strike up an agreement with Brighton Girl's school, to be part of the same walled garden.

No, they will only establish a trust (or friendship) relaships between institutions, thus members of walled Worthing Girl's school will be able to interract with members of Brighton Girl's school (e.g. make friends), irrespective to Brighton Girl's school walled garden settings. Walled garden is applicable to single institution only, it is a sort of "institution communication mode": if it is set - people can communicate between each other and between people in trusted institutions only, if not, people can communicate to anyone in similar (non walled) institutions plus to people in walled institutions with trust relashinships (but not to walled institutions without trust relashinships). Please, look again at the example with user search in my first post at this thread which explains these relathinships.

>So the Worthing admin dissolves the relationship.

I guess, at this point those who are already friends, should be left as those. However, members of Worthing Girl's school will not be able to find friends in Brighton Girl's school any more (i.e. they no longer see each other once the trust relathinships are broken).

Will publish some screenshorts as soon as they are ready, hopefully this week.

01 November 2009, 20:33

I think we're in agreement, I think I was just confused about "institution friends" 

The only difference between what you and I are thinking is what happens when the two institutions dissolve the relationship. I'm betting that the administrators will want to dissolve the friendships too. Maybe it can be an option when the relationship is broken - break friendships between individuals too? Yes/No

Of course if you don't, then you can't do the simple look-at-the-walled-garden-settings approach to seeing whether one user can see another in search results etc, you have to take friendship status into account too. But I guess that's not much harder. 

02 November 2009, 15:09

>Maybe it can be an option when the relationship is broken - break friendships between individuals too? Yes/No

This is a good idea! Will implement it as an option on the "Break trust relationships" screen.

19 November 2009, 12:54

As has been discussed on the forum, the Walled Garden concept relies on a trust relationships control mechanism, which is implemented at both Site Admin and Intuition Admin levels. The main difference between these two levels is that the site admin can change trust relationships explicitly, while the Institution admin needs to negotiate it with Admin(s) of target institution. Trust relationships control requires two tables for its functioning (see lib/db/upgrade.php for details):

institution_trust

Column |            Type             | Modifiers
--------+-----------------------------+-----------
 inst1  | character varying(255)      | not null
 inst2  | character varying(255)      | not null
 ctime  | timestamp without time zone | not null
 
institution_trust_request

  Column   |            Type             | Modifiers
-----------+-----------------------------+-----------
 owner     | character varying(255)      | not null
 requester | character varying(255)      | not null
 ctime     | timestamp without time zone | not null
 message   | text                        |

"institution_trust" is a mapping table which is used for storing current trust relationships between institutions.  "institution_trust_request" is used for registering  trust requests initiated by the "requester" institution and addressed to the "owner" institution. The "message" field stores a message if one has been specified at the time of trust request had been sent.

Below I have provided a description of each level with reference to particular files and functions that can be found in the committed patches.

Walled Garden Trust Relationships Control on institution admin level.

For institution admin level Walled Garden Trust Relationships are controlled using two additional menu items (defined in lib/web.php): "institutions we trust" (admin/users/institutiontrust.php) and "Find Institution" (admin/users/findinstitutions.php).  Both of these pages allow the institutional admin to choose the institutions to act on behalf of, which is relevant when the user can administer more than one institution. The "Find Institution" page shows all available institutions, and provides searching and pagination features for convenience. The "Institutions we trust" page shows only institutions with which trust relationships are already established, and allows the admin to filter the list based on trust relationships status (pending or current). The list of institutions itself shows the name of institution and a number of actions/notes at the right hand side based on the status of institution which is outlined in the table below:

Institution status	Elements shown
general institution	“Send Trust Request!” link
current trustee	“is exiting trustee” note near the institution name, “break trust relationships” link
institution to which trust proposal has been sent	“Trust relationships requested!” note
institution that is pending trust approval	“pending” note near the institution name, the message sent by proposal (if any) underneath the institution name, “Approve request” button, “Deny Request” link

By clicking on the institution name, the View Institution page (admin/users/viewinstitution.php) is opened where the user can see the actions they can perform with chosen institution. If the chosen institution is one the user can administer, the interface for explicit trust relationships control between "own" institutions is shown (similar to one that is used by site administrator but the number of institutions is limited to those each user is administrator of (see Trust Relationships Control on site admin level section below for more details).

Each of the actions named above has corresponding page or function which performs the action:

Action	Page or function	Desciption
Sending trust request	admin/users/requesttrust.php	Takes target institution name, own institution name and the page to return as parameters. Provides interface for sending trust request with optional message. On submitting it inserts the record in “institution_trust_request” table and sends message to all admins of target institution (if institution does not have admins, the message is sent to site admins with a note that institution does not have admins). Besides 'addtrustrequest' event is fired.
Denying trust request	admin/users/denytrustrequest.php	Takes target institution name (which proposal came from), own institution name and the page to return as parameters. Provides interface for denying trust request with optional message. On submitting it deletes the corresponding record in “institution_trust_request” table and sends message to all admins of target institution (if institution does not have admins, the message is sent to site admins with a note that institution does not have admins). In addition 'removetrustrequest' event is fired.
Approving trust request	accepttrust_submit(Pieform $form, $values) in lib/institution.php	Initiated when “Approve request” button is clicked. It deletes the corresponding record in “institution_trust_request” table and inserts the record in “institution_trust“ table. Also it send message to all admins of institution from which proposal came (if institution does not have admins, the message is sent to site admins with a note that institution does not have admins). In addition'addtrust' event is fired.
Breaking trust relationships	admin/users/breaktrust.php	Takes target institution name, own institution name and the page to return as parameters. Provides interface for breaking trust request with optional message. On submitting it deletes the corresponding record in “institution_trust” table and sends message to all admins of target institution (if institution does not have admins, the message is sent to site admins with a note that institution does not have admins). In addition, 'removetrust' event is fired. Nigel's suggestion - the option to break friendship, will be added at the later stage)

Relevant patches (6):

commit 444a63e51eb51eb4858caf38522a25e2107e057f

Check for attempt to manage trust relationships with 'mahara'
institution on site and institution admin levels. Changes in Institutions page
output.

commit 2b0219c7bdb4f8f704dca2dc7698904ba09d682e

db upgrade code to add walled garden tables

commit 2346ff323d1a26c7692c1528aa4359801f6c0cb8

Changes in Inst. admin trust control, links to site admin trust
control interface

commit b5f7f9663fc96547edf0a5bef1472731175909a8

Walled garden intitutional admin control - minor changes in urls and
sorting

commit 8ffbf22987bc575819fc0f50c61355e2cd1832d1

Notification about pending trust requests on sideblock

commit c70ddd3ef7351c0843d36d0ba3e01840f65ad5ac

Walled Garden Trust Relationships Control on institution admin level. This commit includes files and methods for searching institutions, viewing them and performing different actions (i.e. sending trust request, denying trust request, approving trust request, breaking existing trust relationships).

Walled Garden Trust Relationships Control on site admin level.

For the site admin, Walled Garden Trust Relationships are controlled explicitly. A searching block and pagination has been added to the institutions page (admin/users/institutions.php) in order to facilitate institution management. The "Trustees" column shows the number of current trustees and it is also the link to View Institution page (admin/users/viewinstitution.php) for controlling trust relationships between corresponding institution and any other.  As was mentioned above, the View Institution page shows elements based on user type and institution administering permissions. For site admin it shows interface for explicit trust relationships control by default (institutions mode), and also it can show interface for responding to trust proposals addressed to chosen institution (requesters mode). The  desired interface is controlled by mode dropdown menu. Both interfaces are based on the `institutionlist' pieform element (lib/form/elements/institutionlist.php).  `institutionlist' element utilizes institution search page (admin/users/institutionsearch.json.php), which is called in the background using AJAX and returns requested data in json format which is reflected in appropriate boxes in the form. Once the changes are submitted, institutiontrust_submit(Pieform $form, $values) function (admin/users/viewinstitution.php) determines what changes have to be done based on the mode. For `institution' mode  it compares differences between current trustees and submitted changes so that it first determines which institutions have been removed and calls `removeTrustees' method (lib/institution.php, Institution class) to perform operation, then it determines which institutions have been added  and  calls `addTrustees' method (lib/institution.php, Institution class). For `requesters' mode, it either creates trust relationships (by calling `addTrustees' method in lib/institution.php) or decline trust requests (by calling `denyTrustees' method in lib/institution.php).  Note, that institution admin can only operate `institutionlist' element on View Institution page in `institution' mode (for maintain trust relationships between institutions one can administer).  The methods mentioned above perform the following actions:

Method	parameters	Description
addTrustees	array of institutions to perform action with, current institution, action	The method is used for adding new trustees to current institution. It first removes corresponding records from ‘institution_trust_request’ table and then inserts the records in “institution_trust“ table. Finally it calls ‘notifyTrustees’ for admins notification. ‘action’ parameter is set to ‘add’ by default and passed to ‘notifyTrustees’.
removeTrustees	array of institutions to perform action with, current institution	The method is used for removing trustees from current institution by deleting the corresponding records in “institution_trust“ table. At the end љit calls ‘notifyTrustees’ for admins notification.
denyTrustees	array of institutions to perform action with, current institution	The method is used for removing trustees that requested trust relationships from current institution by deleting the corresponding records in “institution_trust_request“ table. At the end љit calls ‘notifyTrustees’ for admins notification.

All three methods call `notifyTrustees' method  (lib/institution.php, Institution class) on completion to notify admins. Depending on the action parameter it chooses language notations related to particular action and fires appropriate event at the end.  First, individual emails are being sent to institution admin(s) of affected institutions (or to the site admin if the target institution does not have institution admins). If the admin user who is making the change is also the admin for one of target institutions, which is the case if the institution admin is changing trust relationships between own institutions, then email will not be sent. In this case an email with a list of changes is sent to institutional admin(s) of current institution (i.e. the one which have been chosen to manage trust relationships with).

Relevant patches (3):

commit 553a7564f5e25c98e8b7f7437763ea428d02a15f

Walled Garden view institution minor changes with styles and javascript

commit bf1805012c2503c62ae2681185410736f40f71d4

Walled Garden Trust Relationships Control on site admin level. This commit includes institutionlist form element, search page that returns json data, changes to viewinstitution page and corresponding methods for direct change of relationships or responding to trust proposals with notification of all relevant institution admins.

commit d5a98bfe5bf98e928073a38827d052101aeac69d

Institution search is added to institutions.php

Walled Garden option and implementation of Walled Garden concept to User Search.

This part has been discussed in Walled Garden technical specification
proposal (see the first post at this thread).

Relevant patches (3):

commit 869ace73a146fc029e1ed5afca19f98626559b68

Walledgarden variable in USER object. It is set to 1 if all insitutions the user is in are set to walledgarden, otherwise it is set to 0.

commit 41805aaeffe7dacd853356e8ae435fc25593bf85

Walled garden option in the institution editing page

commit 55f860568e6fa6482eb87b534fd94896a0e76598

User search based on walled garden option and trust relationships between institutions

Any comments and feedback are very welcome.

PS: Many thanks to Dan Poltawski for revision this description before it has been published here.

19 November 2009, 14:46

Hi Ruslan,

I have just had a very quick look, without trying to get the source or apply the patch or anything, and I have two immediate comments:

 First, would it be possible to generate a patch containing the complete delta of the changes?  That would be great in addition to the patchset that you've provided - I often find that having both for a large change like this is very helpful.

Second, the only thing that immediately leaps to mind is the use of sql in "script" pages - I know there's quite a bit of this already in Mahara (and I am certainly to blame for a lot of it!), but I don't think it's something that we should really be adding to if we can help it.

Quick lookups are less problematic, but anything that involves anything more than a single get_record or get_field call should be abstracted to a method somewhere, imo ;)

We also have a db abstraction function for ilike - db_ilike.

I haven't looked throroughly yet though, I hope someone else will have the time and have some feedback for you as well ;) thanks heaps for contributing, and the writeup - it looks as though you've copied the "style" of mahara very well :)