13 November 2013, 22:27

Really annoyed that I'm going to have to unsubscribe from this forum, but I seem to be bombarded with spam through the email digest.

I don't know if the spam is cleared from the forums themselves, but it is coming through the mail out cron in buckets.

Shame that it is spoiling the community

14 November 2013, 0:55

Hi Roger,

How is it going at Solent?

Sorry your are bombarded with spams. Sorry we are all bombarded. I discussed about this issue with Kristina months ago and it appears that this kind of spam is difficult to impossible to track before it happens. People are paid to register blogs, forums or web sites such as Mahara and post their s..t on it.

What we try to do on Mahara is that the first who has the admin right and sees a spam, he or she will destroy it and block its author. But it means that the message would already have been sent to all participants.

It would maybe interesting to see how/why moodles' forums are not or less poluted by spams that the ones on Mahara.

Sorry about that.

D. JAN

14 November 2013, 1:39

Well I found that I was getting so much that I ended up with a mail rule stuffing it all into a folder that never got read and therefore missing interesting conversations, so it seemed sensible to unsubscribe.

I wonder if it is just the nature of the free sign up without human validation that allows anyone to post - or if moodle has some good filtering?

14 November 2013, 11:43

By coincidence we were just discussing this yesterday around the office. I even opened a bug on Launchpad about it: https://bugs.launchpad.net/mahara/+bug/1250641

Currently the mahara.org forums have Mahara's anti-spam settings set to "advanced", which means that it filters out forum posts that contain blacklisted URLs from the SURBL and Spamhaus lists, and that there are measures in the registration screen to make automated user registration more difficult... although really, those measures only stop the most basic of automated scripts and can be bypassed by a slightly more targeted script.

There are some other good ideas on this page: http://docs.moodle.org/25/en/Reducing_spam_in_Moodle

The main difference I'm aware of with moodle.org is that they use a Captcha for new user registration. We have discussed putting a Captcha in place on the user registration page, but Kristina's ardently against it because of accessibility concerns, and we suspect the spam may be posted by human agents anyway. Although perhaps it would be interesting to put a Captcha on there and see whether it eliminates the spam messages.

If the spam is being posted by human beings, then I think the only options would be to add a Bayesian spam filter and/or a moderation queue.

Cheers,

Aaron

14 November 2013, 13:40

Is there any scope for using the community to filter new users' messages - 

eg:

For new users don't send their post out to everyone at once - send it to 1% of users every minute with a "this is spam" link for ppl to click if it's spam. Above a certain response rate, the sendout is suspended until marked as spam, or ham by an admin.

Or send it to online/active ppl first,

or ...

14 November 2013, 19:12

Hi Peter,

Fancy seeing you here! Thanks for the suggestion. I don't know that we have the necessary round-the-clock/globe coverage of community involvement for that scheme to actually reduce the amount of spam that winds up in inboxes, though. Other than the News forum (which is currently broken and doesn't send out emails at all) the next highest-subscribed forum only has about 400 subscribers.

But I think it could work if we instead used a Bayesian filter (like spamassassin) to put things into the moderation queue.

Cheers,

Aaron

14 November 2013, 14:56

Hi,

We used to have Captcha in Mahara, but eliminated it for accessibility reasons some versions ago. Many people don't like Captchas because they are often very difficult to read. Instead, we introduced advanced settings to avoid bots generating spam messages by mixing up the signup fields and having a honeypot. However, that doesn't stop human spammers.

A Captcha would not stop human spammers either as companies outsource the solving of those. See for example http://www.npr.org/templates/story/story.php?storyId=130594039 So I think it's pretty much a dead end if it can be circumvented so easily.

Unfortunately, there aren't any good spam prevention systems out there. Implementing a moderation queue could catch a number of spammers as could probably bumping up filtering options as well as making sure that they work in the forums. But I've also seen that people create pages for advertising things on mahara.org that we don't yet catch easily. They look like regular pages.

Cheers

Kristina

14 November 2013, 19:09

The issue of spammers creating spam pages on mahara.org is exactly the same as spammers creating spam profile pages on Moodle, described here: http://docs.moodle.org/25/en/Reducing_spam_in_Moodle

They do it only because the pages are publicly visible. Then the pages can get spidered by search engines, and/or they can link to them in other spam messages elsewhere.

Moodle's solution was to add a setting to prevent the creation of public-viewable pages. Potentially we could do something similar on mahara.org, although we'd want to be able to grant that permission to trusted users. Another possibility would be the wiki approach, which is to have people who monitor the "recent changes" list (in our case, "Latest Pages") and check to see if each new page is spam or not.

Cheers,

Aaron

14 November 2013, 23:17

A forum I use to keep myy wonky old van on the road has a new member policy that prevents posting of images/for sales and certain other things until they reach 10 normal posts where other users have interacted.

I wonder if something could be implimented such that new members can only post to a new memebr/introduction area and their first 10 posts don't get mailed out (or similar).
A kind of holding pen until trust is established. Maybe just 2 posts would be enough -to say hello and then get a response from the community welcome. Something like this would allow human confirmation of new users before they allowed to do more without too much labourious admin?

The forum I use is phpBB based - www.mdocuk.co.uk/forums

15 November 2013, 11:54

Well, I've gone ahead and implemented reCAPTCHA support and deployed it to the mahara.org registration page. We'll see whether that has any impact. I'll also try to upstream this feature for Mahara 1.9.0.

On the related topic of spam pages, while doing that implementation I noticed that Mahara actually does already have a setting to disable public pages! But as mentioned elsewhere, a blanket ban on public pages is probably not appropriate for mahara.org, since we have a lot of legitimate users using mahara.org pages to share Mahara-related information, host Mahara plugins, etc.

Cheers,

Aaron