21 September 2009, 5:20

Today we found some of our views had penis enlargement adds splattered all over them.

After looking into this it seems that when you use a secret URL any non logged in user can place feedback, which of course cannot be moderated.

I looked into the source of the spam... it looks like a generic spambot, nothing mahara specific.. good news in one sense... more embarrasing in another....

This needs to be sorted out ASAP, the consequences are quite serious. Secret URL's are primarily used for sharing information externally, with other institutions or potential employers.

To be honest, I think you should disable the entire feedback feature until this issue, and the issues around moderation are resolved.

On that note, can anyone tell me how I could modify Mahara to do this, disable feedback?

Thanks, Jeremy

21 September 2009, 6:42

Hello Again,

A bit more info on this. FIrst, I wondered how the spammer found our views as I had robots.txt. set to dissallow all search engines. 

The problem with that is that Google will still index any pages that are linked from elsewhere, so if a user links to a secret URL from another web page it can be indexed. To prevent this one would need to add "no cache" to in the meta tags of Mahara. That would be a useful admin option.

My guess is that this page was found using a dork on "place feedback" in a popular search engine. I dont think there is anything that can be done with this, unless an image were used to replace the text, with no Alt text etc. The point here is that spammers dork search engines to find target sites, so removing terms like "leave feedback" "have your say" "comments" can reduce unwelcome attention.

The other thing I found in view.php is $anonfeedback

My view is that we should NEVER allow anonfeedback. But, if this feature is wanted by others, then it should be an admin setting disabled by default.

If you are going to allow anon feedback then you really want to add a captcha to that feedback form, and, not the captcha you have now as it looks like an easy crack.

Adding support for re-captcha would be a good idea, it is a lot better than the captcha you are using for contact forms: http://recaptcha.net/

Soo, in summary, there are three things I would suggest:

1. Control $anonfeedback from admin settings ... disable by default in admin settings
   
2. Add feature for no-cache in meta tags to stop search engines indexing pages
3. Add support for re-captcha to all feedback forms (just enter an API key)
   

Regards, Jeremy

 

 

21 September 2009, 9:19

 Sorry, I meant no index, not no cache:

http://www.robotstxt.org/meta.html

I am not sure where I would need to insert that (which file) to stop search engines indexing our site, I would be grateful if someone could point me in the right direction.

Regards anonymous feedback, I just botched line 133 in htdocs/view/view.php as a temporary fix, so now only logged in users can post feedback.

If left un-checked it is only a matter of time before some poor unfortunate gets hotlinked porn images injected into their views
 

30 September 2009, 23:02

Hi Jez, sorry for the extremely slow reply.

noindex would be useful to add on views that weren't shared publicly. We should do that - can you file a bug to remind us? 

We should also have a captcha on it too. And of course, the moderation stuff is all good improvements and thanks to the University of Luxembourg, we'll get to implement those. Though saying that we should kill the ability for anyone to use any feedback right now is a bit of overkill . It has its uses, even if flawed. 

29 October 2010, 6:43

Hello:

Can anyone tell me the status of this "noindex" idea?

I have a user group who are very concerned about having secret URL pages meant to be very private show up in Google searches.

I'll try to find it in bugs, but if anyone knows already it would be much appreciated!

31 October 2010, 20:35

Don, I've just updated your bug report.  We haven't done anything about this yet.

But I'd say that once a secret URL is posted out on the public web, or even emailed to the wrong person, it's not secret anymore regardless of whether or not it has been indexed.  In this case the owner of the view should really delete the old secret url, create a new one, and only send it out to more trustworthy people.

The spam issue should be at least partly addressed in 1.3.  Anonymous feedback can be disabled by the site admin.  But even when it's enabled, there's also an antispam option for forms filled in by logged out users (including feedback on public/secret url views).  The feedback content is checked for urls containing blacklisted spam domains.

In future we'll add this spam checking to content submitted by logged in users too, including forum posts on mahara.org.

01 November 2010, 6:00

Hi Richard

Thanks for the prompt response. I'm still a bit concerned about the secret URL issue from the pespective of indexing rather than spam. Nothing to do with posting the secret link on the public web or sending to the wrong person.

My understanding of secret URLs in Mahara is that they are public pages that are hard to find, because the pointer is a coded number rather than a logical URL that contains the view number. But....the function of Google and other search engines is to make web pages easy to find.

If a user creates a view and sends someone a secret URL, then that view is open for Google (or another robot) to index it and provide a link to it as a search result (likely using the underlying logical URL with the embedded view number).

Granted, it's good practice to limit accessibility with start and stop dates. That's a great feature. But will that stop the view from being cached by the search engine and therefore being available to read anyway?

I'm happy to be convinced otherwise. But if this issue is real, it will limit the deployment of Mahara beyond schools and pre-employment programs, since employed people will be afraid to use it, for fear of being busted for job searching.

Since this is not really about spam, perhaps it should be a new bug. I'm not sure.

Thanks!

01 November 2010, 14:35

Hello Don,

I think that your concerns are very important. Not just for employment, but also if a child shares a view via a secret URL so that his parents can view it who don't have a login.

Would it be a possibility to use the noindex and nofollow meta tags (cf. http://www.robotstxt.org/meta.html )? Google has info at http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=93710

Cheers

Kristina

01 November 2010, 16:02

Hi Kristina:

That's what I'm suggesting, but my understanding is that would need to be added automatically when secret URL access is added, implying a change to the code.

At least that's what I took from previous messages in this thread, by Nigel McNie and others.

And you're right, that's another good example use case!

01 November 2010, 17:48

Don, I don't want you to think I don't agree with Nigel's suggestion.   I do agree with it, as I mentioned in the bug report.  But as Nigel implies there, it's not really about the secret url: noindex should be added on any view that doesn't have public access (for example, a view with both public and secret url access *should* be indexable).

But regardless of this, view owners need to remember that it's up to them to keep their secret urls secret.  Respectable crawlers obey robots.txt and noindex, but dodgy ones might still index these pages.  So while adding the noindex tag is an extra defence against people who might publish a secret url, it's not a guarantee of privacy.