21 June 2019, 22:18

Hi,

We have an issue with Mahara since it was upgraded from 18.04.3 to 18.04.4.

The search has stopped working. The site is using ElasticSearch.

I have tried resetting the index and following all the steps in the following discussion :https://mahara.org/interaction/forum/topic.php?id=8226#post32981
but the issue is not resolved. The elastic search indices are green and all seems fine with them.
I don't know what else to do.

Every time a search item is entered into the search box, the following error is displayed 

on the page:

"A nonrecoverable error occurred. This probably means you have encountered a bug in the system"

and the following appears in the error logs:

[:error] [pid 38984] [client 86.47.50.67:45225] [WAR] 06 (lib/errors.php:536) [Elasticsearch\\Common\\
Exceptions\\BadRequest400Exception]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on tex
t fields by default. Set fielddata=true on [mainfacetterm] in order to load fielddata in memory by uninverting the inverted index. Note 
that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","r
eason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"dcumahara.enovation.ie","node":"5lcETj9fT
nWLxojAe0pB9A","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=tr
ue on [mainfacetterm] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant
 memory. Alternatively use a keyword field instead."}}]},"status":400}, referer: https://dcumahara.enovation.ie/search/elasticsearch/ind
ex.php   

Elastic search version is:

curl -XGET 'localhost:9200' 
{                                                                                                                                       
  "name" : "5lcETj9",                                                                                                                   
  "cluster_name" : "elasticsearch",                                                                                                     
  "cluster_uuid" : "_dMInsIyRkSO0RSXxRDWUQ",                                                                                            
  "version" : {                                                                                                                         
    "number" : "5.6.14",                                                                                                                
    "build_hash" : "f310fe9",                                                                                                           
    "build_date" : "2018-12-05T21:20:16.416Z",                                                                                          
    "build_snapshot" : false,                                                                                                           
    "lucene_version" : "6.6.1"                                                                                                          
  },                                                                                                                                    
  "tagline" : "You Know, for Search"                                                                                                    
}

24 June 2019, 11:59

Hi Robert S,

I'm unable to replicate the error you are having.

Looking at the error 'Fielddata is disabled on text fields by default. Set fielddata=true on [mainfacetterm]'

In the htdocs/search/elasticsearch/type/ directory the 'mainfacetterm' option in the files should be like

            'mainfacetterm' => array (
                    'type' => 'keyword',
            ),

Are there any set to 'text'?

Cheers

Robert L

24 June 2019, 21:11

Hi Robert L,

Thanks for replying. In the htdocs/search/elasticsearch/type/ directory, there are none set to

text in any of the files in the mapping arrays, however in the following files I do see a static variable set to "Text":

search/elasticsearch/type/ElasticsearchType_interaction_forum_post.php
public static $mainfacetterm = 'Text';

search/elasticsearch/type/ElasticsearchType_block_instance.php
public static $mainfacetterm = 'Text';

search/elasticsearch/type/ElasticsearchType_artefact.php
public static $mainfacetterm = 'Text'; // can be Text or Media depending on artefacttype

search/elasticsearch/type/ElasticsearchType_interaction_instance.php
public static $mainfacetterm = 'Text';

Best Regards,

Robert

25 June 2019, 9:20

Hi Robert

It is the bit within the array: public static $mappingconfv6

that I am thinking about - there the mainfacetterm should be                     'type' => 'keyword',

Cheers

Robert

25 June 2019, 20:28

Hi Robert,

Yes they are all keyword within the public static $mappingconfv6 = array in 

all the files. I found a more detailed error message while debugging:

[xdebug_message] => ( ! ) Elasticsearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [mainfacetterm] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"reflect.dcu.ie","node":"vfAmsA7fQPq-zk6tb7-2Lg","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [mainfacetterm] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}]},"status":400} in /opt/vhosts/securesandbox.enovation.lan/www/lib/elasticsearch/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php on line 610
Call Stack
#TimeMemoryFunctionLocation
10.0001364904{main}(  ).../index.php:0
20.01451611408PluginSearchElasticsearch::search_all(  ).../index.php:62
30.01451611408ElasticsearchPseudotype_all::search(  ).../lib.php:1072
40.12922807392Elasticsearch\Client->search(  ).../lib.php:1816
50.12942811536Elasticsearch\Client->performRequest(  ).../Client.php:916
60.12942811584Elasticsearch\Transport->performRequest(  ).../Client.php:1506
70.12942811584Elasticsearch\Connections\Connection->performRequest(  ).../Transport.php:106
80.13042923304Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(  ).../Connection.php:171
90.16062938592GuzzleHttp\Ring\Core::proxy(  ).../Connection.php:294
100.16082942752GuzzleHttp\Ring\Future\CompletedFutureValue->then(  ).../Core.php:341
110.16103009688React\Promise\FulfilledPromise->then(  ).../CompletedFutureValue.php:55
120.16103009688Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(  ).../FulfilledPromise.php:25
130.16103011336Elasticsearch\Connections\Connection->process4xxError(  ).../Connection.php:273

Best Regards, Robert

26 June 2019, 7:44

Hi Robert

I'm not able to tell from that error stack what your problem is.

Are you able to edit the file search/elasticsearch/lib.php

and just before the line 1072:

    return ElasticsearchPseudotype_all::search($query_string, $limit, $offset, $options, $mainfacetterm, $USER);

add some debugging lines:

    log_debug($query_string);

    log_debug($options);

    log_debug($mainfacetterm);

and add that output here

Cheers

Robert

28 August 2019, 4:19

Hi Robert,

Thanks for your help. The reason for this issue was the maxClauseCount in ElasticSearch.

"caused_by" : {
"type" : "too_many_clauses",
"reason" : "maxClauseCount is set to 1024"

The number of clauses returned in the search in Mahara which was causing the issue was 5401. 

So the following configuration change was required in ElasticSearch:

Add following configuration:
indices.query.bool.max_clause_count: 10000
to:
/etc/elasticsearch/elasticsearch.yml

And then the issue was resolved.

Best Regards,

Robert