Forums | Mahara Community

Support / Elastic Search Issues after upgrading to 18.04.4

Posts: 7

21 June 2019, 10:18 PM

Hi,

We have an issue with Mahara since it was upgraded from 18.04.3 to 18.04.4.

The search has stopped working. The site is using ElasticSearch.

I have tried resetting the index and following all the steps in the following discussion :https://mahara.org/interaction/forum/topic.php?id=8226#post32981
but the issue is not resolved. The elastic search indices are green and all seems fine with them.
I don't know what else to do.

Every time a search item is entered into the search box, the following error is displayed

on the page:

"A nonrecoverable error occurred. This probably means you have encountered a bug in the system"

and the following appears in the error logs:

[:error] [pid 38984] [client 86.47.50.67:45225] [WAR] 06 (lib/errors.php:536) [Elasticsearch\\Common\\
Exceptions\\BadRequest400Exception]: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on tex
t fields by default. Set fielddata=true on [mainfacetterm] in order to load fielddata in memory by uninverting the inverted index. Note
that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","r
eason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"dcumahara.enovation.ie","node":"5lcETj9fT
nWLxojAe0pB9A","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=tr
ue on [mainfacetterm] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant
memory. Alternatively use a keyword field instead."}}]},"status":400}, referer: https://dcumahara.enovation.ie/search/elasticsearch/ind
ex.php   

Elastic search version is:

curl -XGET 'localhost:9200'
{
"name" : "5lcETj9",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "_dMInsIyRkSO0RSXxRDWUQ",
"version" : {
"number" : "5.6.14",
"build_hash" : "f310fe9",
"build_date" : "2018-12-05T21:20:16.416Z",
"build_snapshot" : false,
"lucene_version" : "6.6.1"
},
"tagline" : "You Know, for Search"
}

Posts: 517
Re: Elastic Search Issues after upgrading to 18.04.4

24 June 2019, 11:59 AM

Hi Robert S,

I'm unable to replicate the error you are having.

Looking at the error 'Fielddata is disabled on text fields by default. Set fielddata=true on [mainfacetterm]'

In the htdocs/search/elasticsearch/type/ directory the 'mainfacetterm' option in the files should be like

'mainfacetterm' => array (
'type' => 'keyword',
),

Are there any set to 'text'?

Cheers

Robert L

Posts: 7
Re: Elastic Search Issues after upgrading to 18.04.4

24 June 2019, 9:11 PM

Hi Robert L,

Thanks for replying. In the htdocs/search/elasticsearch/type/ directory, there are none set to

text in any of the files in the mapping arrays, however in the following files I do see a static variable set to "Text":

search/elasticsearch/type/ElasticsearchType_interaction_forum_post.php
public static $mainfacetterm = 'Text'; search/elasticsearch/type/ElasticsearchType_block_instance.php public static$mainfacetterm = 'Text';

search/elasticsearch/type/ElasticsearchType_artefact.php
public static $mainfacetterm = 'Text'; // can be Text or Media depending on artefacttype search/elasticsearch/type/ElasticsearchType_interaction_instance.php public static$mainfacetterm = 'Text';

Best Regards,

Robert

Posts: 517
Re: Elastic Search Issues after upgrading to 18.04.4

25 June 2019, 9:20 AM

Hi Robert

It is the bit within the array: public static $mappingconfv6 that I am thinking about - there the mainfacetterm should be 'type' => 'keyword', Cheers Robert Posts: 7 Re: Elastic Search Issues after upgrading to 18.04.4 25 June 2019, 8:28 PM Hi Robert, Yes they are all keyword within the public static$mappingconfv6 = array in

all the files. I found a more detailed error message while debugging:

[xdebug_message] => ( ! ) Elasticsearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [mainfacetterm] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"reflect.dcu.ie","node":"vfAmsA7fQPq-zk6tb7-2Lg","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [mainfacetterm] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}]},"status":400} in /opt/vhosts/securesandbox.enovation.lan/www/lib/elasticsearch/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php on line 610
Call Stack
#TimeMemoryFunctionLocation
10.0001364904{main}(  ).../index.php:0
20.01451611408PluginSearchElasticsearch::search_all(  ).../index.php:62
30.01451611408ElasticsearchPseudotype_all::search(  ).../lib.php:1072
40.12922807392Elasticsearch\Client->search(  ).../lib.php:1816
50.12942811536Elasticsearch\Client->performRequest(  ).../Client.php:916
60.12942811584Elasticsearch\Transport->performRequest(  ).../Client.php:1506
70.12942811584Elasticsearch\Connections\Connection->performRequest(  ).../Transport.php:106
80.13042923304Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(  ).../Connection.php:171
90.16062938592GuzzleHttp\Ring\Core::proxy(  ).../Connection.php:294
100.16082942752GuzzleHttp\Ring\Future\CompletedFutureValue->then(  ).../Core.php:341
110.16103009688React\Promise\FulfilledPromise->then(  ).../CompletedFutureValue.php:55
120.16103009688Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(  ).../FulfilledPromise.php:25
130.16103011336Elasticsearch\Connections\Connection->process4xxError(  ).../Connection.php:273


Best Regards, Robert

Posts: 517
Re: Elastic Search Issues after upgrading to 18.04.4

26 June 2019, 7:44 AM

Hi Robert

I'm not able to tell from that error stack what your problem is.

Are you able to edit the file search/elasticsearch/lib.php

and just before the line 1072:

return ElasticsearchPseudotype_all::search($query_string,$limit, $offset,$options, $mainfacetterm,$USER);

log_debug($query_string); log_debug($options);

log_debug(\$mainfacetterm);

Cheers

Robert

Posts: 7
Re: Elastic Search Issues after upgrading to 18.04.4

28 August 2019, 4:19 AM

Hi Robert,

Thanks for your help. The reason for this issue was the maxClauseCount in ElasticSearch.

"caused_by" : {
"type" : "too_many_clauses",
"reason" : "maxClauseCount is set to 1024"

The number of clauses returned in the search in Mahara which was causing the issue was 5401.

So the following configuration change was required in ElasticSearch:

indices.query.bool.max_clause_count: 10000
to:
/etc/elasticsearch/elasticsearch.yml

And then the issue was resolved.

Best Regards,

Robert

7 results