Invalid shift value (xx) in prefixCoded bytes (is encoded value really a geo point?) #17537
Comments
|
@jaksmid could you provide some documents and the stack trace that is produced when you see this exception please? |
clintongormley
added
>bug
:Analytics/Geo
|
I can reproduce something that looks just like this with a lucene test if you apply the patch on https://issues.apache.org/jira/browse/LUCENE-7185 I suspect it may happen with extreme values such as latitude = 90 or longitude = 180 which are used much more in tests with the patch. See seed: |
|
Hi @clintongormley, thank you for your message. The stack trace is as follows: The field cyberdyne_metadata.ner.mitie.model.DISEASE.tag should not be a geopoint according to the dynamic template. |
|
@rmuir oh, good catch |
+1 to fail early if the user explicitly defined a non text field to highlight on and exclude non text fields when using wildcards |
|
I was running into this bug during a live demo... Yes I know, I've should have tested all demo scenario's after updating ES 😬 . Anyway, +1 for fixing this! |
|
-I´m having the same error. It's happends with doc having location and trying to use thanks! |
|
I'm unclear as to what exactly is going on here, but I'm running into the same issue. I'm attempting to do a geo bounding box in Kibana while viewing the results in the Discover tab. Disabling highlighting in Kibana fixes the issue, but I would actually like to keep highlighting enabled, since it's super useful otherwise. It sounds from what others are saying that this should fail when querying on any non-string field, but I am not getting the same failure on numeric fields. Is it just an issue with geoip fields? I suppose another nice thing would be to explicitly allow for configuration of which fields should be highlighted in Kibana. |
|
Please fix this issue. |
|
I wrote two tests so that everyone can reproduce what happens easily: brwe@ffa2429 In brief: I am unsure yet how a correct fix would look like but do wonder why we try highlingting on numeric and geo fields at all? If anyone has an opinion let me know. |
|
I missed @jpountz comment:
I agree. Will make a pr for that. |
clintongormley
added
:Search Relevance/Highlighting
|
@clintongormley Yes you are right. #11364 only addresses problems one gets when the way text is indexed is not compatible with the highlighter used. I do not remember why I did not exclude numeric fields then. |
… in fieldname We should prevent highlighting if a field is anything but a text or keyword field. However, someone might implement a custom field type that has text and still want to highlight on that. We cannot know in advance if the highlighter will be able to highlight such a field and so we do the following: If the field is only highlighted because the field matches a wildcard we assume it was a mistake and do not process it. If the field was explicitly given we assume that whoever issued the query knew what they were doing and try to highlight anyway. closes elastic#17537
Geo queries and plain highlighter do not seem to work well together (https://issues.apache.org/jira/browse/LUCENE-7293) so we need to skip all geo related queries when we highlight. closes elastic#17537
Geo queries and plain highlighter do not seem to work well together (https://issues.apache.org/jira/browse/LUCENE-7293) so we need to skip all geo related queries when we highlight. closes #17537
Geo queries and plain highlighter do not seem to work well together (https://issues.apache.org/jira/browse/LUCENE-7293) so we need to skip all geo related queries when we highlight. closes #17537
|
Hi @brwe , I am using ES 2.4.0, and the query is of below nature, and I still get that error. So, is this issue still open? |
|
@ajayar I cannot reproduce the failure on 2.4.1 Could you paste the whole query including the actual geo points and the full error message please? Stacktrace should be in the elasticsearch log. |
|
Sorry for the delay. Please check the details below: ` |
|
@ajayar Thanks a lot for the example, that was very helpful. The example works fine for me on 2.4 but on 2.3.5 (where this error should not occur either) the query fails for me. I will open another pr to address this. |
We skip GeoPointInBBoxQuery already but not when it was rewritten it appears as GeoPointInBBoxQuery and needs to be skipped as well. see elastic#17537
* skip GeoPointMultiTermQuery when highlighting We skip GeoPointInBBoxQuery already but not when it was rewritten it appears as GeoPointInBBoxQuery and needs to be skipped as well. see #17537
* skip GeoPointMultiTermQuery when highlighting We skip GeoPointInBBoxQuery already but not when it was rewritten it appears as GeoPointInBBoxQuery and needs to be skipped as well. see #17537
|
Hi @brwe thanks a lot for looking into it. Please check the output of the query |
|
@ajangus you are saying you are using 2.4 but this index is created with 2.3.4 maybe you are missing something on your end? |
|
I too noticed the version, but am not sure why it is so. I am sure that I am running 2.4. But, I will recheck the same on a different machine and come back on this. |
|
@ajayar I am sure you are not creating this index with a cluster that has only 2.4 nodes. if you have a single 2.3.4 node in the cluster this will happen. Maybe you can just paste the output of |
|
Apologies. Yes, you are right it was indeed pointing to 2.3.4. Had my scripts pointing to a wrong node. I don't see the issue happening on 2.4 |
|
yes. @ajayar thanks again for the detailed report! |
javanna
added
the
Team:Search Relevance
Hi guys,
we have upgraded ElasticSearch from 2.3.0 and reindexed our geolocations so the latitude and longitude are stored separately. We have noticed that some of our visualisation started to fail after we add a filter based on geolocation rectangle. However, map visualisation are working just fine. The problem occurs when we include actual documents. In this case, we get some failed shards (usually 1 out of 5) and error: Invalid shift value (xx) in prefixCoded bytes (is encoded value really a geo point?).
Details:
Our geolocation index is based on:
The ok query with the error is as follows. If we change the query size to 0 (map visualizations example), the query completes without problem.
Elasticsearch version**: 2.3.0
OS version**: Elasticsearch docker image with head plugin, marvel and big desk installed
Thank you for your help,
regards,
Jakub Smid
The text was updated successfully, but these errors were encountered: