Fix distributed aggregative query error

[li-ang]

Fix issue #4937 issue #4965
When distributed aggregative query covers two more seriesKeys(measurement + tagSets) in a shard file, the aggregative query result which is passed accross cluster, would has two more mapperValues. The node of cluster, revieving the aggregative query result, could regard the query result as raw query result. It will lead to ERR: unexpected end of JSON input or double results described by issue #4965 .

• CHANGELOG.md updated
• Rebased/mergable
• Tests pass
• Sign CLA

[li-ang]

@otoolep Can you review the PR?

[corylanou]

I would like to hold off on merging this until we merge #4848. This will allow us to write a cluster test to make sure we don't regress. Hopefully we can merge the cluster test PR today.

[corylanou]

Ok, that PR is merged. I'm happy to write the test for you if you can give me an idea of what type of query was failing before. If you can provide a couple of sample writes with a query that should be all I need to reproduce in the master branch.

[li-ang]

@corylanou
I suggest you should create a InfluxDB cluster and set replication of the retention policy as 1. Then, write some points _which are different from tagSets_ . At last, execute a aggregative query which can _cover all points_ you write in.

The following is a repreduce sample:

Create InfluxDB cluster with three node. There is cluster stats:

> show servers
id  cluster_addr    raft    raft-leader
1   127.0.0.1:8088  true    true
2   127.0.0.1:9099  true    false
3   127.0.0.1:10101 true    false

Create a database named foo and a retention policy named one_hour with duration 1h and replication 1:

> create database foo
> create retention policy one_hour on foo duration 1h replication 1
> show databases
name: databases
---------------
name
foo

> show retention policies on foo
name        duration    replicaN    default
default     0       3       true
one_hour    1h0m0s      1       false

> 

Write several points to one_hour retention policy of foo database using curl:

➜  ~  curl -i -XPOST 'http://127.0.0.1:8086/write?db=foo&rp=one_hour' --data-binary 'cpu,host=host_1,region=region_1 value=1'
HTTP/1.1 204 No Content
Request-Id: 0632942a-98a4-11e5-8013-000000000000
X-Influxdb-Version: 0.9
Date: Wed, 02 Dec 2015 03:23:15 GMT
Connection: close

➜  ~  curl -i -XPOST 'http://127.0.0.1:8086/write?db=foo&rp=one_hour' --data-binary 'cpu,host=host_2,region=region_2 value=2'
HTTP/1.1 204 No Content
Request-Id: 0c490066-98a4-11e5-8014-000000000000
X-Influxdb-Version: 0.9
Date: Wed, 02 Dec 2015 03:23:25 GMT
Connection: close

➜  ~  curl -i -XPOST 'http://127.0.0.1:8086/write?db=foo&rp=one_hour' --data-binary 'cpu,host=host_3,region=region_3 value=3'
HTTP/1.1 204 No Content
Request-Id: 100443e8-98a4-11e5-8015-000000000000
X-Influxdb-Version: 0.9
Date: Wed, 02 Dec 2015 03:23:31 GMT
Connection: close

➜  ~  curl -i -XPOST 'http://127.0.0.1:8086/write?db=foo&rp=one_hour' --data-binary 'cpu,host=host_4,region=region_4 value=4'
HTTP/1.1 204 No Content
Request-Id: 1432f71e-98a4-11e5-8016-000000000000
X-Influxdb-Version: 0.9
Date: Wed, 02 Dec 2015 03:23:38 GMT
Connection: close

Now, there are four points in foo database:

➜  ~  curl -G 'http://127.0.0.1:8086/query?pretty=true' --data-urlencode "db=foo" --data-urlencode "q=SELECT * FROM one_hour.cpu"          
{
    "results": [
        {
            "series": [
                {
                    "name": "cpu",
                    "columns": [
                        "time",
                        "host",
                        "region",
                        "value"
                    ],
                    "values": [
                        [
                            "2015-12-02T03:23:15.070506552Z",
                            "host_1",
                            "region_1",
                            1
                        ],
                        [
                            "2015-12-02T03:23:25.283799043Z",
                            "host_2",
                            "region_2",
                            2
                        ],
                        [
                            "2015-12-02T03:23:31.544199738Z",
                            "host_3",
                            "region_3",
                            3
                        ],
                        [
                            "2015-12-02T03:23:38.561141279Z",
                            "host_4",
                            "region_4",
                            4
                        ]
                    ]
                }
            ]
        }
    ]
}

use mean or other functions to execute aggregative query:

 ➜  ~  curl -G 'http://127.0.0.1:8086/query?pretty=true' --data-urlencode "db=foo" --data-urlencode "q=SELECT mean(value) FROM one_hour.cpu"
{
    "results": [
        {
            "error": "unexpected end of JSON input"
        }
    ]
}
➜  ~  curl -G 'http://127.0.0.1:8086/query?pretty=true' --data-urlencode "db=foo" --data-urlencode "q=SELECT sum(value) FROM one_hour.cpu"
{
    "results": [
        {
            "error": "unexpected end of JSON input"
        }
    ]
}
➜  ~  curl -G 'http://127.0.0.1:8086/query?pretty=true' --data-urlencode "db=foo" --data-urlencode "q=SELECT min(value) FROM one_hour.cpu"
{
    "results": [
        {
            "error": "unexpected end of JSON input"
        }
    ]
}
➜  ~  curl -G 'http://127.0.0.1:8086/query?pretty=true' --data-urlencode "db=foo" --data-urlencode "q=SELECT max(value) FROM one_hour.cpu"
{
    "results": [
        {
            "error": "unexpected end of JSON input"
        }
    ]
}

[li-ang]

@corylanou
The followings are curl command.

curl -G 'http://127.0.0.1:8086/query' --data-urlencode "q=create database foo"
curl -G 'http://127.0.0.1:8086/query' --data-urlencode "q=create retention policy one_hour on foo duration 1h replication 1"
curl -X POST 'http://127.0.0.1:8086/write?db=foo&rp=one_hour' --data-binary 'cpu,host=host_1,region=region_1 value=1'
curl -X POST 'http://127.0.0.1:8086/write?db=foo&rp=one_hour' --data-binary 'cpu,host=host_2,region=region_2 value=2'
curl -X POST 'http://127.0.0.1:8086/write?db=foo&rp=one_hour' --data-binary 'cpu,host=host_3,region=region_3 value=3'
curl -X POST 'http://127.0.0.1:8086/write?db=foo&rp=one_hour' --data-binary 'cpu,host=host_4,region=region_4 value=4'

[otoolep]

@dgnorton @corylanou -- what do you think? Should we get this in for 0.9.6?

[li-ang]

@otoolep Please wait a miniute, I will add some commits to this pr.

[li-ang]

@otoolep Done!

[otoolep]

Thanks @li-ang

@dgnorton @corylanou -- what do you think?

[corylanou]

We have no testing around this yet. It passes the existing tests so it doesn't introduce any new single node bugs that we are aware of. We can add more tests in a following PR if you want to ship this.

[otoolep]

Thanks @li-ang