Add support for Composite spans in the intake API

[estolfo]

Motivation/summary

Support a composite object in the span model and in the intake API. The composite object holds metrics on a group of spans represented by a single one.

The composite object has the following format

{
   "name": "SELECT FROM product_types WHERE user_id=?",
   "timestamp": 1613575086000000,
   "duration": 3781,
   "composite": {
       	"count": 10,
        "sum": 3592,
        "compression_strategy": "exact_match"
    }
}

Checklist

• Update CHANGELOG.asciidoc
• Documentation has been updated

How to test these changes

To test these changes specifically, not including the changes to span metrics calculations:

• make
• docker compose up -d
• Add the ES credentials to apm-server.yml (username: "admin", password: "changeme")
• ./apm-server -c apm-server.yml -e -d "*"
• curl -H "Content-type: application/x-ndjson" --data-binary @testdata/intake-v2/spans.ndjson http://localhost:8200/intake/v2/events
• In Kibana Dev Tools, run the following query and verify the span document has the expected composite nested object.

GET /apm-8.0.0-span/_search
{
  "query": {
    "match": {
      "trace.id": "edcbaf0123456789abcdef9876543210"
    }
  }
}

Expected nested document:

            "composite" : {
              "count" : 10,
              "sum" : {
                "us" : 359298
              },
              "compression_strategy" : "exact_match"
            }

This can also be tested along with #5595.

Related issues

Related #5485
Related #5595

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-07-21T13:25:54.754+0000

• Duration: 39 min 55 sec

• Commit: f65e583

Test stats 🧪

Test	Results
Failed	0
Passed	5965
Skipped	14
Total	5979

Trends 🧪

[axw]

Nice work! What's there looks good, just a few very minor comments aside from the one about end. I'm not sure if we really need that. If we do need it, I'm wondering if we could store it as the ECS field event.end instead.

[axw]

Basically LGTM, but I just realised we still need to update https://github.com/elastic/apm-server/blob/master/model/span/_meta/fields.yml. This defines how fields are mapped in Elasticsearch. After that file is updated, you will need to run make update to update generated docs and integration package files.

[estolfo]

As you're aware @axw, the agent spec is still being discussed and updated. So I'm going to wait until that's settled a bit more, update this PR, then request a review again. Thanks for your review so far!

[mergify]

This pull request is now in conflicts. Could you fix it @estolfo? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b composite-spans upstream/composite-spans
git merge upstream/master
git push upstream composite-spans

[axw]

Looks great, thanks! Just one minor comment. Please go ahead and merge after adding a comment on Composite.Sum. When you do, please add the backport-7.x label to have Mergify automatically create the backport.

[stuartnelson3]

Confirmed with BC2