Ensure integration package validity using elastic-package tool

[axw]

Motivation/summary

Ensure the integration package is valid by running elastic-package check as part of make check.

Add Make targets:

• make check-package
• make format-package (run by make gen-package)
• make build-package (run by make gen-package)

Some minor formatting was needed, as well as adding a link to the changelog entry.

Later we should consider getting rid of our custom README.template rendering, and rely on elastic-package build to render a template located at _dev/build/docs/README.template, like in integrations.

How to test these changes

Mess up the formatting of the package, and run make check-package. It should fail and tell you the package is not formatted correctly.

Related issues

Closes #4930

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: Pull request #4953 opened

• Start Time: 2021-03-11T12:29:08.999+0000

• Duration: 39 min 28 sec

• Commit: 6ef8efe

Test stats 🧪

Test	Results
Failed	0
Passed	4773
Skipped	117
Total	4890

Trends 🧪

[jalvz]

Later we should consider getting rid of our custom README.template rendering, and rely on elastic-package build to render a template located at _dev/build/docs/README.template, like in integrations

FYI, the main difference is that we render an additional column with ✅ / ❌ for ECS fields. The ECS version comes from https://github.com/elastic/apm-server/blob/master/go.mod#L17.
I guess something like that could be added for integrations.

[jalvz]

Hmmm... I thought that _dev was a convention for things that don't actually belong to the published package.
Probably too far fetched, but I worry a bit about this change. Imagine eg. that instead of using README.md Kibana starts reading all *.md files and show only the first one, or concats them, etc.

[axw]

FYI, the main difference is that we render an additional column with white_check_mark / x for ECS fields. The ECS version comes from https://github.com/elastic/apm-server/blob/master/go.mod#L17. I guess something like that could be added for integrations.

Understood. I started looking at changing it over, but stopped when I realised there was that difference. I think if it's important to us it'll be important to others too -- so let's add it to the elastic-package tool. Maybe just as an option to start with.

Hmmm... I thought that _dev was a convention for things that don't actually belong to the published package.
Probably too far fetched, but I worry a bit about this change. Imagine eg. that instead of using README.md Kibana starts reading all *.md files and show only the first one, or concats them, etc.

We could move the template somewhere outside of the apmpackage/version tree, so that can't possibly happen. If we can switch over to using elastic-package's doc generation then it's moot anyhow.

[jalvz]

I think if it's important to us it'll be important to others too -- so let's add it to the elastic-package tool

I recall that comes from an specific to APM, that's why only we have it. But that makes sense, yes :+1

[jalvz]

We could move the template somewhere outside of the apmpackage/version tree, so that can't possibly happen

In the beginning it was like that, but then occurred to me that that external folder would have to be versioned too, so instead of having 2 folders with the same versioning structure I though it was simpler to put it inside a _dev folder akin to what integrations do.
I don't know if _dev is spec'ed out somewhere, but there is also https://github.com/elastic/package-spec/tree/master/versions/1/_dev, etc.

[axw]

Yes, that's where it's specified: https://github.com/elastic/package-spec/blob/master/versions/1/_dev/spec.yml
The additionalContents: false bit towards the top is what causes the tool to complain.