Distributor: Add limit for exemplar per series per request

[ying-jeanne]

What this PR does

In this pull request, we propose safeguarding the write path by implementing a restriction on the number of exemplars/series per request. Exemplars exceeding this limit will be discarded, and these discards will be monitored through the cortex_discarded_exemplars_total metric.

Which issue(s) this PR fixes or relates to

Fixes #

Checklist

• Tests updated.
• Documentation added.
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
• about-versioning.md updated with experimental features.

[pstibrany]

Left some nits on top of Nick's comments.

[aknuds1]

I found some typos. I also wonder if instead the flag name should be -validation.max-exemplars-per-series-per-request, to be consistent with -validation.max-label-names-per-series?

[56quarters]

LGTM. Approving but I'd like to see @aknuds1's suggestions implemented. Thanks!

[grafanabot]

The backport to r287 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-7989-to-r287 origin/r287
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 6af4273313e901f97fe2bffc6dc07b6e8b11e7cb
# Push it to GitHub
git push --set-upstream origin backport-7989-to-r287
git switch main
# Remove the local backport branch
git branch -D backport-7989-to-r287

Then, create a pull request where the base branch is r287 and the compare/head branch is backport-7989-to-r287.

[aknuds1]

I think the reason should include the fact that it's per request, so one doesn't have the impression that series have too many exemplars, and to reflect the flag name: "too_many_exemplars_per_series_per_request".