Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gitlab Collector: Index size collector and test #835

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

Sticksman
Copy link
Contributor

Split from #819

Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Copy link
Contributor

@sysadmind sysadmind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your work on this collector and the other PRs that you have submitted. I know that it took a lot of time to put these together and I appreciate it. I have a few comments on this PR. I will try to get to the other PRs soon, but I think much of the feedback will be similar. My priority in reviewing the code is to consider how we can maintain this project long term. I am thrilled with the forward progress on this exporter but I don't want to sacrifice quality for the sake along the way.

As for the SQL formatting, I am particular about this. Poorly formatted SQL is much harder to read and understand. We do not have a style guide and I haven't seen a good formatter for SQL inside of go yet so I don't have anything good to help you directly. This site looks like it is close to what I would like to see: https://www.sqlstyle.guide/ but I haven't read it in detail yet. I think as long as we are close with style, we can make it work.

collector/pg_index_size.go Outdated Show resolved Hide resolved
collector/pg_index_size.go Outdated Show resolved Hide resolved
if err := rows.Scan(&schemaname, &relname, &indexrelname, &indexSize); err != nil {
return err
}
schemanameLabel := "unknown"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm really not sure that these metrics even make sense if all of these are NULL. I understand that we used NULL to fix all of the bugs in the 0.13.1 release, but these are net new collectors and I would like to be much more thoughtful about how we handle them in code. For example, if the indexSize metric is NULL, why would we even report the metric? How is the zero value useful.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, if the labels are null, we should Debug log and continue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code is still emitting metrics when these values are NULL.

Sticksman and others added 3 commits June 28, 2023 18:26
Co-authored-by: Joe Adams <github@joeadams.io>
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
Co-authored-by: Joe Adams <github@joeadams.io>
Signed-off-by: Felix Yuan <felix.yuan@reddit.com>
if err := rows.Scan(&schemaname, &relname, &indexrelname, &indexSize); err != nil {
return err
}
schemanameLabel := "unknown"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code is still emitting metrics when these values are NULL.

schemaname,
tablename AS relname,
indexname AS indexrelname,
pg_class.relpages * 8192::bigint AS index_size
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't rely on 8192 being the block size. It's default, but could be changed. See references:

  1. https://www.postgresql.org/docs/current/catalog-pg-class.html (see relpages)
  2. https://www.postgresql.org/docs/current/runtime-config-preset.html (see block_size)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so we have to select block size and multiply by that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants