Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix infoSchemaInnodbMetricsEnabledColumnQuery #687

Merged
merged 1 commit into from
Apr 14, 2023
Merged

Fix infoSchemaInnodbMetricsEnabledColumnQuery #687

merged 1 commit into from
Apr 14, 2023

Conversation

cezmunsta
Copy link
Contributor

@cezmunsta cezmunsta commented Nov 29, 2022 •
edited
Loading

The query introduced with #523 performs poorly when there are sufficient numbers of rows in information_schema.columns.

  • Added using the table_schema as a filter
  • Added limit to the query

@cezmunsta cezmunsta marked this pull request as draft November 29, 2022 12:59
@cezmunsta
Fix infoSchemaInnodbMetricsEnabledColumnQuery
e25198d 
The query introduced with prometheus#523 performs poorly when there
are sufficient numbers of columns in `information_schema.columns`.

* Added using the `table_schema` as a filter
* Added limit to the query

Signed-off-by: cezmunsta <github@incoming-email.co.uk>
@cezmunsta cezmunsta force-pushed the fix-infoSchemaInnodbMetricsEnabledColumnQuery branch from 212ea91 to e25198d Compare November 29, 2022 13:02
@cezmunsta cezmunsta marked this pull request as ready for review November 29, 2022 13:04
@cezmunsta
Copy link
Contributor Author

@SuperQ please review

@cezmunsta cezmunsta mentioned this pull request Nov 29, 2022
Merged
@ademidoff
Copy link

when there are sufficient numbers of columns in information_schema.columns

@cezmunsta I wonder, what would the definition of sufficient be?

@cezmunsta
Copy link
Contributor Author

@cezmunsta I wonder, what would the definition of sufficient be?

@atymchuk other factors influence it, such as whether or not it needs to open tables (information_schema pre-8.0 MySQL). On servers where there are extremely high numbers of tables, even certain OS commands should be avoided for the MySQL datadir.
The PMM client includes a safety net for pmm-admin for tablestats, with the default being 1000 tables. There is also a simple query performance example on the PMM ticket:

SELECT
column_name
FROM information_schema.columns
WHERE table_name = 'INNODB_METRICS';

That alone took 3.56 sec for the user.

SuperQ
SuperQ approved these changes Apr 14, 2023
View reviewed changes
Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks

@SuperQ SuperQ merged commit 7f4eb83 into prometheus:main Apr 14, 2023
@SuperQ SuperQ mentioned this pull request Apr 14, 2023
Merged
SuperQ added a commit that referenced this pull request Apr 14, 2023
@SuperQ
Release v0.15.0-rc.0
b67b861 
BREAKING CHANGES:

The exporter no longer supports the monolithic `DATA_SOURCE_NAME` environment variable.
To configure connections to MySQL you can either use a `my.cnf` style config file or command line arguments.

For example:

    export MYSQLD_EXPORTER_PASSWORD=secret
    mysqld_exporter --mysqld.address=localhost:3306 --mysqld.username=exporter

We have also dropped some internal scrape metrics:
* `mysql_exporter_scrapes_total`
* `mysql_exporter_scrape_errors_total`
* `mysql_last_scrape_failed`

Changes:

* [CHANGE] Allow tlsCfg.InsecureSkipVerify outside of mTLS #631
* [CHANGE] Update to exporter-toolkit v0.8.1 #677
* [CHANGE] Fix shared metrics between requests #722
* [FEATURE] Add support for collecting metrics from sys.user_summary #628
* [FEATURE] Support for multi-target mysqld probes #651
* [FEATURE] Add MySQL TLS configurations #718
* [BUGFIX] Fix infoSchemaInnodbMetricsEnabledColumnQuery #687

Signed-off-by: SuperQ <superq@gmail.com>
SuperQ added a commit that referenced this pull request Apr 14, 2023
@SuperQ
Release v0.15.0-rc.0
6e73fce 
BREAKING CHANGES:

The exporter no longer supports the monolithic `DATA_SOURCE_NAME` environment variable.
To configure connections to MySQL you can either use a `my.cnf` style config file or command line arguments.

For example:

    export MYSQLD_EXPORTER_PASSWORD=secret
    mysqld_exporter --mysqld.address=localhost:3306 --mysqld.username=exporter

We have also dropped some internal scrape metrics:
* `mysql_exporter_scrapes_total`
* `mysql_exporter_scrape_errors_total`
* `mysql_last_scrape_failed`

Changes:

* [CHANGE] Allow tlsCfg.InsecureSkipVerify outside of mTLS #631
* [CHANGE] Update to exporter-toolkit v0.8.1 #677
* [CHANGE] Fix shared metrics between requests #722
* [FEATURE] Add support for collecting metrics from sys.user_summary #628
* [FEATURE] Support for multi-target mysqld probes #651
* [FEATURE] Add MySQL TLS configurations #718
* [BUGFIX] Fix infoSchemaInnodbMetricsEnabledColumnQuery #687

Signed-off-by: SuperQ <superq@gmail.com>
SuperQ added a commit that referenced this pull request Jun 19, 2023
@SuperQ
Release 0.15.0
a726b4d 
* [CHANGE] Allow `tlsCfg.InsecureSkipVerify` outside of mTLS #631
* [CHANGE] Update to exporter-toolkit v0.8.1 #677
* [CHANGE] Fix shared metrics between requests #722
* [FEATURE] Add support for collecting metrics from `sys.user_summary` #628
* [FEATURE] Support for multi-target mysqld probes #651
* [FEATURE] Add MySQL TLS configurations #718
* [ENHANCEMENT] Add UNIX domain socket support for multi-target scraping #707
* [BUGFIX] Fix `infoSchemaInnodbMetricsEnabledColumnQuery` #687
* [BUGFIX] Allow empty passwords #742

Signed-off-by: SuperQ <superq@gmail.com>
@SuperQ SuperQ mentioned this pull request Jun 19, 2023
Merged
SuperQ added a commit that referenced this pull request Jun 22, 2023
@SuperQ
Release 0.15.0
0eeb39b 
* [CHANGE] Allow `tlsCfg.InsecureSkipVerify` outside of mTLS #631
* [CHANGE] Update to exporter-toolkit v0.8.1 #677
* [CHANGE] Fix shared metrics between requests #722
* [FEATURE] Add support for collecting metrics from `sys.user_summary` #628
* [FEATURE] Support for multi-target mysqld probes #651
* [FEATURE] Add MySQL TLS configurations #718
* [ENHANCEMENT] Add UNIX domain socket support for multi-target scraping #707
* [BUGFIX] Fix `infoSchemaInnodbMetricsEnabledColumnQuery` #687
* [BUGFIX] Allow empty passwords #742

Signed-off-by: SuperQ <superq@gmail.com>
SuperQ added a commit that referenced this pull request Jun 22, 2023
@SuperQ
Release 0.15.0
95b3d20 
BREAKING CHANGES:

The exporter no longer supports the monolithic `DATA_SOURCE_NAME` environment variable.
To configure connections to MySQL you can either use a `my.cnf` style config file or command line arguments.

For example:

    export MYSQLD_EXPORTER_PASSWORD=secret
    mysqld_exporter --mysqld.address=localhost:3306 --mysqld.username=exporter

We have also dropped some internal scrape metrics:
- `mysql_exporter_scrapes_total`
- `mysql_exporter_scrape_errors_total`
- `mysql_last_scrape_failed`

The default client configuration file is now `.my.cnf` in the process working directory. Use `--config.my-cnf="$HOME/.my.cnf"` to retain the previous default.

Changes:

* [CHANGE] Allow `tlsCfg.InsecureSkipVerify` outside of mTLS #631
* [CHANGE] Update to exporter-toolkit v0.8.1 #677
* [CHANGE] Fix shared metrics between requests #722
* [CHANGE] Allow empty passwords #742
* [CHANGE] Don't use HOME env in the my-cnf config path. #745
* [FEATURE] Add support for collecting metrics from `sys.user_summary` #628
* [FEATURE] Support for multi-target mysqld probes #651
* [FEATURE] Add MySQL TLS configurations #718
* [FEATURE] Add config reload via /-/reload #734
* [ENHANCEMENT] Add UNIX domain socket support for multi-target scraping #707
* [ENHANCEMENT] Use `STRAIGHT_JOIN` in infoSchemaAutoIncrementQuery #726
* [BUGFIX] Fix `infoSchemaInnodbMetricsEnabledColumnQuery` #687
* [BUGFIX] Allow empty passwords #742

Signed-off-by: SuperQ <superq@gmail.com>
SuperQ added a commit that referenced this pull request Jun 24, 2023
@SuperQ
Release 0.15.0 (#744)
6ca2a42 
BREAKING CHANGES:

The exporter no longer supports the monolithic `DATA_SOURCE_NAME` environment variable.
To configure connections to MySQL you can either use a `my.cnf` style config file or command line arguments.

For example:

    export MYSQLD_EXPORTER_PASSWORD=secret
    mysqld_exporter --mysqld.address=localhost:3306 --mysqld.username=exporter

We have also dropped some internal scrape metrics:
- `mysql_exporter_scrapes_total`
- `mysql_exporter_scrape_errors_total`
- `mysql_last_scrape_failed`

The default client configuration file is now `.my.cnf` in the process working directory. Use `--config.my-cnf="$HOME/.my.cnf"` to retain the previous default.

Changes:

* [CHANGE] Allow `tlsCfg.InsecureSkipVerify` outside of mTLS #631
* [CHANGE] Update to exporter-toolkit v0.8.1 #677
* [CHANGE] Fix shared metrics between requests #722
* [CHANGE] Allow empty passwords #742
* [CHANGE] Don't use HOME env in the my-cnf config path. #745
* [FEATURE] Add support for collecting metrics from `sys.user_summary` #628
* [FEATURE] Support for multi-target mysqld probes #651
* [FEATURE] Add MySQL TLS configurations #718
* [FEATURE] Add config reload via /-/reload #734
* [ENHANCEMENT] Add UNIX domain socket support for multi-target scraping #707
* [ENHANCEMENT] Use `STRAIGHT_JOIN` in infoSchemaAutoIncrementQuery #726
* [BUGFIX] Fix `infoSchemaInnodbMetricsEnabledColumnQuery` #687
* [BUGFIX] Allow empty passwords #742

Signed-off-by: SuperQ <superq@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SuperQ SuperQ SuperQ approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@cezmunsta @ademidoff @SuperQ