Skip to content

Commit

Permalink
[receiver/oracledb] Fix incorrect values for a couple of metrics (#32028
Browse files Browse the repository at this point in the history
)

**Description:** 
Values were being scraped incorrectly for the metrics
`oracledb.tablespace_size.limit` and `oracledb.tablespace_size.usage`.
The changes these metrics to be scraped from the
[`DBA_TABLESPACE_USAGE_METRICS`](https://docs.oracle.com/en/database/oracle/oracle-database/19/refrn/DBA_TABLESPACE_USAGE_METRICS.html#GUID-FE479528-BB37-4B55-92CF-9EC19EDF4F46)
table. This results in a slight loss of granularity in these metrics, as
values will always be in multiples of the respective tablespace's block
size, but I think the clarity and simplicity is worth the trade off.

Note: The value of the usage metric was generally close to the expected
value, but the limit was being calculated as potential theoretical
capacity, unbound by server capacity. For example, in testing in a
docker container on my local machine, limit was set to **17TB**. This
doesn't line up with user expectations.

**Link to tracking Issue:**
Fixes
#31451

**Testing:** 
Updated existing tests, added a couple new ones.

Also, the original issue filed was comparing
`DBA_TABLESPACE_USAGE_METRICS` output for percent used to what we got
from `usage/limit * 100`. Here's the local testing outputs compared to
show they now line up.
```
2024-03-27T16:31:57.938-0700    info    oracledbreceiver/scraper.go:285 DBA_TABLESPACE_USAGE_METRICS: Tablespace name: SYSTEM, used space: 111288, tablespace size: 3518587, percent used: 3.16286054600895188892586711654422641816    {"kind": "receiver", "name": "oracledb", "data_type": "metrics"}
```

```
Metric #20
Descriptor:
     -> Name: oracledb.tablespace_size.usage
     -> Description: Used tablespace in bytes.
     -> Unit: By
     -> DataType: Gauge
NumberDataPoints #0
Data point attributes:
     -> tablespace_name: Str(SYSTEM)
StartTimestamp: 2024-03-27 23:31:56.873576 +0000 UTC
Timestamp: 2024-03-27 23:32:12.523295 +0000 UTC
Value: 911671296
```

```
Metric #19
Descriptor:
     -> Name: oracledb.tablespace_size.limit
     -> Description: Maximum size of tablespace in bytes, -1 if unlimited.
     -> Unit: By
     -> DataType: Gauge
NumberDataPoints #0
Data point attributes:
     -> tablespace_name: Str(SYSTEM)
StartTimestamp: 2024-03-27 23:31:56.873576 +0000 UTC
Timestamp: 2024-03-27 23:32:12.523295 +0000 UTC
Value: 28824264704
```
Doing the same calculation, we get:
```
(911671296 / 28824264704) * 100 = ~3.16%
```
  • Loading branch information
crobert-1 authored Mar 28, 2024
1 parent b8efeeb commit ac74006
Show file tree
Hide file tree
Showing 6 changed files with 85 additions and 41 deletions.
27 changes: 27 additions & 0 deletions .chloggen/oracledb_tablespace_fix.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: bug_fix

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: oracledbreceiver

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Fix incorrect values being set for oracledb.tablespace_size.limit and oracledb.tablespace_size.usage

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [31451]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion receiver/oracledbreceiver/metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,6 @@ metrics:
enabled: true
gauge:
value_type: int
input_type: string
unit: By
oracledb.db_block_gets:
description: Number of times a current block was requested from the buffer cache.
Expand Down
58 changes: 33 additions & 25 deletions receiver/oracledbreceiver/scraper.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,10 @@ const (
consistentGets = "consistent gets"
sessionCountSQL = "select status, type, count(*) as VALUE FROM v$session GROUP BY status, type"
systemResourceLimitsSQL = "select RESOURCE_NAME, CURRENT_UTILIZATION, LIMIT_VALUE, CASE WHEN TRIM(INITIAL_ALLOCATION) LIKE 'UNLIMITED' THEN '-1' ELSE TRIM(INITIAL_ALLOCATION) END as INITIAL_ALLOCATION, CASE WHEN TRIM(LIMIT_VALUE) LIKE 'UNLIMITED' THEN '-1' ELSE TRIM(LIMIT_VALUE) END as LIMIT_VALUE from v$resource_limit"
tablespaceUsageSQL = "select TABLESPACE_NAME, BYTES from DBA_DATA_FILES"
tablespaceMaxSpaceSQL = "select TABLESPACE_NAME, (BLOCK_SIZE*MAX_EXTENTS) AS VALUE FROM DBA_TABLESPACES"
tablespaceUsageSQL = `
select um.TABLESPACE_NAME, um.USED_SPACE, um.TABLESPACE_SIZE, ts.BLOCK_SIZE
FROM DBA_TABLESPACE_USAGE_METRICS um INNER JOIN DBA_TABLESPACES ts
ON um.TABLESPACE_NAME = ts.TABLESPACE_NAME`
)

type dbProviderFunc func() (*sql.DB, error)
Expand All @@ -48,7 +50,6 @@ type clientProviderFunc func(*sql.DB, string, *zap.Logger) dbClient

type scraper struct {
statsClient dbClient
tablespaceMaxSpaceClient dbClient
tablespaceUsageClient dbClient
systemResourceLimitsClient dbClient
sessionCountClient dbClient
Expand Down Expand Up @@ -88,7 +89,6 @@ func (s *scraper) start(context.Context, component.Host) error {
s.sessionCountClient = s.clientProviderFunc(s.db, sessionCountSQL, s.logger)
s.systemResourceLimitsClient = s.clientProviderFunc(s.db, systemResourceLimitsSQL, s.logger)
s.tablespaceUsageClient = s.clientProviderFunc(s.db, tablespaceUsageSQL, s.logger)
s.tablespaceMaxSpaceClient = s.clientProviderFunc(s.db, tablespaceMaxSpaceSQL, s.logger)
return nil
}

Expand Down Expand Up @@ -274,41 +274,49 @@ func (s *scraper) scrape(ctx context.Context) (pmetric.Metrics, error) {
}
}
}
if s.metricsBuilderConfig.Metrics.OracledbTablespaceSizeUsage.Enabled {

if s.metricsBuilderConfig.Metrics.OracledbTablespaceSizeUsage.Enabled ||
s.metricsBuilderConfig.Metrics.OracledbTablespaceSizeLimit.Enabled {
rows, err := s.tablespaceUsageClient.metricRows(ctx)
if err != nil {
scrapeErrors = append(scrapeErrors, fmt.Errorf("error executing %s: %w", tablespaceUsageSQL, err))
} else {
now := pcommon.NewTimestampFromTime(time.Now())
for _, row := range rows {
tablespaceName := row["TABLESPACE_NAME"]
err := s.mb.RecordOracledbTablespaceSizeUsageDataPoint(now, row["BYTES"], tablespaceName)
usedSpaceBlockCount, err := strconv.ParseInt(row["USED_SPACE"], 10, 64)
if err != nil {
scrapeErrors = append(scrapeErrors, err)
scrapeErrors = append(scrapeErrors, fmt.Errorf("failed to parse int64 for OracledbTablespaceSizeUsage, value was %s: %w", row["USED_SPACE"], err))
continue
}
}
}
}
if s.metricsBuilderConfig.Metrics.OracledbTablespaceSizeLimit.Enabled {
rows, err := s.tablespaceMaxSpaceClient.metricRows(ctx)
if err != nil {
scrapeErrors = append(scrapeErrors, fmt.Errorf("error executing %s: %w", tablespaceMaxSpaceSQL, err))
} else {
now := pcommon.NewTimestampFromTime(time.Now())
for _, row := range rows {
tablespaceName := row["TABLESPACE_NAME"]
var val int64
inputVal := row["VALUE"]
if inputVal == "" {
val = -1

tablespaceSizeOriginal := row["TABLESPACE_SIZE"]
var tablespaceSizeBlockCount int64
// Tablespace size should never be empty using the DBA_TABLESPACE_USAGE_METRICS query. This logic is done
// to preserve backward compatibility for with the original metric gathered from querying DBA_TABLESPACES
if tablespaceSizeOriginal == "" {
tablespaceSizeBlockCount = -1
} else {
val, err = strconv.ParseInt(inputVal, 10, 64)
tablespaceSizeBlockCount, err = strconv.ParseInt(tablespaceSizeOriginal, 10, 64)
if err != nil {
scrapeErrors = append(scrapeErrors, fmt.Errorf("failed to parse int64 for OracledbTablespaceSizeLimit, value was %s: %w", inputVal, err))
scrapeErrors = append(scrapeErrors, fmt.Errorf("failed to parse int64 for OracledbTablespaceSizeLimit, value was %s: %w", tablespaceSizeOriginal, err))
continue
}
}
s.mb.RecordOracledbTablespaceSizeLimitDataPoint(now, val, tablespaceName)

blockSize, err := strconv.ParseInt(row["BLOCK_SIZE"], 10, 64)
if err != nil {
scrapeErrors = append(scrapeErrors, fmt.Errorf("failed to parse int64 for OracledbBlockSize, value was %s: %w", row["BLOCK_SIZE"], err))
continue
}

s.mb.RecordOracledbTablespaceSizeUsageDataPoint(now, usedSpaceBlockCount*blockSize, tablespaceName)

if tablespaceSizeBlockCount < 0 {
s.mb.RecordOracledbTablespaceSizeLimitDataPoint(now, -1, tablespaceName)
} else {
s.mb.RecordOracledbTablespaceSizeLimitDataPoint(now, tablespaceSizeBlockCount*blockSize, tablespaceName)
}
}
}
}
Expand Down
31 changes: 23 additions & 8 deletions receiver/oracledbreceiver/scraper_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,7 @@ var queryResponses = map[string][]metricRow{
sessionCountSQL: {{"VALUE": "1"}},
systemResourceLimitsSQL: {{"RESOURCE_NAME": "processes", "CURRENT_UTILIZATION": "3", "MAX_UTILIZATION": "10", "INITIAL_ALLOCATION": "100", "LIMIT_VALUE": "100"},
{"RESOURCE_NAME": "locks", "CURRENT_UTILIZATION": "3", "MAX_UTILIZATION": "10", "INITIAL_ALLOCATION": "-1", "LIMIT_VALUE": "-1"}},
tablespaceUsageSQL: {{"TABLESPACE_NAME": "SYS", "BYTES": "1024"}},
tablespaceMaxSpaceSQL: {{"TABLESPACE_NAME": "SYS", "VALUE": "1024"}},
tablespaceUsageSQL: {{"TABLESPACE_NAME": "SYS", "USED_SPACE": "111288", "TABLESPACE_SIZE": "3518587", "BLOCK_SIZE": "8192"}},
}

func TestScraper_Scrape(t *testing.T) {
Expand Down Expand Up @@ -76,11 +75,11 @@ func TestScraper_Scrape(t *testing.T) {
{
name: "no limit on tablespace",
dbclientFn: func(_ *sql.DB, s string, _ *zap.Logger) dbClient {
if s == tablespaceMaxSpaceSQL {
if s == tablespaceUsageSQL {
return &fakeDbClient{Responses: [][]metricRow{
{
{"TABLESPACE_NAME": "SYS", "VALUE": "1024"},
{"TABLESPACE_NAME": "FOO", "VALUE": ""},
{"TABLESPACE_NAME": "SYS", "TABLESPACE_SIZE": "1024", "USED_SPACE": "111288", "BLOCK_SIZE": "8192"},
{"TABLESPACE_NAME": "FOO", "TABLESPACE_SIZE": "", "USED_SPACE": "111288", "BLOCK_SIZE": "8192"},
},
}}
}
Expand All @@ -92,11 +91,11 @@ func TestScraper_Scrape(t *testing.T) {
{
name: "bad value on tablespace",
dbclientFn: func(_ *sql.DB, s string, _ *zap.Logger) dbClient {
if s == tablespaceMaxSpaceSQL {
if s == tablespaceUsageSQL {
return &fakeDbClient{Responses: [][]metricRow{
{
{"TABLESPACE_NAME": "SYS", "VALUE": "1024"},
{"TABLESPACE_NAME": "FOO", "VALUE": "ert"},
{"TABLESPACE_NAME": "SYS", "TABLESPACE_SIZE": "1024", "USED_SPACE": "111288", "BLOCK_SIZE": "8192"},
{"TABLESPACE_NAME": "FOO", "TABLESPACE_SIZE": "ert", "USED_SPACE": "111288", "BLOCK_SIZE": "8192"},
},
}}
}
Expand All @@ -106,6 +105,22 @@ func TestScraper_Scrape(t *testing.T) {
},
errWanted: `failed to parse int64 for OracledbTablespaceSizeLimit, value was ert: strconv.ParseInt: parsing "ert": invalid syntax`,
},
{
name: "Empty block size",
dbclientFn: func(_ *sql.DB, s string, _ *zap.Logger) dbClient {
if s == tablespaceUsageSQL {
return &fakeDbClient{Responses: [][]metricRow{
{
{"TABLESPACE_NAME": "SYS", "TABLESPACE_SIZE": "1024", "USED_SPACE": "111288", "BLOCK_SIZE": ""},
},
}}
}
return &fakeDbClient{Responses: [][]metricRow{
queryResponses[s],
}}
},
errWanted: `failed to parse int64 for OracledbBlockSize, value was : strconv.ParseInt: parsing "": invalid syntax`,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
Expand Down

0 comments on commit ac74006

Please sign in to comment.