Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add short_metrics #4148

Merged
merged 5 commits into from
Nov 3, 2022
Merged

Add short_metrics #4148

merged 5 commits into from
Nov 3, 2022

Conversation

JackCaoG
Copy link
Collaborator

@JackCaoG JackCaoG commented Nov 2, 2022

This pr added an api short_metrics_report which by defualt will emit a very short list of relevant metrics. User can also customize which metrics they want to see. IMO full metrics is way too detail in most time.

We will always print out the aten:: counters since they indicate fallback and are important to users.

example output:

Metric: CompileTime
  TotalSamples: 2
  Accumulator: 059ms522.421us
  ValueRate: 01s330ms003.908us / second
  Rate: 45.4528 / second
  Percentiles: 1%=026ms399.367us; 5%=026ms399.367us; 10%=026ms399.367us; 20%=026ms399.367us; 50%=032ms123.054us; 80%=032ms123.054us; 90%=032ms123.054us; 95%=032ms123.054us; 99%=032ms123.054us
Metric: ExecuteTime
  TotalSamples: 3
  Accumulator: 017ms966.371us
  ValueRate: 322ms039.563us / second
  Rate: 56.9432 / second
  Percentiles: 1%=001ms307.637us; 5%=001ms307.637us; 10%=001ms307.637us; 20%=001ms307.637us; 50%=008ms736.607us; 80%=008ms922.127us; 90%=008ms922.127us; 95%=008ms922.127us; 99%=008ms922.127us
Metric: TransferToServerTime
  TotalSamples: 1
  Accumulator: 002ms847.252us
  Percentiles: 1%=002ms847.252us; 5%=002ms847.252us; 10%=002ms847.252us; 20%=002ms847.252us; 50%=002ms847.252us; 80%=002ms847.252us; 90%=002ms847.252us; 95%=002ms847.252us; 99%=002ms847.252us
Metric: TransferFromServerTime
  TotalSamples: 3
  Accumulator: 013ms067.510us
  ValueRate: 279ms381.785us / second
  Rate: 64.1396 / second
  Percentiles: 1%=850.899us; 5%=850.899us; 10%=850.899us; 20%=850.899us; 50%=005ms494.984us; 80%=007ms721.627us; 90%=007ms721.627us; 95%=007ms721.627us; 99%=007ms721.627us
Counter: CachedCompile
  Value: 1
Counter: MarkStep
  Value: 2
Counter: aten::_local_scalar_dense
  Value: 1

@JackCaoG JackCaoG changed the title [WIP]Add short_metrics Add short_metrics Nov 2, 2022
@JackCaoG JackCaoG force-pushed the short_metrics_report branch from ee594e8 to c70be32 Compare November 2, 2022 21:16
Copy link
Collaborator

@will-cromar will-cromar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise LGTM

@JackCaoG JackCaoG merged commit 1f37069 into master Nov 3, 2022
alanwaketan added a commit to pytorch/pytorch that referenced this pull request Nov 17, 2022
Summary:
This pull request upstreams pytorch/xla#4148.

Test Plan:
xla CI.
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this pull request Nov 18, 2022
Summary:
This pull request upstreams pytorch/xla#4148.

Test Plan:
xla CI.

Pull Request resolved: #89186
Approved by: https://github.com/JackCaoG
kulinseth pushed a commit to kulinseth/pytorch that referenced this pull request Dec 10, 2022
Summary:
This pull request upstreams pytorch/xla#4148.

Test Plan:
xla CI.

Pull Request resolved: pytorch#89186
Approved by: https://github.com/JackCaoG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants