Skip to content

Commit

Permalink
Apply isort and black reformatting
Browse files Browse the repository at this point in the history
Signed-off-by: guyueh1 <guyueh1@users.noreply.github.com>
  • Loading branch information
guyueh1 committed Nov 30, 2024
1 parent 4ba23e4 commit 08ac16c
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions nemo/collections/common/metrics/perf_metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -254,15 +254,15 @@ def _grok(self):
vocab_size = LLM_VOCAB_SIZE_MAP["grok"]
return (
self.layers
* self.gbs
* self.gbs
* self.enc_seq_len
* self.hs
* self.hs
* self.hs
* self.hs
* (
12
+ (12 * self.query_groups / self.attention_heads)
+ (18 * self.moe_router_topk * self.ffn_hs / self.hs)
+ (12 * self.enc_seq_len / self.hs)
+ (6 * vocab_size / (self.layers * self.hs))
)
)
)

0 comments on commit 08ac16c

Please sign in to comment.