Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

separate generator behavior into phases #2274

Merged
merged 19 commits into from
Feb 21, 2020
Merged

separate generator behavior into phases #2274

merged 19 commits into from
Feb 21, 2020

Conversation

debs-sifive
Copy link
Contributor

@debs-sifive debs-sifive commented Feb 3, 2020

Related issue: #2234

Type of change: feature request

Impact: API modification

Development Phase: implementation

Release Notes

This is an extension of @seldridge's work to move Rocket towards stages/phases.

I have so far tested making vsim and emulator, but I'm not sure if I've missed any other calls to the generator...

TODO

  • code cleanup — should I strip out the old generator entirely? yes
    • reparameterize path to generator in makefiles
    • clean up test suite generation
  • extending ChiselStage vs calling ChiselStage? extend
    • discuss: creating a RunPhaseAnnotation?
    • wait for refactoring of Chisel/Firrtl stages
  • solicit outside feedback — is it ok to change the generator command line? yes
    • fix config delimiting
    • merge package + module names
    • use firrtl target dir anno
  • fix dependencies

Copy link
Member

@seldridge seldridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a clean port. Thanks for picking up where I left off @debs-sifive!

High level comments

I expect the PR can go further with the stage/phase conversion (e.g., not using any Driver methods), but I like this as a first step.

The dependencies generally look fine. I do tend to include all prerequisites. E.g., GenerateFirrtl has only Elaborate as a prerequisite. However, it is doing blind .gets on a view[RocketChipOptions]. Users may use GenerateFirrtl in a separate PhaseManager and that phase can throw None.get exceptions if Checks hasn't run. It makes the dependency specification more verbose, but should make them bulletproof for whatever arbitrary composition users want to do.

Some questions

  1. Does this PR want to keep the existing Rocket approach of separating the config package from the config name and the top-level package from the top-level name?
  2. I think the PR's approach of just hard-coding the default rocket test suites makes sense. However, have you thought about how to wrap that up in a RocketTestSuiteAnnotation(tests: Seq[RocketTestSuite])?

Answers to @debs-sifive Questions

code cleanup — should I strip out the old generator entirely?

Rocket Chip doesn't have the same stringent requirements on backwards compatibility as Chisel/FIRRTL. Still, I'd try to notify stakeholders and definitely consider doing this in a subsequent PR.

extending ChiselStage vs calling ChiselStage?

I'd tend towards calling it vs. extending it (or defining at new PhaseManager).

solicit outside feedback — is it ok to change the generator command line?

Good thought. I think the Generator API is more problematic than the command line as I anticipate most Rocket Chip users are just using the makefiles directly. Rocket Chip was never published as a fat jar (though it could and probably should be).

src/main/scala/stage/phases/Checks.scala Outdated Show resolved Hide resolved
src/main/scala/stage/RocketChipAnnotations.scala Outdated Show resolved Hide resolved
src/main/scala/stage/RocketChipAnnotations.scala Outdated Show resolved Hide resolved
src/main/scala/stage/phases/RocketChipStageUtils.scala Outdated Show resolved Hide resolved
src/main/scala/stage/phases/PreElaboration.scala Outdated Show resolved Hide resolved
@debs-sifive
Copy link
Contributor Author

Thanks for all the feedback, @seldridge!

Does this PR want to keep the existing Rocket approach of separating the config package from the config name and the top-level package from the top-level name?

I was trying to adhere to the old CLI as closely as possible, but it seems the consensus is that we can scrap the old CLI. Given this, I think the way you created class instead of string annotations in #2234 is the cleanest way of doing things, so I will try to go back to that.

I think the PR's approach of just hard-coding the default rocket test suites makes sense. However, have you thought about how to wrap that up in a RocketTestSuiteAnnotation(tests: Seq[RocketTestSuite])?

They were hardcoded in the old generator, and I just copy/pasted the function 😬 . I will try out your suggestion.

There's actually a bunch of stuff from the generator/generator utils that I copied into different files to avoid dependency issues, and I will start to clean them up... First step is probably removing the old generator entirely, since that has been okayed in ucb-bar/chipyard#424.

@debs-sifive debs-sifive marked this pull request as ready for review February 13, 2020 18:48
@debs-sifive
Copy link
Contributor Author

@seldridge not sure why I can't request review from you, but would you mind taking another glance?

@seldridge
Copy link
Member

Sure @debs-sifive. (I'm not a reviewer for rocket-chip, so I can't be requested.) Can provide feedback, though.

@debs-sifive
Copy link
Contributor Author

debs-sifive commented Feb 18, 2020

have you thought about how to wrap that up in a RocketTestSuiteAnnotation(tests: Seq[RocketTestSuite])?

@seldridge sorry to keep bothering you, but I am not sure of how to go about this cleanly.

I initially tried separating out the test generation into another phase that created RocketTestSuiteAnnotations, but it looks like there are actually two separate test generators in use — one for the rocketchip.unittest project and one for rocketchip.system. So I broke the unit test generation with my approach.

Do you have an idea of what the ideal RocketTestSuiteAnnotation generation should be like? I can think of two options that would work:

  1. Check the project to determine which test generator to use. This will be brittle if there are other projects added on (not sure how likely this is). [EDIT: I think this option may also require hardcoding the project name into the test generator, so that is not great.]
  2. Take in the list of tests to run from an external file. This gives the test runner the greatest flexibility if they just want to run some subset of tests. But I am also not sure this approach makes sense in the context of the generator b/c the external file would likely have to be hardcoded or externally generated.

@seldridge
Copy link
Member

External file seems like a bad API. I think what we want here is two custom phases that are dynamically added to the list of targets. So, there's a default Rocket Chip Stage and then if you're doing a normal build, you add the RocketTestSuitesPhase and if you are doing unittests you add the RocketUnitTestsSuites phase.

Copy link
Contributor

@azidar azidar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Please just add Scaladoc comments to the classes in RocketChipAnnotations.scala, then I'm happy to merge.

@azidar
Copy link
Contributor

azidar commented Feb 20, 2020

@seldridge I think a better API would be to write something like an aspect which introspects on the design and generates the test collateral (and presumably consuming additional annotations for customization). However, IMO its out of scope of this PR so I'm personally happy as is. What do you think?

Copy link
Member

@seldridge seldridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Other future work would be to pull in the new Dependency API (part of the broader move to 3.3).

src/main/scala/stage/RocketChipOptions.scala Outdated Show resolved Hide resolved
@seldridge seldridge mentioned this pull request Feb 20, 2020
@debs-sifive debs-sifive merged commit 9d99750 into master Feb 21, 2020
@sequencer
Copy link
Member

Build failed with latest:
chipsalliance/chisel@403cd5b
chipsalliance/firrtl@c127444

maybe forget to bump firrtl into master?

[info] Compiling 18 Scala sources to .../rocket-playground/out/rocketchip/compile/dest/classes ...
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:15:12: type mismatch;
[error]  found   : Class[freechips.rocketchip.stage.phases.Checks](classOf[freechips.rocketchip.stage.phases.Checks])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[freechips.rocketchip.stage.phases.Checks],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:16:12: type mismatch;
[error]  found   : Class[freechips.rocketchip.stage.phases.PreElaboration](classOf[freechips.rocketchip.stage.phases.PreElaboration])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[freechips.rocketchip.stage.phases.PreElaboration],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:17:12: type mismatch;
[error]  found   : Class[chisel3.stage.phases.Checks](classOf[chisel3.stage.phases.Checks])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[chisel3.stage.phases.Checks],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:18:12: type mismatch;
[error]  found   : Class[chisel3.stage.phases.Elaborate](classOf[chisel3.stage.phases.Elaborate])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[chisel3.stage.phases.Elaborate],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:19:12: type mismatch;
[error]  found   : Class[chisel3.stage.phases.MaybeAspectPhase](classOf[chisel3.stage.phases.MaybeAspectPhase])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[chisel3.stage.phases.MaybeAspectPhase],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:20:12: type mismatch;
[error]  found   : Class[freechips.rocketchip.stage.phases.GenerateFirrtl](classOf[freechips.rocketchip.stage.phases.GenerateFirrtl])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[freechips.rocketchip.stage.phases.GenerateFirrtl],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:21:12: type mismatch;
[error]  found   : Class[freechips.rocketchip.stage.phases.GenerateFirrtlAnnos](classOf[freechips.rocketchip.stage.phases.GenerateFirrtlAnnos])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[freechips.rocketchip.stage.phases.GenerateFirrtlAnnos],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:22:12: type mismatch;
[error]  found   : Class[freechips.rocketchip.stage.phases.AddDefaultTests](classOf[freechips.rocketchip.stage.phases.AddDefaultTests])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[freechips.rocketchip.stage.phases.AddDefaultTests],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:23:12: type mismatch;
[error]  found   : Class[freechips.rocketchip.stage.phases.GenerateTestSuiteMakefrags](classOf[freechips.rocketchip.stage.phases.GenerateTestSuiteMakefrags])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[freechips.rocketchip.stage.phases.GenerateTestSuiteMakefrags],
[error]            ^
[error] .../rocket-playground/rocketchip/src/main/scala/system/RocketChipStageGenerator.scala:24:12: type mismatch;
[error]  found   : Class[freechips.rocketchip.stage.phases.GenerateROMs](classOf[freechips.rocketchip.stage.phases.GenerateROMs])
[error]  required: firrtl.options.PhaseManager.PhaseDependency
[error]     (which expands to)  firrtl.options.Dependency[firrtl.options.Phase]
[error]     classOf[freechips.rocketchip.stage.phases.GenerateROMs],
[error]            ^
[error] 11 errors found
1 targets failed

@seldridge
Copy link
Member

You'll need to wait for the actual Chisel3/FIRRTL bumps. There are API changes to the Dependency API that will land in 3.3/1.3 that will need to migrate into Rocket Chip. These changes are merged in Chisel3/FIRRTL master currently, but not into rocket as there are other accumulated changes in Chisel3/FIRRTL that need to dealt with in rocket.

Sorry for the delay, but all the dependency stuff is almost there and almost fully load-bearing.

@sequencer
Copy link
Member

yes, these commits are current chisel/firrtl master branch, but still failed to compile, I'm not sure which commits are required by this change

@debs-sifive debs-sifive deleted the staged-generator branch February 26, 2020 21:45
debs-sifive added a commit that referenced this pull request Mar 10, 2020
debs-sifive added a commit that referenced this pull request Mar 10, 2020
* Revert "updating README examples with Make changes (#2322)"

This reverts commit 875dcd3.

* Revert "fixing aspect generation (#2309)"

This reverts commit bbeb257.

* Revert "separate generator behavior into phases (#2274)"

This reverts commit 9d99750.
debs-sifive added a commit that referenced this pull request Mar 12, 2020
ucbjrl added a commit that referenced this pull request Apr 21, 2020
Revert staged generator PRs (#2329)

* Revert "updating README examples with Make changes (#2322)"

This reverts commit 875dcd3.

* Revert "fixing aspect generation (#2309)"

This reverts commit bbeb257.

* Revert "separate generator behavior into phases (#2274)"

This reverts commit 9d99750.

(cherry picked from commit e170f8f)
@AMing922
Copy link

You'll need to wait for the actual Chisel3/FIRRTL bumps. There are API changes to the Dependency API that will land in 3.3/1.3 that will need to migrate into Rocket Chip. These changes are merged in Chisel3/FIRRTL master currently, but not into rocket as there are other accumulated changes in Chisel3/FIRRTL that need to dealt with in rocket.

Sorry for the delay, but all the dependency stuff is almost there and almost fully load-bearing.

Excuse me, did these questions sloved?

@debs-sifive
Copy link
Contributor Author

Master of rocket is still along 3.2 and 1.2 of Chisel/FIRRTL. Not sure who has a good sense of when these will be bumped, perhaps @azidar or @seldridge?

@seldridge
Copy link
Member

@jackkoenig was working on bumping things here: #2399.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants