[RFC] Provide mechanism to load memories directly in simulation

[chick]

Motivation

For certain classes of large Chisel designs loading memories takes a very long time.

Provide annotations to load a memory from a file

Verilog has methods $readmemb and $readmemh See here
to do this operation but there currently is no simple way to specify this in Chisel.

Strategy

Using $readmemb and $readmemh could be done directly in the emitted verilog but @jackkoenig points out that it would be better to keep direct verilog emission free of these construct and instead use verilog's bind mechanism.

Other considerations

• Simplest implementation would be to just specify a file in an annotation
• It is desirable that the file could be specified at test time (Is there a good way to do this?)
• Can we make a Chisel support of bind into a more general API that would allow users to take advantage of bind for other purposes such as adding tests and assertions
• Where possible this mechanism should work across the various backends such as the Interpreter and Treadle.

[edcote]

@jchang0 FYI

[seldridge]

I had some thoughts about this a while ago in chipsalliance/firrtl#510. Also, there was a small discussion on the chisel-users mailing list.

I'm in agreement with your first two bullet points in other considerations. Annotation for a file that gets loaded during FIRRTL compilation makes sense, but you will likely want to change this file at run time (motivating examples: a neural network accelerator with pre-loaded weights, immediately loading a microprocessor's simulated memory with a test program).

I had considered doing this with exposed VPI/DPI hooks when emitting memory structures that would allow you to, optionally, preload the memories. This only, however, aligns with a testing model using simulated/compiled Verilog.

This brings up an additional consideration:

• Do we want/need a solution that works for all possible backends? Specifically, do we need to think about a solution for Treadle and Verilator and what would that look

I had to look up bind files... I think @jackkoenig has a point here in that any auxiliary stuff should be moved over there. This brings up a tangential point of moving all the assertions (and possibly anything in an ifdef SYNTHESIS block) to a separately emitted bindfile or multiple bindfiles.

[chick]

@seldridge I think this should work for other backends where possible, this should be relatively simple to implement in the interpreter and treadle. I am less clear on what the larger playing field is like FPGA's etc. I'll update the other considerations to list this.
I think the third bullet in other considerations was meant to at least consider assertions and other blocks in to a bind section.

[seldridge]

@chick: It helps when I fully read the bullet... We're thinking the same thing here.

[chick]

@jackkoenig I'm interested in feedback on this implementation. I have the initial work done. I've created a LoadMemoryAnnotation and a Transform/Pass that finds these annotated memories. It seems like a reasonable solution would be to extend the BlackBox system to be able to mark BlackBoxInLine annotations as Bound. The firrtl verilog emitter would find these annotations and would emit the bound modules along with the bind statements necessary to link them to the original module. This extra logic would include parameter and input handling. Does this seem like a reasonable approach. Or do you have some other idea on how to proceed here.

[jackkoenig]

@chick Overall I think the implementation sounds good, just a few questions.

By extending the BlackBox system, do you mean the BlackBoxInline, etc. pass or do you mean the IR nodes?

What changes are needed in the VerilogEmitter? Just the ability to mark a module with the name of a Verilog module that should be bound to it?

What does the proposed solution do if a signal that we are trying to bind to (or rather, use as an input or output to the bound module) gets optimized away? Annotations and renaming are supposed to give us sufficient flexibility to do this linking, but it's tricky when things get optimized. For something like this I think dontTouching the requisite signals in the design is sensible so it's probably okay.

I'm wondering if we shouldn't have some way to provide post-Verilog emission hooks for information in Annotations to be used to emit stuff. In fact, an artifact system as you have proposed could perhaps ask all annotations if they want to "emit" or something like that which would allow them to emit a file and/or be deleted before we emit the remaining/resulting annotations in an anno file. This would have the added benefit of solving chipsalliance/firrtl#793. I don't think this is necessary for this PR, but they are complementary.

[chick]

@jackkoenig My initial thought was that when I would create an external module with the $readmem directive in it and a bind statement. I'm concerned that the approach is not very general, if assertions were to be added with this method I don't think this would scale well if every assertion required a separate BlackBoxInline.
So an alternate would be to have some system that allowed custom pass writers to add overriding verilog to some object that would collect it by module. I think this would require that there would be code in the verilog emitter to collect the independent augmentations for each module and then add a single bound overriding module to the verilog output. Does that sound too ambitious? I don't want to overthink this but I am leery of adding too much hacky stuff to the firrtl.

[jackkoenig]

Have you prototyped the external module with the $readmem directive? I don't think you can have vectors as ports in Verilog, although it is supported in SystemVerilog so perhaps Verilator supports it. The external module also could use a cross-module reference to refer to the mem so I'm just curious how you have/will get this to work!

I agree that we should think about the possibility of this pattern leading to tons of external modules being included via bind. For something like assertions, I think the onus would be on the assertion library to aggregate them before emitting external modules. I think for the purposes of this you shouldn't worry about it though and just try to build a robust system that could be an example of this pattern that we can examine to see how it could be replicated/scaled.

[chick]

It looks like this cannot be done without some supporting changes in Firrtl.

• Emitting a bindable module requires the ability to generate a header with the same signature
• This is currently tangled up with the verilog emitter.
• The simplest way to get the bindable modules generated would be to use BlackBoxInlineAnnos

[chick]

Issues that I could use feedback on.

1. Memories that are not a simple non-composite type are hard to handle.

• The memories are broken up into separate memories for each member.
• Ideally the specified data file could be CSV with a column per element.
  • This could get ugly with nesting.
  • The load memory transform could split the input file by column into separate input files.
  • This would seem the most natural and user friendly way of solving the problem.
• Am not planning on any support for this in the first cut unless it is thought to be critical

2. The idea of allowing run-time selection of the file seems worth deferring as well.

• The file name is currently embedded in the $readmemh statement.
• Maybe that could be parameterized but that would complicate the bindable module code generation
• Moving different files into place as the named memory input seems simpler and should be relatively easy to do with make or other tool flows

@jackkoenig @seldridge