Implement Solidity AST printer

[axic]

This would take the internal AST structure and return a "Solidity source code output" which can be parsed by the parser.

This can also be used by the "import AST from JSON" feature (see #2096) to create the hash of the sourcecode that is part of the onchain metadata.

[chriseth]

What about just including the hash of the source code into the AST?

[yxliang01]

May I know how this feature is going? I think this would be useful. I realize that there are some source code files look related.

[yxliang01]

I think this is implemented already. The AST printer can be used via CLI argument like --ast-json.

[chriseth]

@axic could you clarify the meaning of this issue, please?

[axic]

It says it in the description:

To print parseable Solidity source code.

[yxliang01]

@axic Did you mean being able to convert AST to Solidity source code?

[axic]

Did you mean being able to convert AST to Solidity source code?

Yes.

[tharindu1992]

Any update here? I think this is very useful.

[tharindu1992]

@axic Did you mean being able to convert AST to Solidity source code?

Any updates? I am trying to do this.

[cameel]

Reposting from #5802 (which has been closed as a duplicate):

@ryanfalzon

Has there been any updates on this? I need to make use of this feature.

@cameel

No, the compiler can parse an AST from JSON but currently does not offer a feature to synthesize code back from that. There are external tools that can do that though. Check out solc-typed-ast.

[cameel]

@axic @chriseth Given that there are now tools that can do this, are we still planning to implement this?

Also pinging @blitz-1306, who might be interested in this topic.

[blitz-1306]

Hello everyone. As I've been referenced here, I might share a few thoughts how this is done in solc-typed-ast. The most of the logic is concentrated in following directory: https://github.com/ConsenSys/solc-typed-ast/tree/master/src/ast/writing (mostly in ast_mapping.ts and yul_mapping.ts).

Writing AST to source code is mostly trivial task, aside from few considerations:

• Version compatibility - produced source code may differ for various compiler versions. We slightly (not completely) compensate that by allowing to pass target compiler version and checking if we need to adjust output to fit into requirements in the most cases.
• Source mapping - for some of dependent packages like Scribble, there is a need to map produced source code fragments back to AST nodes. We produce complete map node -> string, but I guess there should be a way to have a workaround with nodeId -> string (as this is still mappable back and node ids are unique within the tree).
• Indentation / Whitespace - AST representation removes any information about whitespace and indentation. First minimal thing is to separate identifiers by at least a single whitespace. For better readability we introduced simplified formatting components that allows to regulate indentation. It may also worth to postpone such functionality to linters.

For us, AST-to-source conversion been beneficial for the following reasons:

• Testing AST and detecting bugs like
  • The "kind" property of ModifierInvocation AST node is not indicating baseConstructorSpecifier #11196
  • Distinct YulHexStringLiterals from common Yul string literals while reproducing the source code #11324
  • SolcJS 0.7.0 JSON AST output is not following Solidity 0.7.0 release notes #9552
  • ImportDirective symbolAliases are not pointing to external nodes #7758
• Tree/Source modification support allowing tools like Scribble to exist. So I guess community will benefit from getting the feature.

Ofc, I do not think that all of the points above are related to the current scope of "just writing source from an AST". My intention here is to share some experience and ideas, also indicate some caveats of the feature alongside with possible benefits.

Regards.

[blitz-1306]

Pinging @cd1m0, who is actively developing Scribble at the moment (in case if there would be any additional input).

[github-actions]

This issue has been marked as stale due to inactivity for the last 90 days.
It will be automatically closed in 7 days.

[ekpyron]

Just seeing the stale notification here and will even manually close this, since we won't be doing this ourselves - however, in case this pops up for anyone, I recently had a call with Certora about them having released an open source tool that does just this even with potential AST mutations. Without spending much time on looking it up, this may have been it: https://github.com/Certora/gambit or a related repo there.