Replace use of libdparse with dmd as a library in dfmt

[RazvanN7]

Description

dfmt is a D code formatter. It uses libdparse - a third party implementation of a D lexer+parser - to obtain a sequence of tokens and an AST and it uses this information to generate D code that respects the given guidelines. Since libdparse has a different implementation from the reference D frontend, every time a parser change occurs, libdparse and (likely) its dependencies need to be updated.

The project aims at using the dmd-as-a-library package, which shares its implementation with the reference D compiler, to substitute the use of libdparse in dfmt.

What are rough milestones of this project?

1. Create a fork of dfmt and set up a testing pipeline.
2. Use dmd-as-a-library to obtain the sequence of tokens.
3. Do the necessary changes to dmd-as-a-lib so that all the required information for step 2 is exposed.
4. Use dmd-as-a-library to obtain an AST.
5. Do the necessary changes to dmd-as-a-lib so that all the required information for step 4 is exposed.
6. Integrate the fork in the official implementation of dfmt.

How does this project help the D community?

It improves the state of the dmd-as-a-library interface which, in turn, will make it easier for tool developers to use it.

Recommended skills

Knowledgeable with AST, parsers, compilers.
Prior experience with D/C/C++/Java
OOP
Visitor pattern
Git
Tenacity, patience
Motivated and able to work independently.

Point of Contact

@RazvanN7

[snprajwal]

@RazvanN7 this project looks really interesting! I have previously worked on static analysis for Go and Rust, and have some experience with compiler frontends. I am also familiar with D from exploring deeper about its "gems" (specifically, UFCS and CTFE). I'm keen on taking this up for SAoC 2023, could you let me know how I can get started off with understanding dmd-as-a-library? I'm currently watching your talk at DConf 2017 :)

[RazvanN7]

Hi @snprajwal ! Thanks for your interest in this project. The next step would be to take a look at some examples that use dmd as a library. You can start by looking at these simple examples . Try to understand what's happening there, compile and run them. Once everything is clear you can move on to a more complex example which is an ongoing project: replacing the usage of libdparse in D-scanner. You can take a look at these PRs which try to replace libdparse bit by bit. The strategy is going to be similar for this project, so understanding what's the process there is key.

Feel free to contact me privately at razvan.nitu@gmail.com if you have any questions or if you encounter any roadblocks.

[snprajwal]

Thanks for the pointers, I'll reach out to you if I need any help!

[snprajwal]

Hi @RazvanN7, I think you might have missed an email from me. Would you be the mentor for this project for SAoC 2023? I was wondering if I should mention that in the proposal :)

[RazvanN7]

Hi @snprajwal ! I haven't received any email. I haven't seen it in my spam also. Anyway, yes, I will be your mentor, you should mention this in your proposal.

[snprajwal]

Thanks, I'll add that detail to the proposal.

I haven't received any email

That's odd, I made sure to send it to razvan.nitu@gmail.com as you mentioned earlier. Should I use the email address in your Github profile instead? (razvan.nitu1305@gmail.com)

[mdparker]

@RazvanN7 We can close this now, yes?

[RazvanN7]

Yes, closing.