It's an add-in for the VBA IDE, the glorious Visual Basic Editor (VBE) - which hasn't seen an update in this century, but that's still in use everywhere around the world. Rubberduck wants to give its users access to features you would find in the VBE if it had kept up with the features of Visual Studio and other IDE's in the past, oh, decade or so.
Rubberduck wants to help its users write better, cleaner, maintainable code. The many code inspections and refactoring tools help harmlessly making changes to the code, and unit testing helps writing a safety net that makes it easy to know exactly what broke when you made that small little harmless modification.
Rubberduck wants to bring VBA into the 21st century, and wants to see more open-source VBA repositories on GitHub - VBA code and source control don't traditionally exactly work hand in hand; unless you've automated it, exporting each module one by one to your local repository, fetching the remote changes, re-importing every module one by one back into the project, ...is a little bit tedious. Rubberduck reduces this tedium by allowing you to export all modules from your project to wherever you have your source control set up.
If you're learning VBA, Rubberduck can help you avoid a few common beginner mistakes, and can probably show you a trick or two - even if you're only ever writing macros. If you're a more advanced programmer, you will appreciate the richness of Rubberduck's feature set. See the Installing wiki page.
If you're a C# developer looking for a fun project to contribute to, see the Contributing wiki page.