SDK Scaffolding Tools

[okwme]

Summary

Scaffolding tools would make building on the SDK faster and easier. It would also be a good way to establish best practices. I've outlined what I might expect as a developer coming from a background with a MVC framework like RoR or CakePHP.

Proposal

Introduction

Scaffolding with the sdk should work like ruby on rails (to be honest i'm more familiar with cakephp which was based on RoR and contained very good scaffolding features). RoR was an example of a Model View Controller (MVC) framework. The scaffolding software available helped to quickly create the basic files needed to run an app and to fill them with enough code as to allow simple Create, Read, Update, Delete (CRUD) commads as API endpoints and a GUI website.

This was done by first defining a series of Models, which represent the data types being used and their relationships to one another. Using these Models it was possible to create Controllers for some or all of the Models. These Controllers had code that was responsible for processing requests in order to retrieve and update data. The scaffolding software also took care of very basic html Views that were designed to compensate the endpoints made available by the Controllers that were built. Common Models, like users, groups and posts, could be used to shape common utilities like Access Control Lists (ACL)

Typically the scaffolding process took place from the command line. You'd begin by saying you'd like to create some models. The CLI would begin asking questions about what fields you wanted in the model. This model would represent a database table with one typed column per field. Field names would be in camel case and whenever one field represented the key of another table it would be types like tableName_id. This made it possible for the scaffolding software to build proper join tables and relationships between models. After asking for all fields and types of a table the CLI would ask to confirm the relationships contained therein. They would infer these from the use of _id suffixes or the present of other models with fields containing the name of the current model.

SDK

With regard to the SDK while building a new module, the equivalent to the Model in a MVC framework is the Type. There will be a series of types that need to be created for a new Module and they should contain information that makes it possible to generate keepers and handlers for them. Types may be able to be divided into two categories: Iterable and Non-Iterable.

Iterable

An iterable type would be closer to a database table. It would either be indexed and contain some UID and act like a mapping. Or it would not be indexed and act more like an array. This Iterable type could be a single value like a uint, stringor bool as well as a more complicated value like a struct containing it's own assortment of types. If it was indexed, the index could be used as part of the key in the kvstore, if it is not indexed, the kvstore could use a generic prefix. Similar to joined tables it may be useful to allow one iterable type reference another indexed and iterable type through a naming sceme like typeName_id.

Non-Iterable

A Non-Iterable Type would be closer to a static value or static group of values that are defined within the module. These could be something like global variables or settings as well as values which are updated by Certain Messages.

CRUD

While the utility of a boierplate CRUD app is often far from what an architect intends in the long run, it is nonetheless a useful basis on which to build. I suggest that the keepers, handlers and messages that are scaffolded all work towards creating the ability to Create, Read, Update and Delete data from the iterable and non-iterable types that are present in the app. This includes REST endpoints and a CLI that is ready to faccilitate in these operations.

Other Modules

Since SDK apps rely heavily on modules that already exist it should be possible at a minimum to include as many available modules as needed. The basic code for importing and making them available should also be included in the app.go file. Maybe the new Module type being worked on at #4033 would make this process easier.

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[alexanderbez]

Thanks for taking the time and writing this up @okwme. I believe that having a RoR-like scaffolding tool for modules in particular will be especially useful as the ecosystem grows. Looking at RoR as a prime example, it's very easy to see how such a tool can grow and become feature-dense, so I do think our first go at it should be a MVP. Namely, the following:

Having the following structure and content automatically generated:

├── alias.go
├── app.go
├── client
│   ├── cli
│   └── rest
├── genesis.go
├── handler.go
├── keeper
├── querier
├── simulation
└── types

Each file in this scaffold would have some commented and/or boilerplate code.

Now, I wonder if the MVP should simply start off with boilerplate to lead-by-example or if we should have user input like RoR (models). The latter would be more complicated.

[fedekunze]

I think a boilerplate should suffice. Dealing with user inputs would be extremely complicated specially to deal with properties' type.

[okwme]

Having a boilerplate module would be amazing. I'd say this would be a different kind of product from scaffolding tho. I'm thinking about the https://github.com/facebook/create-react-app, https://github.com/vuejs/vue-cli or truffle-boxes as starter kits that don't actually scaffold content, just produce boilerplate with some variation on templates. The first version could just be one static module with no options and we could add various templates or interactive options from there.

I do still think it's interesting to have scaffolding software as well though. In the hopes of starting simple maybe we could focus on what is the most tedious or easiest to automate process in the build? I might suggest building the clients as that candidate? If a module has well designed messages and queriers it should be possible to automate a bunch of CLI commands and REST endpoints, right? Or maybe better yet to take one step backwards and just convert the CLI commands to REST endpoints? Maybe these functions could work backwards until it got to the point of fully automated interactive building of modules via definitions of Types and Msgs?

Does anyone oppose me making a new issue for a boilerplate module to separate from scaffolding?

[alexanderbez]

Imho @okwme it should be really simple and minimal, ie. generate/scaffold a module directory structure with boilerplate/templated code.

[fedekunze]

Does anyone oppose me making a new issue for a boilerplate module to separate from scaffolding?

I think it should be separate yes, we can start with the boilerplate, eg:

// create general boilerplate
cosmos-sdk init <app_name> <imported_module_1> ... <imported_module_n>

// add custom module
cosmos-sdk add-module <module_name>

[rigelrozanski]

We actually used to have some scaffolding tools be we removed them in favour of 3rd party scaffolding tools which were being build outside of the sdk (CC @jackzampolin).

I think for the time being gaia already acts as more-or-less a boilerplate (there is only a tiny bit of extra stuff in there, such a testnet command). Currently there isn't too too many options for your blockchain besides just copying gaia and then modifying the module functionality as desired/adding new modules. I think however, as the module collection grows in the SDK, and as the modules are abstracted in a better way, there may come a future point when users could want to choose between multiple versions of modules presented... This is really cool and scaffolding tooling would support this as you've described.

It would be nice to somehow build all of this scaffolding connectability properties into the module pattern interface (just merged) such that modules would effectively be able to chart each others compatibility with each other as well as the traits they provide to a blockchain (maybe like the scaffolding CLI "questions" which were proposed earlier). - This would allow for the scaffolding tooling to pretty much run simply off of a list of modules, without really needing to understand how they all connect together and what they individually mean, as that would be implicitly handled by the module pattern.

[shanev]

Happy to integrate some of these ideas into https://github.com/shanev/cosmos-gen and upstream it.

[tac0turtle]

scaffold and starport has completed this issue. closing this here