Inter-module communication

[hu55a1n1]

Modules currently own their ModuleStores and are guaranteed exclusive (write-)access to them. But a module may depend on the services of another module. Such a service is usually providing some sort of validation and reading from or writing to the ModuleStore. We need a mechanism through which a module can define how other modules can interact with it and use its services.
This is a proposal for one such system of interfaces to enable inter-module communication heavily inspired by the Cosmos SDK's Keepers concept.

Module design

A module must define the following -

• A base type that owns the Store instance and optionally implements the Module trait.

pub struct Auth<S> {
    store: SharedStore<S>,
}

• Reader traits that provide read-only access to certain Paths in the ModuleStore. Modules may define multiple Reader traits depending on the granularity of access control required.

pub trait AccountReader {
    type Error;
    type Address;
    type Account: Account;

    fn get_account(&self, address: Self::Address) -> Result<Self::Account, Self::Error>;
}

• Keeper traits that provide write access to certain Paths in the ModuleStore.

pub trait AccountKeeper {
    type Error;
    type Account: Account;

    fn set_account(&mut self, account: Self::Account) -> Result<(), Self::Error>;

    fn remove_account(&mut self, account: Self::Account) -> Result<(), Self::Error>;
}

• Types that implement the Readers and Keepers that the module wants to expose. Ideally, these types are composed out of TypedStores so they only have access to specific store Paths.

pub struct AuthAccountReader<S> {
    account_store: ProtobufStore<SharedStore<S>, AccountsPath, AuthAccount, BaseAccount>,
}

impl<S: Store> AccountReader for AuthAccountReader<S> {
    type Error = ();
    type Address = AccountId;
    type Account = AuthAccount;

    fn get_account(&self, address: Self::Address) -> Result<Self::Account, Self::Error> {
        self.account_store
            .get(Height::Pending, &AccountsPath(address))
            .ok_or(())
    }
}

• Types that implement the gRPC query server. (this is optional)

pub struct AuthQuery<S> {
    account_reader: AuthAccountReader<S>,
}

• Public API to instantiate Readers & Keepers.

impl<S: 'static + ProvableStore> Auth<S> {
    pub fn new(store: SharedStore<S>) -> Self {
        Self { store }
    }

    pub fn query(&self) -> QueryServer<AuthQuery<S>> {
        QueryServer::new(AuthQuery {
            account_reader: self.account_reader(),
        })
    }

    pub fn account_reader(&self) -> AuthAccountReader<S> {
        AuthAccountReader {
            account_store: TypedStore::new(self.store.clone()),
        }
    }

    pub fn account_keeper(&self) -> AuthAccountKeeper<S> {
        AuthAccountKeeper {
            account_store: TypedStore::new(self.store.clone()),
        }
    }
}

Dependent modules

Modules that depend on other modules must provide a ctor that indicates this dependency and must not be constructible in any other way.

pub struct Bank<S, AR, AK> {
    store: SharedStore<S>,
    account_reader: AR,
    account_keeper: AK,
    // ...
}

impl<S: ProvableStore + Default, AR: AccountReader, AK: AccountKeeper> Bank<S, AR, AK> {
    pub fn new(store: SharedStore<S>, account_reader: AR, account_keeper: AK) -> Self {
        Self {
            store: store.clone(),
            account_reader,
            account_keeper,
            // ...
        }
    }
}

App initialization

// instantiate the application with a KV store implementation of choice
let app = BaseCoinApp::new(InMemoryStore::default()).expect("Failed to init app");

// instantiate modules and setup inter-module communication (if required)
let auth = Auth::new(app.module_store(&prefix::Auth {}.identifier()));
let bank = Bank::new(
    app.module_store(&prefix::Bank {}.identifier()),
    auth.account_reader(),
    auth.account_keeper(),
);
let ibc = Ibc::new(app.module_store(&prefix::Ibc {}.identifier()));
let staking = Staking::new(app.module_store(&prefix::Staking {}.identifier()));

// register modules with the app
let app = app
    .add_module(prefix::Bank {}.identifier(), bank)
    .add_module(prefix::Bank {}.identifier(), ibc.clone());

// run the blocking ABCI server on a separate thread
let server = ServerBuilder::new(opt.read_buf_size)
    .bind(format!("{}:{}", opt.host, opt.port), app.clone())
    .unwrap();
std::thread::spawn(move || {
    server.listen().unwrap();
});

// run the gRPC server
let grpc_server = Server::builder()
    .add_service(auth.query())
    .add_service(staking.query())
    // ...
    .serve(format!("{}:{}", opt.host, opt.grpc_port).parse().unwrap());
Runtime::new()
    .unwrap()
    .block_on(async { grpc_server.await.unwrap() });

[hxrts]

There were many extended conversations about this design space / trade-offs in the SDK over the years. Few things you may want to take a look at:

• Parallelize Execution cosmos/cosmos-sdk#8134
• Do Proper Ocaps for x/bank (or not): A Case Study cosmos/cosmos-sdk#7093

If you're aiming to make this a production-grade framework I'd actually consider getting on a call with Aaron / Ethan Frey at some point to share their learnings.