Indexer core

[bogdanfazakas]

Fixes #21 .

Changes proposed in this PR:

• create config and take list of : RPC,chainId,chunkSize from env for every network,
  need to add chunkSize
• for each network, start a working thread (use nodejs Working Threads)
• should detect the block used to deploy the contracts from addresses.json
• should store the last indexed block. if none, use above
  need db interaction to save the last indexed block and get the last indexed block
• check network height, if > than last indexed block, process remaining blocks in chunks. If something fails, reduce chunk size to 1/2 and retry
• based on event, use a specific processor
• if something fails or processing of an event takes more that than a couple of secs, insert that event to a retry queue and move to next one

[paulo-ocean]

Hi @bogdanfazakas
Just a couple of suggestions as i think you could make a little bit better use of our custom logger :-)
for instance, when you create the custom Logger you could specify the module as 'Indexer, rather than 'Database'

export const INDEXER_LOGGER: CustomNodeLogger = getCustomLoggerForModule(
  LOGGER_MODULE_NAMES.DATABASE, // INDEXER
  LOG_LEVELS_STR.LEVEL_INFO,
  defaultConsoleTransport
)

And when you're logging, you could set the log level directly by calling 'log(level,message,includeModuleName)' // include module name
instead of always setting log level before and after the message, like:

INDEXER_LOGGER.loggerOptions.level = LOG_LEVELS_STR.LEVEl_ERROR
INDEXER_LOGGER.logMessage('Error retrieving last indexed block')
INDEXER_LOGGER.loggerOptions.level = LOG_LEVELS_STR.LEVEL_INFO

thanks 👍

[bogdanfazakas]

Hi @bogdanfazakas Just a couple of suggestions as i think you could make a little bit better use of our custom logger :-) for instance, when you create the custom Logger you could specify the module as 'Indexer, rather than 'Database'

export const INDEXER_LOGGER: CustomNodeLogger = getCustomLoggerForModule(
  LOGGER_MODULE_NAMES.DATABASE, // INDEXER
  LOG_LEVELS_STR.LEVEL_INFO,
  defaultConsoleTransport
)

And when you're logging, you could set the log level directly by calling 'log(level,message,includeModuleName)' // include module name instead of always setting log level before and after the message, like:

INDEXER_LOGGER.loggerOptions.level = LOG_LEVELS_STR.LEVEl_ERROR
INDEXER_LOGGER.logMessage('Error retrieving last indexed block')
INDEXER_LOGGER.loggerOptions.level = LOG_LEVELS_STR.LEVEL_INFO

thanks 👍

thanks for the hints, will update

[paulo-ocean]

alright 👍