Guide on customizing prediction/feedback log logger config

[parano]

Is your feature request related to a problem? Please describe.

Currently, the bentoml logging config is mostly defined here: https://github.com/bentoml/BentoML/blob/v0.8.6/bentoml/utils/log.py

It is a good default logging config but users may want to further configure it. Especially around configuring the prediction and feedback logs.

Users can already do this via the Python logging library today. e.g.

import logging
bentoml_prediction_logger = logging.getLogger('bentoml.prediction')
bentoml_prediction_logger.addHandler(my_custom_log_handler)

We should add a guide on how best practices around customizing and managing the prediction logs, as well as ways a user can set up log collection sidecar, etc

[mikedewar]

being able to manage the logging using the cli would be very helpful - we're running into a problem when we don't have write access to the local disk and it would be very handy to have all the loggers just have console handlers. It's not entirely clear how to override this from the command line.

[parano]

Hi @mikedewar - unfortunately, there isn't a way to configure that behavior in the current version of BentoML. This has been reported by quite a few users recently and it is being tracked here: #1009 It should be a fairly small code change and I will prioritize it for the next release. Thank you for reporting it!

[mikedewar]

ah I should have dug a little deeper before commenting! I appreciate the prioritisation. 🙏

[parano]

@mikedewar actually found a quick workaround for you without waiting for the next release. You can try adding the following line in your BentoService class definition file, after importing BentoML:

import logging
bentoml_file_handler = next(handler for handler in logging.getLogger('bentoml').handlers if type(handler) is logging.handlers.RotatingFileHandler)
logging.getLogger('bentoml').removeHandler(bentoml_file_handler)

We should be able to make this more easily configurable in the next release 🙏

[mikedewar]

@parano thanks for this. We're actually stuck where we're using the CLI tool e.g. bentoml serve-gunicorn ... and it's crashing in our deployment env as it can't write to the local disk. We'll figure out something to let us keep moving in the short term, and look forward to the new feature in the next release!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[parano]

This has been added by @ssheng here: https://docs.bentoml.org/en/latest/guides/logging.html