Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mongo sink improvements #2519

Merged
merged 8 commits into from
Oct 19, 2022
Merged

Mongo sink improvements #2519

merged 8 commits into from
Oct 19, 2022

Conversation

sandorzm
Copy link

When using the Mongo sink in an application that also uses MongoCXX separately, there's the question of who should own the mongocxx::instance object because exactly one may exist in a program, and its lifetime must enclose the lifetimes of all other MongoCXX objects in the program (MongoCXX docs). Currently, mongo_sink has a static field that owns the instance, causing an exception when any other code in the app tries to construct an instance.

I think it makes more sense for the main application to own it for these reasons:

  1. The Mongo sink never does anything with it, but the application might have to access or configure it. From MongoCXX's instance management example:

    real programs will typically require shared access to a commonly configured instance and connection pool across different translation units and components.

    This is impossible if mongo_sink owns the instance.

  2. If the app was already using MongoCXX before starting to use Spdlog or mongo_sink, it already has code to construct and manage the MongoCXX instance, and that code would now break because an instance was already constructed by mongo_sink.

I implemented this by making a non-breaking change that makes mongo_sink optionally own the instance, only not owning it if one already exists or if an additional optional argument to the constructor is passed. It's still as convenient to use mongo_sink if not managing a MongoCXX instance separately, but it's also possible to prevent mongo_sink from creating one. Let me know if there's a better way.

A couple other small changes: querying logs by a certain log level or above (e.g., warn, error, critical or debug, warn, error, critical) is a common use case, and this is made much easier by putting the numerical log level in MongoDB along with the string log level. Also removed a G++ warning about catching an exception by value.

Sandor Magyar added 3 commits October 17, 2022 16:04
Filtering to a certain log level or above, a useful operation, can now be done
with an integer comparison as opposed to comparing to a list of strings in the
database query.
There can only be one instance in the whole program, so programs that use the
Mongo sink and also separately use MongoCXX may have problems if the Mongo sink
owns the instance. MongoCXX recommends that the main application manage its own
instance so configuration parameters can be passed to the constructor:
http://mongocxx.org/api/current/classmongocxx_1_1instance.html

However, this commit is not a breaking change. If no instance has been created
at construction time, the Mongo sink will still create and own the instance.
@gabime
Copy link
Owner

gabime commented Oct 18, 2022

I think a prefferable approach would be to have new ctor overload with a shared_ptr param and keep the orig API. something like

mongo_sink(std::shared_ptr<mongocxx::instance> instance, const std::string &db_name, const std::string &collection_name, const std::string &uri): 
   instance_(instance), 
   db_name_(db_name),
   collection_name_(collection_name)
{
    client_ = spdlog::details::make_unique<mongocxx::client>(mongocxx::uri{uri});
}

This is would also ensure that the instance doesn't get destroyed while the sink holds it.
Then the orig ctor could just create the instance and pass it to this new ctor.
Also, the instance_ better not be static now since it is not thread safe to copy same shared_ptr concurrently, and there is no good reason anyway. (to create a singleton instance its better to use a function with static member).

@sandorzm
Copy link
Author

Yeah, good point. I split the constructor into two, one creating the instance and delegating to the other. If the creation of the instance throws an exception, I just let that propagate because the constructor that creates the instance should not be called if one already exists.

include/spdlog/sinks/mongo_sink.h Outdated Show resolved Hide resolved
}
catch (const std::exception)
catch (const std::exception&)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding a catch clause to catch the mongo exception to extract its what() string and add to the spdlog_ex message as well ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the what() string to the spdlog_ex in the std::exception catch clause, which should also catch mongocxx::exception. Did you specifically want a separate catch clause for mongocxx::exception?

Also caught the mongocxx::exception that may be triggered by constructing the mongocxx::instance in the first constructor overload with a function try block, but other std::exceptions are not handled in this overload and would be re-thrown as-is. Is it better to catch all std::exceptions and re-throw them as spdlog_ex?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. but since the first ctor just forwards to the other ctor, the try catch is redundant in the first ctor anyway

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But can't the try-catch in the first constructor catch exceptions from the std::make_shared<mongocxx::instance>() that appears in its initializer list while the second can't?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm mistaken, I'll change the first constructor's try-catch to catch any std;:exception and re-throw as spdlog_ex.

Copy link
Owner

@gabime gabime Oct 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake, you are right. Merging..

}
catch (const std::exception)
catch (const std::exception&)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. but since the first ctor just forwards to the other ctor, the try catch is redundant in the first ctor anyway

@gabime gabime merged commit bced424 into gabime:v1.x Oct 19, 2022
@gabime
Copy link
Owner

gabime commented Oct 19, 2022

Thanks @sandorzm

@sandorzm
Copy link
Author

Thank you for Spdlog, @gabime. I made this PR because my team wants to use the Mongo sink, and we would prefer to incorporate it with this PR's changes in the next tagged release of Spdlog (otherwise, we'll just use a recent commit from the main branch). Do you have a plan for the next release?

@gabime
Copy link
Owner

gabime commented Oct 19, 2022

There is still work to be done before next release (bump fmt to 9.1.0) so might take a while. Taking the latest commit sounds fine to me anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants