Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Motor -> Async PyMongo #1113

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Motor -> Async PyMongo #1113

wants to merge 2 commits into from

Conversation

roman-right
Copy link
Member

Switch from Motor to Async PyMongo

Possible breaking change: Some internal fields and method names have been updated to remove references to Motor.

@dantetemplar
Copy link
Contributor

Why do you want to migrate?

@roman-right
Copy link
Member Author

Hi @dantetemplar ,
The performance has improved. It will not be merged into the main branch until they confirm that it is ready for production. However, the way they implemented it is better. Here is the explanation: https://www.mongodb.com/docs/languages/python/pymongo-driver/current/motor-async-migration/

Copy link
Contributor

@staticxterm staticxterm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, great to see you are back and to see these changes!
I also tried making the same changes to see how much work it is, but then got stuck on the "cursor" changes...

Apologies in advance for the number of review comments. Please note that I didn't test this out on some production code, only have done a visual inspection of the code. I plan to test this out with the proposed changes.

PyMongo Async is still in Beta according to MongoDB official documentation, so I propose to wait for an official release as well, and as you already stated.
Also, these changes should result in a next Major version release of Beanie due to changes to Beanie API.

@@ -15,7 +15,6 @@ To set the root model you have to set `is_root = True` in the inner Settings cla

```py hl_lines="20 20"
from typing import Optional, List
from motor.motor_asyncio import AsyncIOMotorClient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing import for AsyncMongoClient:

from pymongo import AsyncMongoClient

@@ -163,7 +164,7 @@ async def run_migration_class(
db = DBHandler.get_db()
if client is None:
raise RuntimeError("client must not be None")
async with await client.start_session() as s:
async with client.start_session() as s:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
async with client.start_session() as s:
with client.start_session() as s:

According to PyMongo documentation, a simple "with" context manager is used.

@@ -118,7 +118,7 @@ async def commit(self) -> Optional[BulkWriteResult]:
raise ValueError(
"The document model class must be specified before committing operations."
)
return await self.object_class.get_motor_collection().bulk_write(
return await self.object_class.get_pymongo_collection().bulk_write(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a breaking change in itself (renaming the public interface method) should this simply be renamed to get_collection()?

@@ -640,7 +640,7 @@ def list_to_index_model(left: List[IndexModelField]):
return [index.index for index in left]

@classmethod
def from_motor_index_information(cls, index_info: dict):
def from_pymongo_index_information(cls, index_info: dict):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, should we drop "motor"/"pymongo" from method name and simply rename to from_index_information() ?

@@ -12,8 +12,8 @@ def get_settings(cls) -> ItemSettings:
pass

@classmethod
def get_motor_collection(cls) -> AsyncIOMotorCollection:
return cls.get_settings().motor_collection
def get_pymongo_collection(cls) -> AsyncCollection:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a breaking change, could it be renamed to simply get_collection()?

@@ -193,8 +193,8 @@ async def test_index_dropping_is_not_allowed(db):
allow_index_dropping=False,
)

collection: AsyncIOMotorCollection = (
DocumentTestModelWithComplexIndex.get_motor_collection()
collection: AsyncCollection = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
collection: AsyncCollection = (
collection = (

@@ -217,8 +217,8 @@ async def test_index_dropping_is_not_allowed_as_default(db):
document_models=[DocumentTestModelWithDroppedIndex],
)

collection: AsyncIOMotorCollection = (
DocumentTestModelWithComplexIndex.get_motor_collection()
collection: AsyncCollection = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
collection: AsyncCollection = (
collection = (

@@ -397,6 +397,6 @@ class Settings:
# To force collection creation
await NewDocument(test_str="Roman Right").save()

collection: AsyncIOMotorCollection = NewDocument.get_motor_collection()
collection: AsyncCollection = NewDocument.get_pymongo_collection()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
collection: AsyncCollection = NewDocument.get_pymongo_collection()
collection = NewDocument.get_pymongo_collection()

@@ -206,7 +206,7 @@ class Config:

def __init__(self, *args: Any, **kwargs: Any) -> None:
super(Document, self).__init__(*args, **kwargs)
self.get_motor_collection()
self.get_pymongo_collection()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line feels out of place. It does nothing.

connection_string
).get_default_database()

self.database: AsyncIOMotorDatabase = database
self.database: AsyncDatabase = database

if multiprocessing_mode:
self.database.client.get_io_loop = asyncio.get_running_loop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no method "get_io_loop" on the "client"/AsyncMongoClient anymore. This is another breaking change.
Also, is there any documentation on what this "multiprocessing_mode" parameter is, and how can it be used?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants