Skip to content

Commit 3158169

Browse files
authored
fix: uninstall bson for mongo connector (#3104)
### Summary Closes #3049. Reenables the MongoDB connector test, which was disabled previously in #3047 due to incompatibility between the `pymongo` and the `bson` package from `pip`, which is a dependency for the Astra connector. Per the `pymongo` docs below, `pymongo` ships with its own version of `bson` and installing `bson` from `pip` breaks `pymongo`. - https://pymongo.readthedocs.io/en/stable/installation.html ### Testing Ingest tests ran successfully for the [source connector](https://github.com/Unstructured-IO/unstructured/actions/runs/9273154676/job/25512636315) and the [destination connector](https://github.com/Unstructured-IO/unstructured/actions/runs/9273154676/job/25512635546).
1 parent 6b400b4 commit 3158169

File tree

4 files changed

+16
-10
lines changed

4 files changed

+16
-10
lines changed

test_unstructured_ingest/dest/mongodb.sh

+7
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,13 @@ function cleanup() {
3333

3434
trap cleanup EXIT
3535

36+
# NOTE(robinson) - per pymongo docs, pymongo ships with its own version of the bson library,
37+
# which is incompatible with the bson installed from pypi. bson is installed as part of the
38+
# astra dependencies.
39+
# ref: https://pymongo.readthedocs.io/en/stable/installation.html
40+
pip uninstall -y bson pymongo
41+
make install-ingest-mongodb
42+
3643
python "$SCRIPT_DIR"/python/test-ingest-mongodb.py \
3744
--uri "$MONGODB_URI" \
3845
--database "$MONGODB_DATABASE_NAME" \

test_unstructured_ingest/src/mongodb.sh

+6-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,12 @@ if [ -z "$MONGODB_URI" ] && [ -z "$MONGODB_DATABASE_NAME" ]; then
2020
exit 8
2121
fi
2222

23-
# trap cleanup EXIT
23+
# NOTE(robinson) - per pymongo docs, pymongo ships with its own version of the bson library,
24+
# which is incompatible with the bson installed from pypi. bson is installed as part of the
25+
# astra dependencies.
26+
# ref: https://pymongo.readthedocs.io/en/stable/installation.html
27+
pip uninstall -y bson pymongo
28+
make install-ingest-mongodb
2429

2530
PYTHONPATH=. ./unstructured/ingest/main.py \
2631
mongodb \

test_unstructured_ingest/test-ingest-dest.sh

+1-4
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,7 @@ all_tests=(
3434
'sqlite.sh'
3535
'vectara.sh'
3636
'weaviate.sh'
37-
# NOTE(robinson) - mongo conflicts with astra because it ships with its
38-
# own version of bson, and installing bson from pip causes mongo to fail
39-
# ref: https://pymongo.readthedocs.io/en/stable/installation.html
40-
# 'mongodb.sh'
37+
'mongodb.sh'
4138
)
4239

4340
full_python_matrix_tests=(

test_unstructured_ingest/test-ingest-src.sh

+2-5
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ all_tests=(
4242
'confluence-diff.sh'
4343
'confluence-large.sh'
4444
'airtable-diff.sh'
45-
# NOTE(ryan): This test is disabled because it is triggering too many requests to the API
45+
# # NOTE(ryan): This test is disabled because it is triggering too many requests to the API
4646
# 'airtable-large.sh'
4747
'local-single-file.sh'
4848
'local-single-file-basic-chunking.sh'
@@ -62,10 +62,7 @@ all_tests=(
6262
'local-embed-voyageai.sh'
6363
'sftp.sh'
6464
'opensearch.sh'
65-
# NOTE(robinson) - mongo conflicts with astra because it ships with its
66-
# own version of bson, and installing bson from pip causes mongo to fail
67-
# ref: https://pymongo.readthedocs.io/en/stable/installation.html
68-
# 'mongodb.sh'
65+
'mongodb.sh'
6966
)
7067

7168
full_python_matrix_tests=(

0 commit comments

Comments
 (0)