[Bug]: pymilvus2.0.2 relies on rgpcio 1.37.1 which cannot be installed on apple silicon

[ponponon]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

中文版本

─➤  pip install pymilvus                                                                                                                                                          
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting pymilvus
  Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a6/03/57b703812b4fb65a42b91bad38d8f79d257733267106965f78a66ee65d36/pymilvus-2.0.2-py3-none-any.whl (119 kB)
Collecting grpcio==1.37.1
  Using cached 

...........

      distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> grpcio

现状，pymilvus 依赖 grpcio==1.37.1，但是无法在 apple silicon 机器上安装 grpcio==1.37.1 。❌

可能的原因是 grpcio 老版本的一个 BUG 🐞

参考下面两个链接：

根据这个 👉 [grpcio fails to install on Apple Silicon](grpc/grpc#25082) 讨论。
和这个 PR 👉 fix: require grpcio >= 1.38.1 to prevent a possible crash](googleapis/python-pubsub#435)

在 grpcio 新版本中已经修复了这问题，安装 grpcio >= 1.38.1 即可解决 ✅

而目前 pymilvus 依赖的 grpcio 被写死为 1.37.1

我们需要让 pymilvus 支持最新的 grpcio 来解决这个问题，我看到已经有人提交了对应的 PR 来解决这个问题了update requirements，但是迟迟没有合并，急需解决 🆘🆘🆘

---

English translation version

─➤ pip install pymilvus                                                                                                                                                          
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting pymilvus
  Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a6/03/57b703812b4fb65a42b91bad38d8f79d257733267106965f78a66ee65d36/pymilvus- 2.0.2-py3-none-any.whl (119 kB)
Collecting grpcio==1.37.1
  Using cached 

...........

      distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1
      
      [end of output]
  
  Note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
buzz-> grpcio

Status, pymilvus relies on grpcio==1.37.1, but cannot install grpcio==1.37.1 on apple silicon machines. ❌

The possible reason is a bug in an older version of grpcio 🐞

Refer to the following two links.

Discussed under this 👉 grpcio fails to install on Apple Silicon](grpc/grpc#25082).
and this PR 👉 fix: require grpcio >= 1.38.1 to prevent a possible crash](googleapis/python-pubsub#435)

This issue has been fixed in the new version of grpcio, installing grpcio >= 1.38.1 will fix it ✅

And currently pymilvus relies on grpcio which is written dead as 1.37.1

We need to make pymilvus support the latest grpcio to solve this problem, I see that someone has already submitted a corresponding PR to solve this problem update requirements, but there is a delay in The merge is urgent 🆘🆘🆘🆘

Translated with www.DeepL.com/Translator (free version)

Expected Behavior

中文版本

我希望可以在 apple silicon 顺利安装 pymilvus 而不受 grpcio 版本的影响

English translation version

I want to be able to install pymilvus on apple silicon without any problems with the grpcio version

中文版本

使用下面的命令安装 pymilvus，会在编译 grcpio 的时候报错

pip install pymilvus

报错内容：

          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/private/var/folders/sx/r0j5n9c54gd_6lyxfr3tzhjh0000gn/T/pip-install-sk25937c/grpcio_ad9f2326c10346a3825de664c6a9c075/src/python/grpcio/commands.py", line 265, in new_compile
          return old_compile(obj, src, ext, cc_args, extra_postargs,
        File "/Users/ponponon/.local/share/virtualenvs/svddb_api-26TSdmul/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py", line 176, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> grpcio

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

English translation version

Installing pymilvus with the following command will report an error when compiling grcpio

pip install pymilvus

The error message reads.

          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/private/var/folders/sx/r0j5n9c54gd_6lyxfr3tzhjh0000gn/T/pip-install-sk25937c/grpcio_ad9f2326c10346a3825de664c6a9c075/src/python/grpcio/commands.py", line 265, in new_compile
          return old_compile(obj, src, ext, cc_args, extra_postargs,
        File "/Users/ponponon/.local/share/virtualenvs/svddb_api-26TSdmul/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py", line 176, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> grpcio

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

---

Environment details

- Hardware/Softward conditions (OS, CPU, GPU, Memory): mac、 apple silicon、apple silicon、16GB
- Method of installation (Docker, or from source): mac raw env
- Milvus version (v0.3.1, or v0.4.0): 2.0.2
- Milvus configuration (Settings you made in `server_config.yaml`): None

─➤  python --version          
Python 3.10.5
─➤  pip --version             
pip 22.0.4 from /Users/ponponon/.local/share/virtualenvs/svddb_api-26TSdmul/lib/python3.10/site-packages/pip (python 3.10)
─➤  clang --version
Apple clang version 13.0.0 (clang-1300.0.27.3)
Target: arm64-apple-darwin21.2.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Anything else?

这是我的计算机信息

[XuanYang-cn]

related issue: #946

[XuanYang-cn]

This grpcio problem is annoying indeed, planning to solve this in pymilvus2.1 and pymilvus2.0.3

[selected-pixel-jameson]

This grpcio problem is annoying indeed, planning to solve this in pymilvus2.1 and pymilvus2.0.3

Would love to see this resolved. Thank you.

[chang48]

I wonder if this could help address the issue:
GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 pip install pymilvus

[kemics]

@XuanYang-cn thank you for your PR, when do you plan to release pymilvus2.0.3? I can't use pymilvus now on GCP environment

[utopictown]

I wonder if this could help address the issue: GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 pip install pymilvus

tried. no

[YuriWerewolf]

Exactly the same issue on Windows machine as well, pip3 install pymilvus==2.1.0 still relies on rgpcio 1.37.1 which is impossible to install.

Rgpcio 1.48.0 install perfectly but pymilvus still tries to get 1.37.1 and then fails to install. Would love to get this depencency on 1.37.1 removed

[kemics]

@XuanYang-cn can you please check this issue? Release 2.1.0 doesn't have desired changes in requirements.txt https://github.com/milvus-io/pymilvus/blob/2.1/requirements.txt

It's really inconvenient to use pymilvus right now and such easy fix should have a shortcut way to production. It's been already 3 months of waiting for it 🙏

[XuanYang-cn]

@kemics requirements.txt is used to develop PyMilvus. The required third-party packages to use PyMilvus are listed in setup.py

pymilvus/setup.py

Line 21 in f4bda68

install_requires=[

Please try not to use cache while installing PyMilvus so that pip would get the latest grpcio >= 1.47.0

pip install pymilvus==2.1.1 --no-cache-dir

[bm777]

This grpcio problem is annoying indeed, planning to solve this in pymilvus2.1 and pymilvus2.0.3

This solved the same issue I had.
Thanks @XuanYang-cn

I have a MacBook M2 13inch

[sdhjl2000]

xcode-select --install
brew install openssl
CFLAGS="-I/opt/homebrew/opt/openssl/include" LDFLAGS="-L/opt/homebrew/opt/openssl/lib" GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 pip install grpcio==1.37.1

[RoacherM]

pip install grpcio==1.56.0 pip install grpcio-tools==1.56.0 can run on my m2 mac
❯ pip list | grep pymilvus
pymilvus 2.3.0
❯ pip list | grep grpc
grpcio 1.56.0
grpcio-tools 1.56.0

[XuanYang-cn]

fixed and closing