pdm init may create non-UTF-8 README.md and break pdm install

[YDX-2147483647]

• I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

1. pdm init. Leave everything as default except the following.
   • Is the project a library that is installable? y
   • Project description? 啊
2. Check the encoding of README.md: it's platform-dependent locale.
3. pdm install ⇒ UnicodeDecodeError

Expected behavior

README.md should be encoded as UTF-8 or self.DEFAULT_ENCODING.

pdm/src/pdm/cli/actions.py

Line 515 in f81e07c

readme.write_text(f"# {name}\n\n{description}\n")

readme.write_text(f"# {name}\n\n{description}\n", encoding='utf-8')

(Sorry I can’t make a PR because I have no idea about how to write a test for it.)

Actual behavior

1. pdm init. Leave everything as default except the following.

   • Is the project a library that is installable? y

   • Project description?

     啊 or any non-ASCII string.

   • Which build backend to use?

     I’ve tested pdm-pep517 (default) and setuptools. Both of them not work.

     For setuptools, the author’s email should also be provided.

   $ pdm init
   Creating a pyproject.toml for PDM...
   Please enter the Python interpreter to use
   0. C:\Program Files\Python310\python.EXE (3.10)
   1. …
   Please select (0):
   Using Python interpreter: C:\Program Files\Python310\python.EXE (3.10)
   Updating executable scripts...
   Would you like to create a virtualenv with C:\Program Files\Python310\python.EXE? [y/n] (y):
   Virtualenv is created successfully at D:\…\pdm-encoding\.venv
   Is the project a library that is installable?
   A few more questions will be asked to include a project name and build backend [y/n] (n): y 👈⭐
   Project name (pdm-encoding):
   Project version (0.1.0):
   Project description (): 啊 👈⭐
   Which build backend to use?
   0. pdm-pep517
   1. setuptools
   2. flit-core
   3. hatchling
   4. pdm-backend
   Please select (0):
   License(SPDX name) (MIT):
   Author name ():
   Author email ():
   Python requires('*' to allow any) (>=3.10):
   Changes are written to pyproject.toml.

2. Check the encoding of README.md: it's platform-dependent locale.

   $ pdm run python
   >> from pathlib import Path
   >> Path('README.md').read_text()
   '…啊\n'
   >> Path('README.md').read_text(encoding='utf-8')
   UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 18: invalid start byte

   Note: Details of the error message depends on your directory name, which determines the project name.

3. pdm install.

   $ pdm install
   …
   Installing the project as an editable package...
   …
       File "…\pdm\pep517\metadata.py", line 89, in _get_readme_content
           return Path(value).read_text(encoding=self.DEFAULT_ENCODING)
   …
   UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 18: invalid start byte

   Full traceback of `pdm install -v`

   Traceback (most recent call last):
     File "<frozen runpy>", line 198, in _run_module_as_main
     File "<frozen runpy>", line 88, in _run_code
     File "c:\users\<my_name>\.local\bin\pdm.exe\__main__.py", line 7, in <module>
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 247, in main
       return Core().main(args)
              ^^^^^^^^^^^^^^^^^
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 181, in main
       raise cast(Exception, err).with_traceback(traceback) from None
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 176, in main
       f(project, options)
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\cli\commands\install.py", line 58, in handle
       actions.do_sync(
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\cli\actions.py", line 209, in do_sync
       handler.synchronize()
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 405, in synchronize
       self.install_candidate(self_key, progress)
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 230, in install_candidate
       self.manager.install(can)
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\installers\manager.py", line 33, in install
       installer(str(prepared.build()), self.environment, prepared.direct_url())
                     ^^^^^^^^^^^^^^^^
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\models\candidates.py", line 375, in build
       self.wheel = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\builders\editable.py", line 49, in build
       filename = self._hook.build_editable(out_dir, config_settings, metadata_directory)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pyproject_hooks\_impl.py", line 270, in build_editable
       return self._call_hook('build_editable', {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pyproject_hooks\_impl.py", line 311, in _call_hook
       self._subprocess_runner(
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\builders\base.py", line 257, in subprocess_runner
       return log_subprocessor(cmd, cwd, extra_environ=env)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     File "D:\ProgramHome\pipx\venvs\pdm\Lib\site-packages\pdm\builders\base.py", line 107, in log_subprocessor
       raise build_error(e) from None
   pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines of the build output:
       if meta.long_description:
     File "C:\Users\bjalp\AppData\Local\Temp\pdm-build-env-nfkxolql-shared\Lib\site-packages\pdm\pep517\metadata.py", line 40, in __get__
       rv = self.fget(instance, rv)
     File "C:\Users\bjalp\AppData\Local\Temp\pdm-build-env-nfkxolql-shared\Lib\site-packages\pdm\pep517\metadata.py", line 89, in _get_readme_content
       return Path(value).read_text(encoding=self.DEFAULT_ENCODING)
     File "C:\Program Files\Python310\lib\pathlib.py", line 1135, in read_text
       return f.read()
     File "C:\Program Files\Python310\lib\codecs.py", line 322, in decode
       (result, consumed) = self._buffer_decode(data, self.errors, final)
   UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 18: invalid start byte

   Other commands (e.g. pdm add) also fail because of the same reason (pdm/pep517/metadata.py).

4. Change the encoding of README.md to UTF-8.

   $ pdm run python
   >> from pathlib import Path
   >> readme = Path('README.md')
   >> readme.write_text(readme.read_text(), encoding='utf-8')
   18

5. Everything works fine now.

   $ pdm install
   …
   🎉 All complete!

Environment Information

$ pdm info && pdm info --env
PDM version:
  2.4.6
Python Interpreter:
  D:\…\pdm-encoding\.venv\Scripts\python.exe (3.10)
Project Root:
  D:/…/pdm-encoding
Project Packages:
  None
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.10",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.19045",
  "python_full_version": "3.10.10",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "win32"
}

Addition Information

• Relevant issue: After published to PyPi, Chinese README displayed incorrectly #1345
• I install pdm by pipx.

[YDX-2147483647]

Thanks for such a quick fix! 👍