Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PERF: Construction of a DatetimeIndex from a list of Timestamp with timezone #51247

Merged
merged 22 commits into from
Mar 15, 2023
Merged

PERF: Construction of a DatetimeIndex from a list of Timestamp with timezone #51247

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
f4c9e0a
require tzdata on windows
Feb 8, 2023
3a9cc7d
put minimum version?
Feb 9, 2023
a0ac092
try installing with pip in conda envs
Feb 9, 2023
5a1983d
fixup ci check
Feb 9, 2023
cafb5cc
whatsnew
Feb 9, 2023
9d6e2df
Merge remote-tracking branch 'upstream/main' into require-tzdata-on-w…
Mar 2, 2023
c19dca9
add tzdata to yaml files
Mar 3, 2023
4a29800
Merge remote-tracking branch 'upstream/main' into require-tzdata-on-w…
Mar 3, 2023
1aaa8d0
add tzdata>=2022.1 to test_wheels_windows.bat
Mar 3, 2023
fa3f2bc
try using OR statement
Mar 3, 2023
b3c006e
try another syntax
Mar 3, 2023
3a561be
try using the pypi install
Mar 3, 2023
046a1e8
Merge remote-tracking branch 'upstream/main' into require-tzdata-on-w…
Mar 10, 2023
e18aebe
try marking windows in yaml
Mar 10, 2023
0f7891e
fixup validation script
Mar 10, 2023
249f345
clarify what tzdata is neeeded for, remove it as optional
Mar 10, 2023
4b054da
remove optional section on timezones
Mar 10, 2023
fc5a2d4
Merge branch 'main' into require-tzdata-on-windows
MarcoGorelli Mar 13, 2023
51d44f4
require tzdata everywhere
Mar 14, 2023
d07b2ec
Merge branch 'require-tzdata-on-windows' of github.com:MarcoGorelli/p…
Mar 14, 2023
624e0e7
remove timezone extra in package-checks
Mar 14, 2023
860338f
remove final timezone extra reference
Mar 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,7 @@ repos:
language: python
files: ^(ci/deps/actions-.*-minimum_versions\.yaml|pandas/compat/_optional\.py)$
additional_dependencies: [tomli]
pass_filenames: false
- id: validate-errors-locations
name: Validate errors locations
description: Validate errors are in appropriate locations.
Expand Down
3 changes: 3 additions & 0 deletions ci/deps/actions-310.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,6 @@ dependencies:
- xlrd
- xlsxwriter
- zstandard

- pip:
- tzdata>=2022.1
3 changes: 3 additions & 0 deletions ci/deps/actions-311.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,6 @@ dependencies:
- xlrd
- xlsxwriter
- zstandard

- pip:
- tzdata>=2022.1
3 changes: 3 additions & 0 deletions ci/deps/actions-38-downstream_compat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,6 @@ dependencies:
- pandas-gbq
- pyyaml
- py

- pip:
- tzdata>=2022.1
1 change: 1 addition & 0 deletions ci/deps/actions-38-minimum_versions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,4 @@ dependencies:

- pip:
- pyqt5==5.15.1
- tzdata==2022.1
3 changes: 3 additions & 0 deletions ci/deps/actions-38.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,6 @@ dependencies:
- xlrd
- xlsxwriter
- zstandard

- pip:
- tzdata>=2022.1
3 changes: 3 additions & 0 deletions ci/deps/actions-39.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,6 @@ dependencies:
- xlrd
- xlsxwriter
- zstandard

- pip:
- tzdata>=2022.1
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v2.0.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -663,6 +663,8 @@ If installed, we now require:
+-------------------+-----------------+----------+---------+
| python-dateutil | 2.8.2 | X | X |
+-------------------+-----------------+----------+---------+
| tzdata | 2022.1 |on Windows| X |
Copy link
Member

@lithomas1 lithomas1 Mar 10, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is misleading, since you'll need this package if you don't have the tz db on your computer, regardless of OS.

It's just that Windows always doesn't have it.

Also, I'm pretty sure I added tzdata as an optional dep a while back, so if you could remove that too, that'd be great.

EDIT: Actually, I'm wrong here, since pandas probably imports fine as long as you don't try to do datetime stuff. It'd still be good to clarify what it's needed for.

+-------------------+-----------------+----------+---------+

For `optional libraries <https://pandas.pydata.org/docs/getting_started/install.html>`_ the general recommendation is to use the latest version.
The following table lists the lowest version per library that is currently being tested throughout the development of pandas.
Expand Down
3 changes: 2 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ dependencies = [
"numpy>=1.21.0; python_version>='3.10'",
"numpy>=1.23.2; python_version>='3.11'",
"python-dateutil>=2.8.2",
"pytz>=2020.1"
"pytz>=2020.1",
"tzdata>=2022.1; platform_system=='Windows'"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be checked in setup.py? or even just import it in timezones.pyx?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly don't know, I couldn't reproduce this when building from source

Tempted to just ship it, and then check with the nightlies?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lithomas1 thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'll tag this as build to run the wheel builders, which should sniff this stuff out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lithomas1 did you try this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah the wheel builder jobs ran on this PR. Looks like some failures. I don't have Windows access so I can't help more sorry.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the wheel builder jobs ran on this PR

Ah, I see now sorry - thanks! Taking a look

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, you might need a check for this in setup.py?

If you uninstall tzdata and try to run python setup.py develop, does this error correctly?

I don't know if setuptools reads this section of pyproject.toml.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

giving this a go

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this worked

In a new virtual environment on Windows, I:

  • pip installed numpy
  • pip installed cython
  • pip install versioneer[toml]
  • ran python setup.py develop

and tzdata got installed, along with python-dateutil. Here's (part of) my output:

copying build\lib.win-amd64-cpython-38\pandas\_libs\window\aggregations.cp38-win_amd64.pyd -> pandas\_libs\window
copying build\lib.win-amd64-cpython-38\pandas\_libs\window\indexers.cp38-win_amd64.pyd -> pandas\_libs\window
copying build\lib.win-amd64-cpython-38\pandas\_libs\writers.cp38-win_amd64.pyd -> pandas\_libs
copying build\lib.win-amd64-cpython-38\pandas\io\sas\_sas.cp38-win_amd64.pyd -> pandas\io\sas
copying build\lib.win-amd64-cpython-38\pandas\io\sas\_byteswap.cp38-win_amd64.pyd -> pandas\io\sas
copying build\lib.win-amd64-cpython-38\pandas\_libs\json.cp38-win_amd64.pyd -> pandas\_libs
Creating c:\users\user\pandas-dev\.venv\lib\site-packages\pandas.egg-link (link to .)
Adding pandas 2.1.0.dev0+186.g4b054da685 to easy-install.pth file

Installed c:\users\user\pandas-dev
Processing dependencies for pandas==2.1.0.dev0+186.g4b054da685
Searching for tzdata>=2022.1
Reading https://pypi.org/simple/tzdata/
C:\Users\User\pandas-dev\.venv\lib\site-packages\pkg_resources\__init__.py:123: PkgResourcesDeprecationWarning:  is an invalid version and will not be supported in a future release
  warnings.warn(
Downloading https://files.pythonhosted.org/packages/fa/5e/f99a7df3ae2079211d31ec23b1d34380c7870c26e99159f6e422dcbab538/tzdata-2022.7-py2.py3-none-any.whl#sha256=2b88858b0e3120792a3c0635c23daf36a7d7eeeca657c323da299d2094402a0d
Best match: tzdata 2022.7
Processing tzdata-2022.7-py2.py3-none-any.whl
Installing tzdata-2022.7-py2.py3-none-any.whl to c:\users\user\pandas-dev\.venv\lib\site-packages
Adding tzdata 2022.7 to easy-install.pth file

Installed c:\users\user\pandas-dev\.venv\lib\site-packages\tzdata-2022.7-py3.8.egg

]
classifiers = [
'Development Status :: 5 - Production/Stable',
Expand Down