Skip to content

Commit

Permalink
PERF: Construction of a DatetimeIndex from a list of Timestamp with t…
Browse files Browse the repository at this point in the history
…imezone (#51247)

* require tzdata on windows

* put minimum version?

* try installing with pip in conda envs

* fixup ci check

* whatsnew

* add tzdata to yaml files

* add tzdata>=2022.1 to test_wheels_windows.bat

* try using OR statement

* try another syntax

* try using the pypi install

* try marking windows in yaml

* fixup validation script

* clarify what tzdata is neeeded for, remove it as optional

* remove optional section on timezones

* require tzdata everywhere

* remove timezone extra in package-checks

* remove final timezone extra reference

---------

Co-authored-by: MarcoGorelli <>
  • Loading branch information
MarcoGorelli authored Mar 15, 2023
1 parent f26031e commit 3ea1780
Show file tree
Hide file tree
Showing 16 changed files with 32 additions and 35 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/package-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
runs-on: ubuntu-22.04
strategy:
matrix:
extra: ["test", "performance", "timezone", "computation", "fss", "aws", "gcp", "excel", "parquet", "feather", "hdf5", "spss", "postgresql", "mysql", "sql-other", "html", "xml", "plot", "output_formatting", "clipboard", "compression", "all"]
extra: ["test", "performance", "computation", "fss", "aws", "gcp", "excel", "parquet", "feather", "hdf5", "spss", "postgresql", "mysql", "sql-other", "html", "xml", "plot", "output_formatting", "clipboard", "compression", "all"]
fail-fast: false
name: Install Extras - ${{ matrix.extra }}
concurrency:
Expand Down
2 changes: 2 additions & 0 deletions ci/deps/actions-310-numpydev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@ dependencies:
- python-dateutil
- pytz
- pip

- pip:
- "cython"
- "--extra-index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple"
- "--pre"
- "numpy"
- "scipy"
- "tzdata>=2022.1"
4 changes: 3 additions & 1 deletion ci/deps/actions-310.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,10 @@ dependencies:
- scipy>=1.7.1
- sqlalchemy>=1.4.16
- tabulate>=0.8.9
- tzdata>=2022a
- xarray>=0.21.0
- xlrd>=2.0.1
- xlsxwriter>=1.4.3
- zstandard>=0.15.2

- pip:
- tzdata>=2022.1
4 changes: 3 additions & 1 deletion ci/deps/actions-311.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,10 @@ dependencies:
- scipy>=1.7.1
- sqlalchemy>=1.4.16
- tabulate>=0.8.9
- tzdata>=2022a
- xarray>=0.21.0
- xlrd>=2.0.1
- xlsxwriter>=1.4.3
- zstandard>=0.15.2

- pip:
- tzdata>=2022.1
3 changes: 3 additions & 0 deletions ci/deps/actions-38-downstream_compat.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,6 @@ dependencies:
- pandas-gbq>=0.15.0
- pyyaml
- py

- pip:
- tzdata>=2022.1
2 changes: 1 addition & 1 deletion ci/deps/actions-38-minimum_versions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,11 @@ dependencies:
- scipy=1.7.1
- sqlalchemy=1.4.16
- tabulate=0.8.9
- tzdata=2022a
- xarray=0.21.0
- xlrd=2.0.1
- xlsxwriter=1.4.3
- zstandard=0.15.2

- pip:
- pyqt5==5.15.1
- tzdata==2022.1
3 changes: 3 additions & 0 deletions ci/deps/actions-38.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,6 @@ dependencies:
- xlrd>=2.0.1
- xlsxwriter>=1.4.3
- zstandard>=0.15.2

- pip:
- tzdata>=2022.1
4 changes: 3 additions & 1 deletion ci/deps/actions-39.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,10 @@ dependencies:
- scipy>=1.7.1
- sqlalchemy>=1.4.16
- tabulate>=0.8.9
- tzdata>=2022a
- xarray>=0.21.0
- xlrd>=2.0.1
- xlsxwriter>=1.4.3
- zstandard>=0.15.2

- pip:
- tzdata>=2022.1
3 changes: 3 additions & 0 deletions ci/deps/actions-pypy-38.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,6 @@ dependencies:
- numpy
- python-dateutil
- pytz

- pip:
- tzdata>=2022.1
2 changes: 1 addition & 1 deletion ci/test_wheels_windows.bat
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ pd.test(extra_args=['-m not clipboard and not single_cpu and not slow and not ne
pd.test(extra_args=['-m not clipboard and single_cpu and not slow and not network and not db'])

python --version
pip install pytz six numpy python-dateutil
pip install pytz six numpy python-dateutil tzdata>=2022.1
pip install hypothesis>=6.34.2 pytest>=7.0.0 pytest-xdist>=2.2.0 pytest-asyncio>=0.17
pip install --find-links=pandas/dist --no-index pandas
python -c "%test_command%"
19 changes: 0 additions & 19 deletions doc/source/getting_started/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -308,25 +308,6 @@ Dependency Minimum Version pip ext
`numba <https://github.com/numba/numba>`__ 0.53.1 performance Alternative execution engine for operations that accept ``engine="numba"`` using a JIT compiler that translates Python functions to optimized machine code using the LLVM compiler.
===================================================== ================== ================== ===================================================================================================================================================================================

Timezones
^^^^^^^^^

Installable with ``pip install "pandas[timezone]"``

========================= ========================= =============== =============================================================
Dependency Minimum Version pip extra Notes
========================= ========================= =============== =============================================================
tzdata 2022.1(pypi)/ timezone Allows the use of ``zoneinfo`` timezones with pandas.
2022a(for system tzdata) **Note**: You only need to install the pypi package if your
system does not already provide the IANA tz database.
However, the minimum tzdata version still applies, even if it
is not enforced through an error.

If you would like to keep your system tzdata version updated,
it is recommended to use the ``tzdata`` package from
conda-forge.
========================= ========================= =============== =============================================================

Visualization
^^^^^^^^^^^^^

Expand Down
4 changes: 3 additions & 1 deletion doc/source/whatsnew/v2.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ When installing pandas using pip, sets of optional dependencies can also be inst
pip install "pandas[performance, aws]>=2.0.0"
The available extras, found in the :ref:`installation guide<install.dependencies>`, are
``[all, performance, computation, timezone, fss, aws, gcp, excel, parquet, feather, hdf5, spss, postgresql, mysql,
``[all, performance, computation, fss, aws, gcp, excel, parquet, feather, hdf5, spss, postgresql, mysql,
sql-other, html, xml, plot, output_formatting, clipboard, compression, test]`` (:issue:`39164`).

.. _whatsnew_200.enhancements.index_can_hold_numpy_numeric_dtypes:
Expand Down Expand Up @@ -650,6 +650,8 @@ If installed, we now require:
+-------------------+-----------------+----------+---------+
| python-dateutil | 2.8.2 | X | X |
+-------------------+-----------------+----------+---------+
| tzdata | 2022.1 | X | X |
+-------------------+-----------------+----------+---------+

For `optional libraries <https://pandas.pydata.org/docs/getting_started/install.html>`_ the general recommendation is to use the latest version.
The following table lists the lowest version per library that is currently being tested throughout the development of pandas.
Expand Down
2 changes: 1 addition & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ dependencies:
- scipy>=1.7.1
- sqlalchemy>=1.4.16
- tabulate>=0.8.9
- tzdata>=2022a
- xarray>=0.21.0
- xlrd>=2.0.1
- xlsxwriter>=1.4.3
Expand Down Expand Up @@ -119,3 +118,4 @@ dependencies:
- pip:
- sphinx-toggleprompt
- typing_extensions; python_version<"3.11"
- tzdata>=2022.1
5 changes: 2 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ dependencies = [
"numpy>=1.21.0; python_version>='3.10'",
"numpy>=1.23.2; python_version>='3.11'",
"python-dateutil>=2.8.2",
"pytz>=2020.1"
"pytz>=2020.1",
"tzdata>=2022.1"
]
classifiers = [
'Development Status :: 5 - Production/Stable',
Expand Down Expand Up @@ -57,7 +58,6 @@ matplotlib = "pandas:plotting._matplotlib"
[project.optional-dependencies]
test = ['hypothesis>=6.34.2', 'pytest>=7.0.0', 'pytest-xdist>=2.2.0', 'pytest-asyncio>=0.17.0']
performance = ['bottleneck>=1.3.2', 'numba>=0.53.1', 'numexpr>=2.7.1']
timezone = ['tzdata>=2022.1']
computation = ['scipy>=1.7.1', 'xarray>=0.21.0']
fss = ['fsspec>=2021.07.0']
aws = ['s3fs>=2021.08.0']
Expand Down Expand Up @@ -112,7 +112,6 @@ all = ['beautifulsoup4>=4.9.3',
'SQLAlchemy>=1.4.16',
'tables>=3.6.1',
'tabulate>=0.8.9',
'tzdata>=2022.1',
'xarray>=0.21.0',
'xlrd>=2.0.1',
'xlsxwriter>=1.4.3',
Expand Down
2 changes: 1 addition & 1 deletion requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ s3fs>=2021.08.0
scipy>=1.7.1
SQLAlchemy>=1.4.16
tabulate>=0.8.9
tzdata>=2022.1
xarray>=0.21.0
xlrd>=2.0.1
xlsxwriter>=1.4.3
Expand Down Expand Up @@ -88,4 +87,5 @@ requests
pygments
sphinx-toggleprompt
typing_extensions; python_version<"3.11"
tzdata>=2022.1
setuptools>=61.0.0
6 changes: 2 additions & 4 deletions scripts/validate_min_versions_in_sync.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
EXCLUDE_DEPS = {"tzdata", "blosc"}
EXCLUSION_LIST = {
"python=3.8[build=*_pypy]": None,
"tzdata": None,
"pyarrow": None,
}
# pandas package is not available
Expand Down Expand Up @@ -228,10 +227,9 @@ def get_versions_from_ci(content: list[str]) -> tuple[dict[str, str], dict[str,
continue
elif seen_required and line.strip():
if "==" in line:
package, version = line.strip().split("==")

package, version = line.strip().split("==", maxsplit=1)
else:
package, version = line.strip().split("=")
package, version = line.strip().split("=", maxsplit=1)
package = package[2:]
if package in EXCLUDE_DEPS:
continue
Expand Down

0 comments on commit 3ea1780

Please sign in to comment.