datetime.utcnow and datetime.utcfromtimestamp are deprecated in Python 3.12

[tirkarthi]

Apache Airflow version

2.6.2

What happened

The usage of datetime.utcnow is deprecated in Python 3.12 and it can be replaced with airflow.utils.timezone.utcnow and a similar method can be introduced for utcfromtimestamp too in timezone module

datetime.datetime’s utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version. Instead, use timezone-aware objects to represent datetimes in UTC: respectively, call now() and fromtimestamp() with the tz parameter set to datetime.UTC. (Contributed by Paul Ganssle in gh-103857.)

What you think should happen instead

No response

How to reproduce

./python                     
Python 3.13.0a0 (heads/main:58906213cc, Jul  3 2023, 21:54:53) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> datetime.datetime.utcnow()
<stdin>:1: DeprecationWarning: datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.now(datetime.UTC).
datetime.datetime(2023, 7, 4, 12, 41, 45, 658447)
>>> datetime.datetime.utcfromtimestamp(10000)
<stdin>:1: DeprecationWarning: datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.fromtimestamp(timestamp, datetime.UTC).
datetime.datetime(1970, 1, 1, 2, 46, 40)

Operating System

Ubuntu

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[tirkarthi]

I am working on a PR for this and will create it once complete.

[potiuk]

Just be warned. We are using pendulum in some places. And in some not. This might have rather undesireable side effects when you replace those deprecated. Not sure how but I know @uranusjr had a document once describing some changes that might be needed to handle the complexity involved and it was not trivial.

[uranusjr]

I talked to Paul Ganssle a while ago and these two specifically have few caveats, you basically need to use the version without utc and make sure to always attach an appropriate timezone. Or since we already use Pendulum we can just use its now(UTC). I’ll take a look at the PR once submitted though.

[tirkarthi]

timezone.utcnow that constructs current time without timezone and then uses replace to update the timezone. It is used in many places. There are also places where datetime.utcnow() is used that has no timezone and as per the linked cpython bug this leads to issues where this gets passed to timezone aware functions. My attempt would be to replace them with timezone.utcnow version. See also python/cpython#81669 (comment) for an example

[jscheffl]

I would not call it a bug though. It is rather an improvement as Python 3.12 is not officially supported yet. So a PR is appreciated but will be something like a feature to prepare for support Python 3.12++ in future releases.

[potiuk]

I would not call it a bug though. It is rather an improvement as Python 3.12 is not officially supported yet. So a PR is appreciated but will be something like a feature to prepare for support Python 3.12++ in future releases.

yes

[tirkarthi]

Unfortunately, I don't have the bandwidth to proceed with the fix. If someone wants to use my work as base please feel free to check the changes at https://github.com/tirkarthi/airflow/tree/fix-datetime