Please expose _PyTime_t, _PyTime_FromTimeval, and _PyTime_AsSecondsDouble as public APIs

[ryancdotorg]

Feature or enhancement

Proposal:

#106316 #106317 removed pytime.h, and I was using some things

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-110850: Add PyTime_t C API #112135
• gh-110850: Add PyTime_t C API #115215
• gh-110850: Partially revert "gh-106316: Remove pytime.h header file (#106317) #115348
• gh-110850: PyTime_Time() return 0 on success #115713
• gh-110850: Enhance PyTime C API tests #115715
• gh-110850: Replace _PyTime_t with PyTime_t #115719
• gh-110850: Cleanup pycore_time.h includes #115724
• gh-110850: Rename internal PyTime C API functions #115734
• gh-110850: Use public PyTime functions #115746
• gh-110850: Replace private _PyTime_MAX with public PyTime_MAX #115751
• gh-110850: Cleanup PyTime API: PyTime_t are nanoseconds #115753
• gh-110850: Add PyTime_TimeUnchecked() function #115973
• gh-110850: Expose _PyTime_TimeUnchecked function for now #116463
• gh-110850: Fix _PyTime_FromSecondsDouble() API #116606
• gh-110850: Add PyTime_TimeRaw() function #118394
• gh-110850: Remove _PyTime_TimeUnchecked() function #118552
• gh-110850: Use _PyDeadline_Get() in EnterNonRecursiveMutex() #118556

[hugovk]

Please could you say something about your use case?

Why do you need these functions?

Have you tried alternative APIs to achieve the same result?

cc @vstinner

[ryancdotorg]

I have a C/C++ module that was using them which I wrote a couple years ago. It broke.

I actually dug into what I was doing, and it turns out it was basically just (error checking omitted):

_PyTime_t tp;
struct timeval tv;
get_timestamp(&tv, ...);
_PyTime_FromTimeval(&tp, &tv);
double ts = _PyTime_AsSecondsDouble(tp); 

and I was able to more sensibly replace it with

double tv_frac = tv.tv_usec / 1e6;
struct timeval tv;
get_timestamp(&tv, ...);
double tv_frac = tv.tv_usec / 1e6;
double ts = tv.tv_sec + tv_frac;

It seems I just didn't want to bother figuring out the safest way to do the conversion.

This can be closed.

[vstinner]

I'm open to make some _Py_Time functions public, but apparently here, there wasn't a strong need for that. Maybe it can wait. Thanks for the report anyway.

[scoder]

We actually expose that API in Cython's cpython/time.pxd. Please move it back to where it was.

[vstinner]

We actually expose that API in Cython's cpython/time.pxd. Please move it back to where it was.

This file uses the following APIs:

• _PyTime_t
• _PyTime_GetSystemClock()
• _PyTime_AsSecondsDouble()

[scoder]

This file uses the following APIs

Correct. It basically mimics the Python time module and reimplements a part of it in C.

I'd still recommend exposing the complete C-API. We are talking about a standard library module here, not a part of the core C-API, so it represents a natural API border already. We are not talking about internals here. Let's just keep that up and allow regular external use as before.

[vstinner]

I designed _PyTime_t API for Python internal usage, but apparently some people discovered it and decided to use it since it's convenient.

_PyTime_t API was designed to be able to write easily "deadline = time + timeout", but later I started to add "clamp" variants in code which cannot report errors to the caller. Clamp the result in the range [_PyTime_MIN; _PyTime_MAX] is better than having an undefined behavior.

One issue is that _PyTime_t range is smaller than time_t range:

// The _PyTime_t API supports a resolution of 1 nanosecond. The _PyTime_t type
// is signed to support negative timestamps. The supported range is around
// [-292.3 years; +292.3 years]. Using the Unix epoch (January 1st, 1970), the
// supported date range is around [1677-09-21; 2262-04-11].

64-bit time_t (standard even on 32-bit platforms) is way larger than that. In practice, it should not be an issue. But the datetime module cannot use _PyTime_t everywhere for example, since datetime max year is 9999, not year 2262.

I tried to make the _PyTime_t unit an implementation details and enforce _PyTime_FromNanoseconds() or _PyTime_FromSeconds() usage.

About raised exception, ValueError is raised for Not-A-Number floating pointer number, and OverflowError on conversions (_PyTime_t, time_t, timeval, timespec, etc.).

[encukou]

Victor, do you have any intention of putting the removed API back?

[vstinner]

Victor, do you have any intention of putting the removed API back?

My plan in this issue is to add public functions replacing removed _PyTime functions. I'm collecting data to see how the private API was used and which API should be added.

[vstinner]

I wrote PR #112135 to add the public functions: PyTime_AsSecondsDouble(), PyTime_GetMonotonicClock(), PyTime_GetPerfCounter() and PyTime_GetSystemClock().

[scoder]

Marking this as a release blocker for 3.13 since the API removal is a regression in 3.13.

[vstinner]

Marking this as a release blocker for 3.13 since the API removal is a regression in 3.13.

I reopened my PR #112135 and created a C API Working Group issue: capi-workgroup/decisions#8

[encukou]

Raw time API was added in #118394.