Skip to content

Commit

Permalink
add reference to DESCRIBE HISTORY, replace Unix time with Unix epoch
Browse files Browse the repository at this point in the history
  • Loading branch information
dhruvarya-db committed Feb 7, 2024
1 parent 33298fb commit 07e4435
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions protocol_rfcs/in-commit-timestamps.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ A delta file can optionally contain additional provenance information about what

Implementations are free to store any valid JSON [object literal](https://www.w3schools.com/js/js_json_objects.asp) as the `commitInfo` action <ins>unless some table feature (e.g. [In-Commit Timestamps](#in-commit-timestamps)) imposes additional requirements on the data</ins>.

When In-Commit Timestamp are enabled, writers are required to include a commitInfo action with every commit, which must include the `inCommitTimestamp` field.
<ins>When In-Commit Timestamp are enabled, writers are required to include a commitInfo action with every commit, which must include the `inCommitTimestamp` field.</ins>

#### Reader Requirements for AddCDCFile
> ***Change to existing section***
Expand All @@ -25,7 +25,7 @@ When In-Commit Timestamp are enabled, writers are required to include a commitIn
Field Name | Data Type | Description
-|-|-
_commit_version|`Long`| The table version containing the change. This can be derived from the name of the Delta log file that contains actions.
_commit_timestamp|`Timestamp`| The timestamp associated when the commit was created. ~~This can be derived from the file modification time of the Delta log file that contains actions.~~ <ins> Depending on whether [In-Commit Timestamps](#in-commit-timestamps) are enabled, this is either the file modification time or the `inCommitTimestamp` stored in the `commitInfo` action of the Delta log file with the version `__commit_version`.</ins>
_commit_timestamp|`Timestamp`| The timestamp associated when the commit was created. ~~This can be derived from the file modification time of the Delta log file that contains actions.~~ <ins>Depending on whether [In-Commit Timestamps](#in-commit-timestamps) are enabled, this is derived from the `inCommitTimestamp` field of the `commitInfo` action of the version's Delta log, or from the Delta log's file modification time.</ins>

# In-Commit Timestamps
> ***New Section after the [Clustered Table](#clustered-table) section***
Expand All @@ -43,7 +43,7 @@ When In-Commit Timestamps is enabled, then:
1. Writers must write the `commitInfo` (see [Commit Provenance Information](#commit-provenance-information)) action in the commit.
2. The `commitInfo` action must be the first action in the commit.
3. The `commitInfo` action must include a field named `inCommitTimestamp`, of type `long` (see [Primitive Types](#primitive-types)), which represents the time (in milliseconds since the Unix epoch) when the commit is considered to have succeeded. It is the larger of two values:
- The Unix wall clock time at which the writer attempted the commit
- The time, in milliseconds since the Unix epoch, at which the writer attempted the commit
- One millisecond later than the previous commit's `inCommitTimestamp`
4. If the table has commits from a period when this feature was not enabled, provenance information around when this feature was enabled must be tracked in table properties:
- The property `delta.inCommitTimestampEnablementVersion` must be used to track the version of the table when this feature was enabled.
Expand All @@ -52,7 +52,7 @@ When In-Commit Timestamps is enabled, then:

## Recommendations for Readers of Tables with In-Commit Timestamps

For tables with In-Commit timestamps enabled, readers should use the `inCommitTimestamp` as the commit timestamp for operations like time travel.
For tables with In-Commit timestamps enabled, readers should use the `inCommitTimestamp` as the commit timestamp for operations like time travel and `DESCRIBE HISTORY` (https://docs.delta.io/latest/delta-utility.html#retrieve-delta-table-history).
If a table has commits from a period before In-Commit timestamps were enabled, the table properties `delta.inCommitTimestampEnablementVersion` and `delta.inCommitTimestampEnablementTimestamp` would be set and can be used to identify commits that don't have `inCommitTimestamp`.
To correctly determine the commit timestamp for these tables, readers can use the following rules:
1. For commits with version >= `delta.inCommitTimestampEnablementVersion`, readers should use the `inCommitTimestamp` field of the `commitInfo` action.
Expand Down

0 comments on commit 07e4435

Please sign in to comment.