[Feature][Zeta] add from_unixtime function #5462

chovy-3012 · 2023-09-11T12:12:56Z

Purpose of this pull request

Check list

Code changed are covered with tests, or it does not need tests for reason:
If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
If you are contributing the connector code, please check that the following files are updated:
1. Update change log that in connector document. For more details you can refer to connector-v2
2. Update plugin-mapping.properties and add new connector information in it
3. Update the pom file of seatunnel-dist
Update the release-note.

add from_unixtime function.
Convert the number of seconds from the UNIX epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment.

Hisoka-X · 2023-09-12T02:22:27Z

cc @rewerma

Hisoka-X

How do you test it? Could you add some test case?

chovy-3012 · 2023-09-14T05:57:40Z

...ransforms-v2/src/test/java/org/apache/seatunnel/transform/sql/zeta/DateTimeFunctionTest.java

yes , I've already added unit test for from_unixtime function

ruanwenjun

Please add doc about this feature. https://seatunnel.apache.org/docs/2.3.3/transform-v2/sql-functions

After this PR is merged, we may need to consider adding benchmark test for the function.

...rms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/DateTimeFunction.java

…unixtime function and unit test

ruanwenjun

LGTM

Hisoka-X · 2023-09-14T08:06:15Z

Seem like the code style should be fixed. Please run mvn spotless:apply to fix the code style.

chovy-3012 · 2023-09-14T08:10:50Z

Seem like the code style should be fixed. Please run mvn spotless:apply to fix the code style.

missed code formatting on sql-functions.md

spotless:apply on doc

Hisoka-X · 2023-09-14T11:05:46Z

The UT not passed. @chovy-3012 Could you check it?

[INFO] Results:
[INFO] 
Error:  Failures: 
Error:    DateTimeFunctionTest.testFromUnixtimeFunction:53 expected: <2023-01-01 00:00:00> but was: <2022-12-31 16:00:00>

chovy-3012 · 2023-09-14T11:26:01Z

The UT not passed. @chovy-3012 Could you check it?

[INFO] Results:
[INFO] 
Error:  Failures: 
Error:    DateTimeFunctionTest.testFromUnixtimeFunction:53 expected: <2023-01-01 00:00:00> but was: <2022-12-31 16:00:00>

I made a mistake on the results of this unit test on host in different time zones.
Your host should be in the UTC+0 time zone.
I have modified the unit test.

ruanwenjun · 2023-09-18T01:49:45Z

Please resolve the conflict.

chovy-3012 · 2023-09-18T04:24:58Z

release-note.md

@@ -76,6 +76,7 @@
 - [Zeta] Fix cpu load problem (#4828)
 - [zeta] Fix the deadlock issue with JDBC driver loading (#4878)
 - [zeta] dynamically replace the value of the variable at runtime (#4950)
+- [Zeta] Add from_unixtime function (#5462)


conflict resolved

rewerma · 2023-09-18T08:40:55Z

Please add e2e test case.

chovy-3012 · 2023-09-23T09:37:15Z

add e2e test case

Hisoka-X · 2023-10-12T09:30:05Z

The CI failed. Could you investigate it?

chovy-3012 · 2023-10-25T04:10:56Z

Retried more than 10 times , still failed.
Error messages show no space left on device , any suggestions?

error message as below:

[INFO] Running org.apache.seatunnel.connectors.seatunnel.jdbc.JdbcAutoGenerateSQLIT
Error:  Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 8.34 s <<< FAILURE! - in org.apache.seatunnel.connectors.seatunnel.jdbc.JdbcAutoGenerateSQLIT
Error:  org.apache.seatunnel.connectors.seatunnel.jdbc.JdbcAutoGenerateSQLIT  Time elapsed: 8.34 s  <<< ERROR!
java.util.concurrent.CompletionException: org.testcontainers.containers.ContainerLaunchException: Container startup failed
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=postgres:14-alpine, imagePullPolicy=DefaultPullPolicy(), imageNameSubstitutor=org.testcontainers.utility.ImageNameSubstitutor$LogWrappedImageNameSubstitutor@489496a9)
Caused by: org.testcontainers.containers.ContainerFetchException: Failed to get Docker client for postgres:14-alpine
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not pull image: failed to register layer: write /usr/lib/libLLVM-15.so: no space left on device

Carl-Zhou-CN

+1

Hisoka-X · 2023-11-02T08:27:53Z

Hi @chovy-3012 , could you try merge from dev to retrigger CI? Thanks.

chovy-3012 · 2023-11-04T05:15:18Z

Hi @chovy-3012 , could you try merge from dev to retrigger CI? Thanks.

@Hisoka-X
Merge to my this feature branch from branch apache dev ？
Wouldn't this result in the feature branch having many commits that have already been merged into the dev branch?

Carl-Zhou-CN · 2023-11-13T10:20:49Z

@chovy-3012

You can start with the branch chovy-3012:feature/from_unixtime_function

Carl-Zhou-CN · 2023-11-14T11:27:45Z

@chovy-3012 Sorry, that's how it's supposed to work. I made a mistake

chovy-3012 · 2023-11-17T01:01:27Z

Merged branch apache:dev into my:feature/from_unixtime_function , but CI still has one error after multiple retries
@Carl-Zhou-CN

Carl-Zhou-CN

LGTM

[feature] add from_unixtime function

673ca4f

chovy-3012 changed the title ~~[feature] add from_unixtime function~~ [Feature][Zeta] add from_unixtime function Sep 11, 2023

[doc] update release-node

91048be

Hisoka-X added First-time contributor First-time contributor feature New feature Zeta SQL Transform labels Sep 12, 2023

Hisoka-X requested a review from rewerma September 12, 2023 02:22

Hisoka-X reviewed Sep 13, 2023

View reviewed changes

[test] add from_unixtime function unit test

a9d9aa0

chovy-3012 commented Sep 14, 2023

View reviewed changes

ruanwenjun reviewed Sep 14, 2023

View reviewed changes

...rms-v2/src/main/java/org/apache/seatunnel/transform/sql/zeta/functions/DateTimeFunction.java Outdated Show resolved Hide resolved

ruanwenjun assigned chovy-3012 Sep 14, 2023

[doc][feature] add doc about this function, add timezone arg in from_…

db15a73

…unixtime function and unit test

ruanwenjun previously approved these changes Sep 14, 2023

View reviewed changes

ruanwenjun added the ready to merge label Sep 14, 2023

github-actions bot added approved reviewed labels Sep 14, 2023

[doc] apply codee style on sql-functions.md

6914112

chovy-3012 dismissed ruanwenjun’s stale review via 6914112 September 14, 2023 08:08

github-actions bot removed approved reviewed labels Sep 14, 2023

[test] modify unit test

46c649d

Merge branch 'dev' into feature/from_unixtime_function

0968572

chovy-3012 commented Sep 18, 2023

View reviewed changes

[test] add e2e test case

e2c0f3c

chovy-3012 added 2 commits October 12, 2023 19:56

modify e2e ut

7da79df

modify e2e ut

ac28d20

hailin0 closed this Oct 25, 2023

hailin0 reopened this Oct 25, 2023

Carl-Zhou-CN reviewed Nov 1, 2023

View reviewed changes

Merge branch 'apache:dev' into feature/from_unixtime_function

25d1b73

Carl-Zhou-CN closed this Nov 19, 2023

Carl-Zhou-CN reopened this Nov 19, 2023

Carl-Zhou-CN approved these changes Nov 19, 2023

View reviewed changes

github-actions bot added approved reviewed labels Nov 19, 2023

EricJoy2048 approved these changes Nov 20, 2023

View reviewed changes

EricJoy2048 merged commit fb24f8e into apache:dev Nov 20, 2023
6 of 7 checks passed

alextinng pushed a commit to alextinng/seatunnel that referenced this pull request Dec 19, 2023

[Feature][Zeta] add from_unixtime function (apache#5462)

8baa822

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature][Zeta] add from_unixtime function #5462

[Feature][Zeta] add from_unixtime function #5462

chovy-3012 commented Sep 11, 2023

Hisoka-X commented Sep 12, 2023

Hisoka-X left a comment

chovy-3012 Sep 14, 2023 •

edited

Loading

ruanwenjun left a comment

ruanwenjun left a comment

Hisoka-X commented Sep 14, 2023

chovy-3012 commented Sep 14, 2023 •

edited

Loading

Hisoka-X commented Sep 14, 2023

chovy-3012 commented Sep 14, 2023

ruanwenjun commented Sep 18, 2023

chovy-3012 Sep 18, 2023

rewerma commented Sep 18, 2023

chovy-3012 commented Sep 23, 2023

Hisoka-X commented Oct 12, 2023

chovy-3012 commented Oct 25, 2023

Carl-Zhou-CN left a comment

Hisoka-X commented Nov 2, 2023

chovy-3012 commented Nov 4, 2023 •

edited

Loading

Carl-Zhou-CN commented Nov 13, 2023

Carl-Zhou-CN commented Nov 14, 2023 •

edited

Loading

chovy-3012 commented Nov 17, 2023

Carl-Zhou-CN left a comment

[Feature][Zeta] add from_unixtime function #5462

[Feature][Zeta] add from_unixtime function #5462

Conversation

chovy-3012 commented Sep 11, 2023

Purpose of this pull request

Check list

Hisoka-X commented Sep 12, 2023

Hisoka-X left a comment

Choose a reason for hiding this comment

chovy-3012 Sep 14, 2023 • edited Loading

Choose a reason for hiding this comment

ruanwenjun left a comment

Choose a reason for hiding this comment

ruanwenjun left a comment

Choose a reason for hiding this comment

Hisoka-X commented Sep 14, 2023

chovy-3012 commented Sep 14, 2023 • edited Loading

Hisoka-X commented Sep 14, 2023

chovy-3012 commented Sep 14, 2023

ruanwenjun commented Sep 18, 2023

chovy-3012 Sep 18, 2023

Choose a reason for hiding this comment

rewerma commented Sep 18, 2023

chovy-3012 commented Sep 23, 2023

Hisoka-X commented Oct 12, 2023

chovy-3012 commented Oct 25, 2023

Carl-Zhou-CN left a comment

Choose a reason for hiding this comment

Hisoka-X commented Nov 2, 2023

chovy-3012 commented Nov 4, 2023 • edited Loading

Carl-Zhou-CN commented Nov 13, 2023

Carl-Zhou-CN commented Nov 14, 2023 • edited Loading

chovy-3012 commented Nov 17, 2023

Carl-Zhou-CN left a comment

Choose a reason for hiding this comment

chovy-3012 Sep 14, 2023 •

edited

Loading

chovy-3012 commented Sep 14, 2023 •

edited

Loading

chovy-3012 commented Nov 4, 2023 •

edited

Loading

Carl-Zhou-CN commented Nov 14, 2023 •

edited

Loading