Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix formatting #20570

Merged
merged 3 commits into from
Dec 16, 2022
Merged

Fix formatting #20570

merged 3 commits into from
Dec 16, 2022

Conversation

suhomud
Copy link
Contributor

@suhomud suhomud commented Dec 16, 2022

What

Formatting fix.

Spotless failed to format next files:
airbyte-integrations/connectors/**/integration_tests/expected_records.json,
With:

Execution failed for task ':spotlessStyling'.
> Unexpected response status code at /prettier/format [HTTP 501] -- (Error while formatting: SyntaxError: Unexpected token (2:1)
    1 | {"stream": "sat_basic_dataset", "data": {"id": 1, "test_column_1": -128, "test_column_10": -128, "test_column_11": -128, "test_column_12": -128.0, "test_column_13": true, "test_column_15": "2021-10-20T00:00:00Z", "test_column_16": "2021-10-20T11:22:33Z", "test_column_17": "2021-10-20T11:22:33Z", "test_column_19": "qwe", "test_column_2": -128, "test_column_21": "15:30:00", "test_column_3": -128, "test_column_4": -128, "test_column_5": -128, "test_column_6": -128, "test_column_7": -128, "test_column_8": -128, "test_column_9": -128 }, "emitted_at": 1669643054045 }
  > 2 | {"stream": "sat_basic_dataset", "data": {"id": 2, "test_column_1": 127, "test_column_10": 127, "test_column_11": 127, "test_column_12": 127.0, "test_column_13": false, "test_column_15": "9999-12-31T00:00:00Z", "test_column_16": "9999-12-31T11:22:33Z", "test_column_17": "2021-10-20T11:22:33Z", "test_column_19": "\u0439\u0446\u0443", "test_column_2": 127, "test_column_21": "15:30:00", "test_column_3": 127, "test_column_4": 127, "test_column_5": 127, "test_column_6": 127, "test_column_7": 127, "test_column_8": 127, "test_column_9": 127 }, "emitted_at": 1669643054045 }
      | ^

How

As a workaround it was excluded from Spotless check :

            '**/integration_tests/expected_records.json',

Ticket for revisiting the issue is created #20569

🚨 User Impact 🚨

No impact

@github-actions
Copy link
Contributor

github-actions bot commented Dec 16, 2022

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector or module version
  • Add changelog
  • Publish the new version

⚠ Sources (28)

Connector Version Changelog Publish
source-alloydb 1.0.34
source-alloydb-strict-encrypt 1.0.34 🔵
(ignored)
🔵
(ignored)
source-bigquery 0.2.3
source-clickhouse 0.1.14
source-clickhouse-strict-encrypt 0.1.14 🔵
(ignored)
🔵
(ignored)
source-cockroachdb 0.1.18
source-cockroachdb-strict-encrypt 0.1.18 🔵
(ignored)
🔵
(ignored)
source-db2 0.1.16
source-db2-strict-encrypt 0.1.16 🔵
(ignored)
🔵
(ignored)
source-dynamodb 0.1.0
source-e2e-test 2.1.3
source-e2e-test-cloud 2.1.1
(doc not found)

(not in seed)
source-elasticsearch 0.1.1
source-jdbc 0.3.5
(doc not found)

(not in seed)
source-mongodb-strict-encrypt 0.1.19 🔵
(ignored)
🔵
(ignored)
source-mongodb-v2 0.1.19
source-mssql 0.4.26
source-mssql-strict-encrypt 0.4.26 🔵
(ignored)
🔵
(ignored)
source-mysql 1.0.18
source-mysql-strict-encrypt 1.0.18 🔵
(ignored)
🔵
(ignored)
source-oracle 0.3.21
source-oracle-strict-encrypt 0.3.21 🔵
(ignored)
🔵
(ignored)
source-postgres 1.0.34
source-postgres-strict-encrypt 1.0.34 🔵
(ignored)
🔵
(ignored)
source-redshift 0.3.15
source-scaffold-java-jdbc 0.1.0
(doc not found)

(not in seed)
source-snowflake 0.1.27
source-tidb 0.2.1
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (3)

Connector Version Changelog Publish
destination-bigquery 1.2.9
destination-bigquery-denormalized 1.2.9
destination-s3-glue 0.1.1
  • See "Actionable Items" below for how to resolve warnings and errors.

👀 Other Modules (1)

  • base-normalization

Actionable Items

(click to expand)

Category Status Actionable Item
Version
mismatch
The version of the connector is different from its normal variant. Please bump the version of the connector.

doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog
doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.

changelog missing
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish
not in seed
The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.

diff seed version
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version.

Copy link
Contributor

@edgao edgao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the problem with the two sql files is that we're using snowflake-specific string escape syntax, so spotlessSql doesn't understand it. Can you try replacing the \' with '' and doing a /test connector=connectors/source-snowflake?

feel free to 🚢 if that doesn't work though

Copy link
Contributor

@ryankfu ryankfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While not familiar with Snowflake string index issues, I would agree with what Ed has mentioned with converting the quotes and retrying to see if that alleviates the formatting issues

Otherwise, the rest of the changes make sense since it's mostly formatting issues

@edgao
Copy link
Contributor

edgao commented Dec 16, 2022

going to just kick this off since yevhen might be asleep and I want to get this in before the code freeze :P

@edgao

This comment was marked as outdated.

@edgao edgao temporarily deployed to more-secrets December 16, 2022 21:17 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 16, 2022 21:17 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 16, 2022 21:25 — with GitHub Actions Inactive
@edgao edgao temporarily deployed to more-secrets December 16, 2022 21:25 — with GitHub Actions Inactive
@edgao
Copy link
Contributor

edgao commented Dec 16, 2022

/test connector=connectors/source-snowflake

🕑 connectors/source-snowflake https://github.com/airbytehq/airbyte/actions/runs/3716656673
✅ connectors/source-snowflake https://github.com/airbytehq/airbyte/actions/runs/3716656673
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       140      5    96%   87, 93, 238, 242-243
	 source_acceptance_test/conftest.py                     208     92    56%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-280, 288-301, 306-312, 319-330, 337-353
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              402    115    71%   53, 58, 93-104, 109-116, 120-121, 125-126, 308, 346-363, 376-387, 391-396, 402, 435-440, 478-485, 528-530, 533, 598-606, 618-621, 626, 682-683, 689, 692, 728-738, 751-776
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 source_acceptance_test/utils/json_schema_helper.py     107     13    88%   30-31, 38, 41, 65-68, 96, 120, 192-194
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1603    336    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:94: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:377: The previous and actual discovered catalogs are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_incremental.py:26: `future_state` not specified, skipping.
=================== 27 passed, 3 skipped in 62.90s (0:01:02) ===================

@edgao
Copy link
Contributor

edgao commented Dec 16, 2022

was able to run ./gradlew spotlessStyling locally, and tests are passing. going to just merge this in.

@edgao edgao merged commit 2a5d034 into master Dec 16, 2022
@edgao edgao deleted the suhomud/fix_format_1 branch December 16, 2022 21:43
@suhomud
Copy link
Contributor Author

suhomud commented Dec 19, 2022

@edgao @ryankfu, Thank you for taking care of it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues connectors/destination/bigquery connectors/destination/clickhouse connectors/destination/csv connectors/destination/elasticsearch connectors/destination/kafka connectors/destination/keen connectors/destination/local-json connectors/destination/mqtt connectors/destination/postgres connectors/destination/pulsar connectors/destination/redshift connectors/destination/s3-glue connectors/destination/s3 connectors/destination/snowflake connectors/destination/tidb connectors/source/babelforce connectors/source/bigcommerce connectors/source/bigquery connectors/source/chartmogul connectors/source/clickhouse connectors/source/close-com connectors/source/cockroachdb connectors/source/cockroachdb-strict-encrypt connectors/source/db2-strict-encrypt connectors/source/db2 connectors/source/delighted connectors/source/dynamodb connectors/source/e2e-test connectors/source/elasticsearch connectors/source/faker connectors/source/gitlab connectors/source/google-pagespeed-insights connectors/source/jdbc connectors/source/kafka connectors/source/metabase connectors/source/monday connectors/source/mongodb-strict-encrypt connectors/source/mongodb-v2 connectors/source/mssql connectors/source/mssql-strict-encrypt connectors/source/mysql connectors/source/mysql-strict-encrypt connectors/source/oracle connectors/source/oracle-strict-encrypt connectors/source/pexels-api connectors/source/plausible connectors/source/postgres connectors/source/redshift connectors/source/relational-db connectors/source/sftp connectors/source/shopify connectors/source/snowflake connectors/source/survey-sparrow connectors/source/surveycto connectors/source/surveymonkey connectors/source/tidb connectors/source/twilio-taskrouter connectors/source/zenloop
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants