feat: Procedures sdk update #3255

sfc-gh-asawicki · 2024-12-09T11:30:53Z

Update and test procedures SDK as part of ongoing functions&procedures rework:

Regenerate SDK after adjusting defs
Wrap procedure definitions in $$
Wrap arguments in double quotes
Extract ProcedureDetails out of DESCRIBE output rows
Add AutoEventLogging to the SDK
Adjust alter's structure (add set/unset)
Add missing fields to SHOW output
Use extended in for SHOW
Add/Generate assertions for procedure, procedure details, and procedure parameters
Add missing unit tests and adjust existing ones
Add missing integration tests and temporarily skip most of the existing ones, adjust them, or remove them
Adjust existing resource

sfc-gh-jcieslak · 2024-12-09T12:42:32Z

pkg/sdk/procedures_ext.go

+	NullHandling               *string // present for all procedure types but SQL
+	Body                       *string // present for all procedure types (hidden when SECURE)
+	Volatility                 *string // present for all procedure types but SQL
+	ExternalAccessIntegrations *string // list present for python, java, and scala


Will be adjusted later on to concrete types instead of raw strings?

yes, most probably when adding the resources

sfc-gh-jcieslak · 2024-12-09T12:43:05Z

pkg/sdk/procedures_ext.go

+}
+
+func (d *ProcedureDetail) setOptionalBoolValueOrError(property string, field **bool) error {
+	if d.Value != nil && *d.Value != "" {


will be extracted for later use in other more complex describe outputs?

most probably, depending on the future shape of ProcedureDetails

pkg/sdk/procedures_gen_test.go

pkg/sdk/testint/procedures_integration_test.go

github-actions · 2024-12-09T13:55:03Z

Integration tests failure for bc25a33f04932d9ef896ab02ed4299d1017427bd

sfc-gh-jmichalak

I added a few comms, but my comments from #3254 apply here as well.

pkg/sdk/procedures_def.go

sfc-gh-jmichalak · 2024-12-09T13:50:37Z

pkg/sdk/procedures_gen_test.go

@@ -206,7 +206,7 @@ func TestProcedures_CreateForJava(t *testing.T) {
 		opts.Comment = String("test comment")
 		opts.ExecuteAs = ExecuteAsPointer(ExecuteAsCaller)
 		opts.ProcedureDefinition = String("return id + name;")
-		assertOptsValidAndSQLEquals(t, opts, `CREATE OR REPLACE SECURE PROCEDURE %s (id NUMBER, name VARCHAR DEFAULT 'test') COPY GRANTS RETURNS TABLE (country_code VARCHAR) LANGUAGE JAVA RUNTIME_VERSION = '1.8' PACKAGES = ('com.snowflake:snowpark:1.2.0') IMPORTS = ('test_jar.jar') HANDLER = 'TestFunc.echoVarchar' EXTERNAL_ACCESS_INTEGRATIONS = ("ext_integration") SECRETS = ('variable1' = %s, 'variable2' = %s) TARGET_PATH = '@~/testfunc.jar' STRICT COMMENT = 'test comment' EXECUTE AS CALLER AS 'return id + name;'`, id.FullyQualifiedName(), secretId.FullyQualifiedName(), secretId2.FullyQualifiedName())
+		assertOptsValidAndSQLEquals(t, opts, `CREATE OR REPLACE SECURE PROCEDURE %s ("id" NUMBER, "name" VARCHAR DEFAULT 'test') COPY GRANTS RETURNS TABLE ("country_code" VARCHAR) LANGUAGE JAVA STRICT RUNTIME_VERSION = '1.8' PACKAGES = ('com.snowflake:snowpark:1.2.0') IMPORTS = ('test_jar.jar') HANDLER = 'TestFunc.echoVarchar' EXTERNAL_ACCESS_INTEGRATIONS = ("ext_integration") SECRETS = ('variable1' = %s, 'variable2' = %s) TARGET_PATH = '@~/testfunc.jar' COMMENT = 'test comment' EXECUTE AS CALLER AS return id + name;`, id.FullyQualifiedName(), secretId.FullyQualifiedName(), secretId2.FullyQualifiedName())


Tthe change in casing should be mentioned in the migration guide.

It will, in the next PRs (when resource will be reworked/split).

pkg/sdk/procedures_gen_test.go

github-actions · 2024-12-10T03:06:24Z

Integration tests failure for d39a768d8864b194b9e525ace7e3c98dca44860f

github-actions · 2024-12-10T15:04:05Z

Integration tests failure for 152f7b6ae84b2e93965161fd7128b7f6f9750c14

🤖 I have created a release *beep* *boop* --- ## [0.100.0](v0.99.0...v0.100.0) (2024-12-12) ### 🎉 **What's new:** * Account v1 readiness ([#3236](#3236)) ([5df33a8](5df33a8)) * Account v1 readiness generated files ([#3242](#3242)) ([3df59dd](3df59dd)) * Account v1 readiness resource ([#3252](#3252)) ([8f5698d](8f5698d)) * Add a new account roles data source ([#3257](#3257)) ([b3d6b9e](b3d6b9e)) * Add account data source ([#3261](#3261)) ([6087fc9](6087fc9)) * Add all other functions and procedures implementations ([#3275](#3275)) ([7a6f68d](7a6f68d)) * Basic functions implementation ([#3269](#3269)) ([6d4a103](6d4a103)) * Basic procedures implementation ([#3271](#3271)) ([933335f](933335f)) * Docs, test, and missing parameter ([#3280](#3280)) ([10517f3](10517f3)) * Functions and procedures schemas and generated sources ([#3262](#3262)) ([9b70f87](9b70f87)) * Functions sdk update ([#3254](#3254)) ([fc1eace](fc1eace)) * Handle missing fields in function and procedure ([#3273](#3273)) ([53e7a0a](53e7a0a)) * Procedures schemas and generated sources ([#3263](#3263)) ([211ad46](211ad46)) * Procedures sdk update ([#3255](#3255)) ([682606a](682606a)) * Rework account parameter resource ([#3264](#3264)) ([15aa9c2](15aa9c2)) * Rework data types ([#3244](#3244)) ([05ada91](05ada91)) * support table data type ([#3274](#3274)) ([13401d5](13401d5)) * Tag association v1 readiness ([#3210](#3210)) ([04f6d54](04f6d54)) * Test imports and small fixes ([#3276](#3276)) ([a712195](a712195)) * Unsafe execute v1 readiness ([#3266](#3266)) ([c4f1e8f](c4f1e8f)) * Use new data types in sql builder for functions and procedures ([#3247](#3247)) ([69f677a](69f677a)) ### 🔧 **Misc** * Add ShowByID filtering generation ([#3227](#3227)) ([548ec42](548ec42)) * Adress small task-related todos ([#3243](#3243)) ([40de9ae](40de9ae)) * Apply masking ([#3234](#3234)) ([c209a8a](c209a8a)) * fix missing references in toOpts and changes with newlines ([#3240](#3240)) ([246547f](246547f)) * function tests ([#3279](#3279)) ([5af6efb](5af6efb)) * Improve config builders ([#3207](#3207)) ([425787c](425787c)) * Revert to proper env ([#3238](#3238)) ([5d4ed3b](5d4ed3b)) * Use service user for ci ([#3228](#3228)) ([2fb50d7](2fb50d7)) ### 🐛 **Bug fixes:** * Make blocked_roles_field optional in OAuth security integrations ([#3267](#3267)) ([7197b57](7197b57)) * Minor fixes ([#3231](#3231)) ([1863bf6](1863bf6)) * Minor fixes 2 ([#3230](#3230)) ([73b7e74](73b7e74)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snowflake-release-please[bot] <105954990+snowflake-release-please[bot]@users.noreply.github.com>

sfc-gh-asawicki requested review from sfc-gh-jcieslak and sfc-gh-jmichalak December 9, 2024 11:50

sfc-gh-asawicki marked this pull request as ready for review December 9, 2024 11:50

sfc-gh-jcieslak reviewed Dec 9, 2024

View reviewed changes

sfc-gh-jmichalak reviewed Dec 9, 2024

View reviewed changes

Base automatically changed from functions-sdk-update to main December 9, 2024 17:11

sfc-gh-asawicki added 23 commits December 9, 2024 18:15

procedures SDK update starts here

fa3e3b9

Rename main procedure integration test

e4a2837

Add procedure parameters, assertions, and missing enum

7311812

Update procedure definition

2be47af

Prepare procedure details

cbbe0bf

Add wrapping to procedure bodies

f64cc09

Fix procedure impl

2f11fc3

Regenerate procedures - validations ready

bca99de

Regenerate procedures - every file besides unit tests

ce3b8d0

Update procedure unit tests

c221248

Generate object assertions for procedures

b535a53

Add procedure details assertions

78b85f4

Pass integration test for procedure

68fc2a7

Pass integration test for full inline java procedure

ecf5362

Introduce proper ordering for null input behavior

8f1c7c8

Test java staged procedures

322dc1d

Test javascript procedure creation

87e6321

Add tests for python, scala, and SQL

77cfa05

Skip old tests

1d6d67f

Pass alter, describe, and drop tests

d5f6359

Pass show tests

df81993

Fill out TODO

2073a57

Fix test

ef6d5dd

sfc-gh-asawicki added 7 commits December 9, 2024 18:15

Run pre-push

9893268

Remove done todo

85045db

Add todos from the review comments

b825418

Fix function and procedure resources

aaa97ec

Add better assertions for external access integrations and secrets

ffe8395

Fix after review and fix tests

f21c3a3

Fix create and call procedure (quickfix)

d39a768

sfc-gh-asawicki force-pushed the procedures-sdk-update branch from bc25a33 to d39a768 Compare December 10, 2024 00:48

sfc-gh-asawicki requested review from sfc-gh-jmichalak and sfc-gh-jcieslak December 10, 2024 00:49

sfc-gh-jcieslak previously approved these changes Dec 10, 2024

View reviewed changes

Fix temporary

152f7b6

sfc-gh-asawicki dismissed sfc-gh-jcieslak’s stale review via 152f7b6 December 10, 2024 12:43

sfc-gh-jcieslak approved these changes Dec 10, 2024

View reviewed changes

sfc-gh-jmichalak approved these changes Dec 10, 2024

View reviewed changes

sfc-gh-asawicki merged commit 682606a into main Dec 10, 2024
8 of 9 checks passed

sfc-gh-asawicki deleted the procedures-sdk-update branch December 10, 2024 15:39

snowflake-release-please bot mentioned this pull request Dec 10, 2024

chore(main): release 0.100.0 #3237

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Procedures sdk update #3255

feat: Procedures sdk update #3255

sfc-gh-asawicki commented Dec 9, 2024 •

edited

Loading

sfc-gh-jcieslak Dec 9, 2024

sfc-gh-asawicki Dec 9, 2024

sfc-gh-jcieslak Dec 9, 2024

sfc-gh-asawicki Dec 9, 2024

github-actions bot commented Dec 9, 2024

sfc-gh-jmichalak left a comment

sfc-gh-jmichalak Dec 9, 2024

sfc-gh-asawicki Dec 9, 2024

github-actions bot commented Dec 10, 2024

github-actions bot commented Dec 10, 2024

feat: Procedures sdk update #3255

feat: Procedures sdk update #3255

Conversation

sfc-gh-asawicki commented Dec 9, 2024 • edited Loading

sfc-gh-jcieslak Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-asawicki Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-jcieslak Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-asawicki Dec 9, 2024

Choose a reason for hiding this comment

github-actions bot commented Dec 9, 2024

sfc-gh-jmichalak left a comment

Choose a reason for hiding this comment

sfc-gh-jmichalak Dec 9, 2024

Choose a reason for hiding this comment

sfc-gh-asawicki Dec 9, 2024

Choose a reason for hiding this comment

github-actions bot commented Dec 10, 2024

github-actions bot commented Dec 10, 2024

sfc-gh-asawicki commented Dec 9, 2024 •

edited

Loading