Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce EnvironmentPostProcessor for R2DBC support in Cloud Sql #772

Merged
merged 27 commits into from
Jan 24, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
31d7ed8
include starter and samples for R2DBC; undo renaming of class
mpeddada1 Dec 6, 2021
7711250
improve documentation and cleanup samples
mpeddada1 Dec 7, 2021
92ca251
Build r2dbc url when database name and instance connection name are p…
mpeddada1 Dec 10, 2021
eeacab9
fix README; use webflux in samples
mpeddada1 Dec 11, 2021
37ee37c
improve check for postgreSQL and mySQL driver in post processor; clea…
mpeddada1 Dec 13, 2021
92b23e0
more clean up
mpeddada1 Dec 13, 2021
139fb92
fix comment in application.properties for postgreSQL sample
mpeddada1 Dec 13, 2021
99202b3
remove public modifier from test class
mpeddada1 Dec 13, 2021
dee8706
Combine R2dbc logic into CloudSqlEnvironmentPostProcessor
mpeddada1 Dec 14, 2021
5796bdf
Merge branch 'main' of github.com:GoogleCloudPlatform/spring-cloud-gc…
mpeddada1 Dec 14, 2021
8da6245
remove redundant else if condition
mpeddada1 Dec 14, 2021
2975a77
Revert "Combine R2dbc logic into CloudSqlEnvironmentPostProcessor"
mpeddada1 Dec 15, 2021
2394ad6
Introduce helper to gather properties
mpeddada1 Dec 15, 2021
a428256
remove unused class
mpeddada1 Dec 16, 2021
8bd52bf
check if ConnectionFactory is present in getEnabledDatabaseType(); cl…
mpeddada1 Dec 20, 2021
ac2334b
add more instructions to sample
mpeddada1 Dec 20, 2021
c8497b8
switch order in comment
mpeddada1 Dec 20, 2021
1fdf4c8
add test for extra condition
mpeddada1 Dec 20, 2021
a494df6
use Flux; only 2 constants in enum; use properties from pom.xml for m…
mpeddada1 Dec 28, 2021
c05e025
fix pom.xml
mpeddada1 Dec 28, 2021
1171bec
Merge branch 'main' of github.com:GoogleCloudPlatform/spring-cloud-gc…
mpeddada1 Dec 28, 2021
e9bab44
apply google java formatter
mpeddada1 Jan 21, 2022
29d1540
resolve merge conficts
mpeddada1 Jan 21, 2022
c44ca56
Merge branch 'main' of github.com:GoogleCloudPlatform/spring-cloud-gc…
mpeddada1 Jan 21, 2022
31e7ddf
more google formatting
mpeddada1 Jan 21, 2022
cf7d15c
Add property to enable r2dbc
mpeddada1 Jan 21, 2022
f4b758d
fix test
mpeddada1 Jan 21, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions spring-cloud-gcp-dependencies/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,11 @@
<artifactId>spring-cloud-gcp-starter-sql-postgresql</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-sql-r2dbc</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-trace</artifactId>
Expand Down Expand Up @@ -248,6 +253,19 @@
<version>${cloud-sql-socket-factory.version}</version>
</dependency>

<!-- spring-cloud-gcp-starter-sql-r2dbc -->
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>cloud-sql-connector-r2dbc-mysql</artifactId>
<version>${cloud-sql-socket-factory.version}</version>
</dependency>

<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>cloud-sql-connector-r2dbc-postgres</artifactId>
<version>${cloud-sql-socket-factory.version}</version>
</dependency>

<!--Google Cloud Platform Libraries BOM -->
<dependency>
<groupId>com.google.cloud</groupId>
Expand Down
2 changes: 2 additions & 0 deletions spring-cloud-gcp-samples/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@
<module>spring-cloud-gcp-integration-pubsub-json-sample</module>
<module>spring-cloud-gcp-security-iap-sample</module>
<module>spring-cloud-gcp-sql-postgres-sample</module>
<module>spring-cloud-gcp-sql-mysql-r2dbc-sample</module>
<module>spring-cloud-gcp-sql-postgres-r2dbc-sample</module>
<module>spring-cloud-gcp-vision-ocr-demo</module>
<module>spring-cloud-gcp-firestore-sample</module>
<module>spring-cloud-gcp-data-multi-sample</module>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
= Spring Cloud GCP MySQL Sample

This code sample demonstrates how to connect to a MySQL Google Cloud SQL instance using the link:../../spring-cloud-gcp-starters/spring-cloud-gcp-starter-sql-r2dbc[Spring Cloud GCP R2DBC Starter].

You will create an instance, a database within the instance, populate the database and then query it.

== Setup

image:http://gstatic.com/cloudssh/images/open-btn.svg[link=https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fspring-cloud-gcp&cloudshell_open_in_editor=spring-cloud-gcp-samples/spring-cloud-gcp-sql-mysql-sample/README.adoc]

1. Follow https://cloud.google.com/sql/docs/mysql/quickstart[these instructions] to set up a Google Cloud Project with billing enabled.

2. Create a Google Cloud MySQL instance from the https://console.cloud.google.com/sql/instances[Google Cloud Console Cloud SQL page].

3. Still within the Google Cloud Console SQL page, create a new database in the instance from the "Databases" section.
If you decided to set a root password for your instance footnoteref:[note,Google Cloud SQL allows for creating MySQL instances without a password,while still remaining safe,as the connection is using Cloud SDK authentication and encrypted communication: see docs on https://cloud.google.com/sql/docs/mysql/sql-proxy[Cloud SQL proxy] and https://cloud.google.com/sql/docs/mysql/connect-external-app#java[the java socketFactory],which is used in spring-cloud-gcp to create the connection!], then don't forget to take note of your root password, as you will need it in the next step!

4. Open the link:src/main/resources/application.properties[application.properties] file and replace the following:
- *[database-name]* with the name of the database you created in step 3.
- *[instance-connection-name]* with the instance name of the MySQL instance you created in step 1.
- *[username]* with the username for the instance you created in step 1. This should be in the "Users" section of your instance in the console.
+
If you set a root password, add the `spring.r2dbc.password` property with the root password as the value.
Note that the password can also be set in the connection url.
The template is as follows: `r2dbc:gcp:mysql://{username}:{password}@{instance-connection-name}/{database-name}`.

5. https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login[If you are authenticated in the Cloud SDK], your credentials will be automatically found by the Spring Boot Starter for Google Cloud SQL.

6. Run `$ mvn clean install` from the root directory of the project.

== Running the application

You can run the `SqlApplication` Spring Boot app by running the following command in the root directory of this sample app (spring-cloud-gcp-samples/spring-cloud-gcp-sql-mysql-sample):

`$ mvn spring-boot:run`

The database will be populated based on the link:src/main/resources/schema.sql[schema.sql] and link:src/main/resources/data.sql[data.sql] files.

When the application is up, navigate to http://localhost:8080/getTuples in your browser, or use the `Web Preview`
button in Cloud Shell to preview the app on port 8080. This will print the contents of the `users` table.

== Deploy to App Engine Standard Environment

If you have Cloud SDK installed, https://cloud.google.com/appengine/docs/standard/java11/testing-and-deploying-your-app[Maven App Engine Plug-in] can be used to deploy the application to App Engine Standard environment:

----
$ mvn clean package appengine:deploy -Dapp.deploy.projectId=$PROJECT_ID -Dapp.deploy.version=$VERSION
----
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<!-- Your own application should inherit from spring-boot-starter-parent -->
<artifactId>spring-cloud-gcp-samples</artifactId>
<groupId>com.google.cloud</groupId>
<version>2.0.7-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

<artifactId>spring-cloud-gcp-sql-mysql-r2dbc-sample</artifactId>
<name>Spring Cloud GCP Code Sample - R2DBC MySQL</name>

<!-- The Spring Cloud GCP BOM will manage spring-cloud-gcp version numbers for you. -->
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-dependencies</artifactId>
<version>${project.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-sql-r2dbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>io.r2dbc</groupId>
<artifactId>r2dbc-spi</artifactId>
<version>0.8.6.RELEASE</version>
</dependency>

<!-- Test-related dependencies. -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
runtime: java11
instance_class: F1

env_variables:
JAVA_TOOL_OPTIONS: "-XX:MaxRAM=256m -XX:ActiveProcessorCount=2 -Xmx32m"
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/*
* Copyright 2021-2022 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example;

import io.r2dbc.spi.ConnectionFactory;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.core.io.ClassPathResource;
import org.springframework.r2dbc.connection.init.ConnectionFactoryInitializer;
import org.springframework.r2dbc.connection.init.ResourceDatabasePopulator;

@SpringBootApplication
public class SqlApplication {

public static void main(String[] args) {
SpringApplication.run(SqlApplication.class, args);
}

@Bean
ConnectionFactoryInitializer initializer(ConnectionFactory connectionFactory) {
ConnectionFactoryInitializer initializer = new ConnectionFactoryInitializer();
initializer.setConnectionFactory(connectionFactory);
initializer.setDatabasePopulator(
new ResourceDatabasePopulator(new ClassPathResource("schema.sql"), new ClassPathResource("data.sql")));
initializer.setDatabaseCleaner(new ResourceDatabasePopulator(new ClassPathResource("cleanup.sql")));
return initializer;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/*
* Copyright 2021-2022 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example;

import java.util.List;

import io.r2dbc.spi.ConnectionFactory;
import reactor.core.publisher.Mono;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

/**
* Web app controller class for sample application. Contains a function that runs a query
* and displays the results.
*
*/
@RestController
public class WebController {

private final ConnectionFactory connectionFactory;

public WebController(ConnectionFactory connectionFactory) {
this.connectionFactory = connectionFactory;
}

@GetMapping("/getTuples")
public List<String> getTuples() {
return Mono.from(connectionFactory.create())
.flatMapMany(connection -> connection.createStatement("SELECT * FROM users").execute())
.flatMap(result -> result
.map((row, metadata) -> String.format("[%s, %s, %s]", row.get("EMAIL", String.class),
row.get("FIRST_NAME", String.class), row.get("LAST_NAME", String.class))))
.collectList().block();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
spring.r2dbc.url=r2dbc:gcp:mysql://username@instance-connection-name/database-name

# Uncomment if username is not specified in url
# spring.r2dbc.username=

# Uncomment if password is present and not specified in url
# spring.r2dbc.password=
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
DROP TABLE IF EXISTS users;
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
INSERT INTO users VALUES
('luisao@example.com', 'Anderson', 'Silva'),
('jonas@example.com', 'Jonas', 'Goncalves'),
('fejsa@example.com', 'Ljubomir', 'Fejsa');
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
CREATE TABLE users (
email VARCHAR(255),
first_name VARCHAR(255),
last_name VARCHAR(255),
PRIMARY KEY (email));
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/*
* Copyright 2021-2022 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.example;

import java.util.List;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIfSystemProperty;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;

import static org.assertj.core.api.Assertions.assertThat;

/**
* Simple integration test to verify the SQL sample application with MySQL.
*
*/
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT, classes = { SqlApplication.class }, properties = {
"spring.r2dbc.url=r2dbc:gcp:mysql://spring-cloud-gcp-ci:us-central1:testmysql/code_samples_r2dbc_db",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put this configuration in under the spring-cloud-gcp-ci-it profile like we did for the non-R2DBC sample?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The integration tests use different properties based on the database they're talking to. This might make it harder for them to be under the same profile? The non-R2DBC samples seem to use a hybrid approach where the mySQL test uses the profile while the postgreSQL test configures properties in the test. However, moving all the setup to the test classes could potentially help with readability. I could also be missing some maven-related knowledge here so let me know what you think.

"spring.r2dbc.username=root",
"spring.r2dbc.password=test"
})
@EnabledIfSystemProperty(named = "it.cloudsql", matches = "true")
public class SqlR2dbcMySqlSampleApplicationIntegrationTests {

@Autowired
private TestRestTemplate testRestTemplate;

@Test
void testSqlRowsAccess() {
ResponseEntity<List<String>> result = this.testRestTemplate.exchange(
"/getTuples", HttpMethod.GET, null, new ParameterizedTypeReference<List<String>>() {
});

assertThat(result.getBody()).containsExactlyInAnyOrder(
"[luisao@example.com, Anderson, Silva]",
"[jonas@example.com, Jonas, Goncalves]",
"[fejsa@example.com, Ljubomir, Fejsa]");
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
= Spring Cloud GCP MySQL Sample

This code sample demonstrates how to connect to a PostgreSQL Google Cloud SQL instance using the link:../../spring-cloud-gcp-starters/spring-cloud-gcp-starter-sql-r2dbc[Spring Cloud GCP R2DBC Starter].

You will create an instance, a database within the instance, populate the database and then query it.

== Setup

image:http://gstatic.com/cloudssh/images/open-btn.svg[link=https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fspring-cloud-gcp&cloudshell_open_in_editor=spring-cloud-gcp-samples/spring-cloud-gcp-sql-mysql-sample/README.adoc]

1. Follow https://cloud.google.com/sql/docs/mysql/quickstart[these instructions] to set up a Google Cloud Project with billing enabled.

2. Create a Google Cloud SQL instance from the https://console.cloud.google.com/sql/instances[Google Cloud Console Cloud SQL page].
Choose "Postgres" as your database engine.
You will be asked to set a password for the user; remember this value.

3. Still within the Google Cloud Console SQL page, create a new database in the instance from the "Databases" section.

4. Open the link:src/main/resources/application.properties[application.properties] file and replace the following:
- *[database-name]* with the name of the database you created in step 3.
- *[instance-connection-name]* with the instance name of the Postgres instance you created in step 1.
- *[username]* with the username for the instance you created in step 1. This should be in the "Users" section of your instance in the console.
+
If you set a root password, add the `spring.r2dbc.password` property with the root password as the value.
Note that the password can also be set in the connection url.
The template is: `r2dbc:gcp:postgres://{username}:{password}@{instance-connection-name}/{database-name}`.

5. https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login[If you are authenticated in the Cloud SDK], your credentials will be automatically found by the Spring Boot Starter for Google Cloud SQL.

6. Run `$ mvn clean install` from the root directory of the project.

== Running the application

You can run the `SqlApplication` Spring Boot app by running the following command in the root directory of this sample app (spring-cloud-gcp-samples/spring-cloud-gcp-sql-postgres-sample):

`$ mvn spring-boot:run`

The database will be populated based on the link:src/main/resources/schema.sql[schema.sql] and link:src/main/resources/data.sql[data.sql] files.

When the application is up, navigate to http://localhost:8080/getTuples in your browser, or use the `Web Preview`
button in Cloud Shell to preview the app on port 8080. This will print the contents of the `users` table.

== Deploy to App Engine Standard Environment

If you have Cloud SDK installed, https://cloud.google.com/appengine/docs/standard/java11/testing-and-deploying-your-app[Maven App Engine Plug-in] can be used to deploy the application to App Engine Standard environment:

----
$ mvn clean package appengine:deploy -Dapp.deploy.projectId=$PROJECT_ID -Dapp.deploy.version=$VERSION
----
Loading