Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure self-managed cluster credentials in properties #443

Closed
jonathanlukas opened this issue Jun 12, 2023 · 12 comments · Fixed by #662
Closed

Configure self-managed cluster credentials in properties #443

jonathanlukas opened this issue Jun 12, 2023 · 12 comments · Fixed by #662

Comments

@jonathanlukas
Copy link
Collaborator

Currently, only the gateway address of a self-managed cluster can be configured in properties.

As Camunda Platform 8 introduced a secured zeebe gateway by default in 8.2, this configuration should be applied by default in the future.

This is only possible by using environment variables at the moment:

https://github.com/camunda-community-hub/spring-zeebe/blob/main/spring-boot-starter-camunda/src/main/java/io/camunda/zeebe/spring/client/properties/ZeebeClientConfigurationProperties.java#L681

Here, client id and client secret are read from environment variables.

Actually, it is also required to configure an auth url (keycloak token endpoint).

This is part of the OAuthCredentialsProviderBuilder which also uses env variables to configure the auth url.

However, it could be set to the builder as well.

In the end, it should be possible to configure the whole client by using a spring boot configuration.

The required parameters are:

zeebe.client.broker.client-id:
zeebe.client.broker.client-secret:
zeebe.client.broker.auth-url:

To better align with the new principle of a unified camunda client, the properties could be renamed to:

camunda.client.client-id:
camunda.client.client-secret:
camunda.client.auth-url:

# These are off-topic, but should show alignment with the configuration options
camunda.client.zeebe.gateway-address:
camunda.client.operate.base-url:
camunda.client.tasklist.base-url:
...
@floric
Copy link

floric commented Jun 27, 2023

This is a pretty basic and important requirement for konsistent configuration. We found the way to configure the credentials with ZEEBE_CLIENT_ID and ZEEBE_CLIENT_SECRET only by reverse engeering the source code. I really hope this issue get's solved in the near future. These missing basic configuration properties and documentation make using Camunda 8 feel like an early Alpha non-production ready version. :(

EDIT: I'm still searching for the OAuth variable. Do you know how to set this one?

EDIT2: It might be OAUTH_ENV_AUTHORIZATION_SERVER, right?

@floric
Copy link

floric commented Jul 13, 2023

@jonathanlukas do you know if the variables mentioned above are right? We like to connect an application with an authenticated instance now and are not really keen for reverse engineering to configure it correctly. :(

@jonathanlukas
Copy link
Collaborator Author

@floric
Copy link

floric commented Jul 13, 2023

Yes, they are correct: To ease your reverse engineering, please review this line: https://github.com/camunda-community-hub/spring-zeebe/blob/main/spring-boot-starter-camunda/src/main/java/io/camunda/zeebe/spring/client/properties/ZeebeClientConfigurationProperties.java#L690

Thanks for your fast reply. Yes, I'm aware of these lines. :) I was more concerned regarding the remaining OAUTH part.

@floric
Copy link

floric commented Jul 13, 2023

Unfortunately no luck :(

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.camunda.zeebe.client.ZeebeClient]: Factory method 'zeebeClient' threw exception; nested exception is java.lang.NullPointerException: Cannot invoke "String.lastIndexOf(int)" because the return value of "io.camunda.zeebe.spring.client.properties.ZeebeClientConfigurationProperties$Broker.access$400(io.camunda.zeebe.spring.client.properties.ZeebeClientConfigurationProperties$Broker)" is null
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
        at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
        ... 69 more
Caused by: java.lang.NullPointerException: Cannot invoke "String.lastIndexOf(int)" because the return value of "io.camunda.zeebe.spring.client.properties.ZeebeClientConfigurationProperties$Broker.access$400(io.camunda.zeebe.spring.client.properties.ZeebeClientConfigurationProperties$Broker)" is null
        at io.camunda.zeebe.spring.client.properties.ZeebeClientConfigurationProperties.getCredentialsProvider(ZeebeClientConfigurationProperties.java:693)
        at io.camunda.zeebe.client.impl.ZeebeClientImpl.buildCallCredentials(ZeebeClientImpl.java:145)
        at io.camunda.zeebe.client.impl.ZeebeClientImpl.buildGatewayStub(ZeebeClientImpl.java:184)
        at io.camunda.zeebe.spring.client.configuration.ZeebeClientProdAutoConfiguration.zeebeClient(ZeebeClientProdAutoConfiguration.java:53)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
        ... 70 more

I currently have the following config:

ZEEBE_CLIENT_ID: my-client
ZEEBE_CLIENT_SECRET: abc
OAUTH_ENV_AUTHORIZATION_SERVER: https://my-instance/realms/my-realm
zeebe:
  client:
    broker:
      # k8s internal service
      gateway-address: http://camunda-platform-zeebe-gateway
    security:
      plaintext: false

It seems as if still the cloud branch is taken leading to the exception.

@jonathanlukas
Copy link
Collaborator Author

Alright. The only thing you should care about is that you do not use the zeebe.client.cloud.clientId and clientSecret but only the env variables

@floric
Copy link

floric commented Jul 13, 2023

Alright. The only thing you should care about is that you do not use the zeebe.client.cloud.clientId and clientSecret but only the env variables

I don't :D That's why I'm so surprised to receive this error. I checked the whole configuration.

@jonathanlukas
Copy link
Collaborator Author

jonathanlukas commented Jul 13, 2023

Could you remove the http:// part from your gateway-address? The used protocol is controlled by plaintext. Can you debug into the method getCredentialsProvider() and check whether your configuration is applied correctly?

Please note that here the broker gateway address could potentially be overridden by env

@floric
Copy link

floric commented Jul 13, 2023

Ah, the error was caused by a different scope where I accidently applied the zeebe client config. Setting it directly changed the error to the http issue mentioned by you.
Now I have an error about a missing audience. I'm confident I can solve it. I will try to sum up the successful configuration here and hope the docs will be updated accordingly for others in the future.

@floric
Copy link

floric commented Jul 13, 2023

Ok, we were successful with the following config:

Environment:

ZEEBE_CLIENT_ID: my-client
ZEEBE_CLIENT_SECRET: my-secret
ZEEBE_AUTHORIZATION_SERVER_URL: https://<my-keycloak>/realms/<my-realm>/protocol/openid-connect/token/
ZEEBE_TOKEN_AUDIENCE: zeebe-api
ZEEBE_CLIENT_CONFIG_PATH: /tmp/camunda_cred_cache

application.yaml:

zeebe:
  client:
    broker:
      gateway-address: camunda-platform-zeebe-gateway:26500
    security:
      plaintext: true

We needed to change ZEEBE_CLIENT_CONFIG_PATH because we run readonly FS and mount only /tmp with an emptyDir. I still would prefer if there would be an option to store the credentials only in memory, writing nothing to the FS.

Figuring out the correct format of ZEEBE_AUTHORIZATION_SERVER_URL to include a slash and lead to the Token endpoint took some time. Same for the needed port for the Gateway address.

Now we have the following log a lot: The request's security level does not guarantee that the credentials will be confidential.

Is this caused by not using SSL? This is really flooding the logs...

@rbcb-bedag
Copy link

I've made a workaround in my project, so that i can configure my own ZeebeClient like this:

Workaround

Dependencies:
java: 17 (temurin-17.0.8+7)
spring-boot-starter-parent: 3.1.2
spring-boot-starter-camunda: 8.2.4

application.yaml

zeebe:
  client:
    id: <id>
    secret: <secret>
    broker:
      gateway-address: <gateway-address>:443
    #security:
    #  plaintext: true
  token:
    audience: zeebe-api
  authorization-server-url: https://<URL>/auth/realms/camunda-platform/protocol/openid-connect/token/

ZeebeConfigurationProperties.java

import jakarta.validation.Valid;
import jakarta.validation.constraints.NotEmpty;
import jakarta.validation.constraints.NotNull;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;

@ConfigurationProperties(prefix = "zeebe")
@Validated
public record ZeebeConfigurationProperties(
        @Valid @NotNull(message = "client must not be null") Client client,
        @Valid @NotNull(message = "token must not be null") Token token,
        @NotEmpty(message = "authorizationServerUrl must not be empty") String authorizationServerUrl) {

    public record Client(@NotEmpty(message = "id must not be empty") String id,
                         @NotEmpty(message = "secret must not be empty") String secret,
                         @Valid @NotNull(message = "broker must not be null") Broker broker, @Valid Security security) {
        public record Broker(@NotEmpty(message = "gatewayAddress must not be empty") String gatewayAddress) {
        }

        public record Security(boolean plaintext) {
        }
    }

    public record Token(@NotEmpty String audience) {
    }
}

ZeebeConfig.java

import io.camunda.zeebe.client.ZeebeClient;
import io.camunda.zeebe.client.impl.oauth.OAuthCredentialsProvider;
import io.camunda.zeebe.client.impl.oauth.OAuthCredentialsProviderBuilder;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ZeebeConfig implements BeanPostProcessor {

    private final ZeebeConfigurationProperties zeebeConfigurationProperties;

    public ZeebeConfig(ZeebeConfigurationProperties zeebeConfigurationProperties) {
        this.zeebeConfigurationProperties = zeebeConfigurationProperties;
    }

    @Override
    public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        //replace ZeebeClient Bean with our own configuration
        if (bean instanceof ZeebeClient && "zeebeClient".equals(beanName)) {
            //https://docs.camunda.io/docs/apis-tools/java-client/
            OAuthCredentialsProvider credentialsProvider =
                    new OAuthCredentialsProviderBuilder()
                            .authorizationServerUrl(zeebeConfigurationProperties.authorizationServerUrl())
                            .audience(zeebeConfigurationProperties.token().audience())
                            .clientId(zeebeConfigurationProperties.client().id())
                            .clientSecret(zeebeConfigurationProperties.client().secret())
                            .build();

            return ZeebeClient.newClientBuilder()
                    .gatewayAddress(zeebeConfigurationProperties.client().broker().gatewayAddress())
                    .credentialsProvider(credentialsProvider)
                    .build();

        }
        return BeanPostProcessor.super.postProcessBeforeInitialization(bean, beanName);
    }
}

How does it work

With ZeebeConfigurationProperties we map the properties of the application.yaml to this config bean.
In the ZeebeConfig class we replace the ZeebeClient Bean provided by the spring-zeebe dependency with our own ZeebeClient, which is instanciated with the help of the previous ZeebeConfigurationProperties bean.

With this workaround we can now set the crendetials with the above application.yaml or still use ENV variables, because spring maps them automatically.

ZEEBE_CLIENT_ID=<id>
ZEEBE_CLIENT_SECRET=<secret>
ZEEBE_BROKER_GATEWAY_ADDRESS=<gateway-address>:443
ZEEBE_TOKEN_AUDIENCE=zeebe-api
ZEEBE_AUTHORIZATION_SERVER_URL=https://<URL>/auth/realms/camunda-platform/protocol/openid-connect/token/

Not sure what the zeebe.client.security.plaintext option does (if it's something to set on the client or otherwise used).

I hope something like these properties can make it to a newer version of the spring-zeebe library, so that we can configure the ZeebeClient for use in a self managed environment with identity component activated.

@jonathanlukas
Copy link
Collaborator Author

@rbcb-bedag any chance this could become a PR? Not the BeanPostProcessor obviously but the rest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants