Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy che server Pod failed on openshfit OKD 3.11 with mulit user deployment #12033

Closed
Caplost opened this issue Nov 23, 2018 · 22 comments
Closed
Labels
kind/question Questions that haven't been identified as being feature requests or bugs.

Comments

@Caplost
Copy link

Caplost commented Nov 23, 2018

useing ./deploy_che.sh --multiuser command to auto deploy success

but eclipse/che-server:nightly image deploy fail.

it shows

Error injecting constructor, java.lang.RuntimeException: Exception while retrieving OpenId configuration from endpoint: http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration 
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:70)
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.class(KeycloakSettings.java:53)
  while locating org.eclipse.che.multiuser.keycloak.server.KeycloakSettings
...
Caused by: java.net.UnknownHostException: keycloak-eclipse-che.openshift.qycloud.com.cn
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
Full Che Server Logs
Error injecting constructor, java.lang.RuntimeException: Exception while retrieving OpenId configuration from endpoint: http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration 
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:70)
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.class(KeycloakSettings.java:53)
  while locating org.eclipse.che.multiuser.keycloak.server.KeycloakSettings
Caused by: java.lang.RuntimeException: Exception while retrieving OpenId configuration from endpoint: http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration 
	at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:104)
	at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings$$FastClassByGuice$$e0d0786b.newInstance(<generated>)
	at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89)
	at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111)
	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)
	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)
	at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
	at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)
	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
	at com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:205)
	at com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:199)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)
	at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:199)
	at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:180)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
	at com.google.inject.Guice.createInjector(Guice.java:99)
	at org.everrest.guice.servlet.EverrestGuiceContextListener.getInjector(EverrestGuiceContextListener.java:140)
	at com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
	at org.everrest.guice.servlet.EverrestGuiceContextListener.contextInitialized(EverrestGuiceContextListener.java:85)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:752)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:988)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1860)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: keycloak-eclipse-che.openshift.qycloud.com.cn
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at java.net.Socket.connect(Socket.java:538)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
	at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
	at sun.net.www.http.HttpClient.New(HttpClient.java:339)
	at sun.net.www.http.HttpClient.New(HttpClient.java:357)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
	at java.net.URL.openStream(URL.java:1045)
	at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:97)
	... 33 more
145) Error injecting constructor, java.lang.RuntimeException: Exception while retrieving OpenId configuration from endpoint: http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration 
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:70)
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.class(KeycloakSettings.java:53)
  while locating org.eclipse.che.multiuser.keycloak.server.KeycloakSettings
    for the 1st parameter of org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.<init>(KeycloakServiceClient.java:72)
  at org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient.class(KeycloakServiceClient.java:66)
  while locating org.eclipse.che.multiuser.keycloak.server.KeycloakServiceClient
    for the 1st parameter of org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.<init>(IdentityProviderConfigFactory.java:76)
  at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.class(IdentityProviderConfigFactory.java:58)
  while locating org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory
  while locating org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientConfigFactory
    for the 1st parameter of org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.<init>(OpenShiftClientFactory.java:70)
  at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.class(OpenShiftClientFactory.java:70)
  while locating org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory
    for the 4th parameter of org.eclipse.che.workspace.infrastructure.openshift.environment.OpenShiftEnvironmentFactory.<init>(OpenShiftEnvironmentFactory.java:82)
  while locating org.eclipse.che.workspace.infrastructure.openshift.environment.OpenShiftEnvironmentFactory
  while locating org.eclipse.che.api.workspace.server.spi.environment.InternalEnvironmentFactory annotated with @com.google.inject.multibindings.Element(setName=,uniqueId=65, type=MAPBINDER, keyType=java.lang.String)
  at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftInfraModule.configure(OpenShiftInfraModule.java:73) (via modules: org.eclipse.che.api.deploy.WsMasterModule -> org.eclipse.che.workspace.infrastructure.openshift.OpenShiftInfraModule -> com.google.inject.multibindings.MapBinder$RealMapBinder)
  while locating java.util.Map<java.lang.String, org.eclipse.che.api.workspace.server.spi.environment.InternalEnvironmentFactory>
    for the 2nd parameter of org.eclipse.che.api.workspace.server.WorkspaceRuntimes.<init>(WorkspaceRuntimes.java:145)
  at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.class(WorkspaceRuntimes.java:93)
  while locating org.eclipse.che.api.workspace.server.WorkspaceRuntimes
    for the 1st parameter of org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator.<init>(WorkspaceLinksGenerator.java:49)
  at org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator.class(WorkspaceLinksGenerator.java:49)
  while locating org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator
Caused by: java.lang.RuntimeException (same stack trace as error #31)
146) Error injecting constructor, java.lang.RuntimeException: Exception while retrieving OpenId configuration from endpoint: http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration 
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:70)
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.class(KeycloakSettings.java:53)
  while locating org.eclipse.che.multiuser.keycloak.server.KeycloakSettings
    for the 2nd parameter of org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.<init>(IdentityProviderConfigFactory.java:76)
  at org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory.class(IdentityProviderConfigFactory.java:58)
  while locating org.eclipse.che.workspace.infrastructure.openshift.multiuser.oauth.IdentityProviderConfigFactory
  while locating org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientConfigFactory
    for the 1st parameter of org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.<init>(OpenShiftClientFactory.java:70)
  at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory.class(OpenShiftClientFactory.java:70)
  while locating org.eclipse.che.workspace.infrastructure.openshift.OpenShiftClientFactory
    for the 4th parameter of org.eclipse.che.workspace.infrastructure.openshift.environment.OpenShiftEnvironmentFactory.<init>(OpenShiftEnvironmentFactory.java:82)
  while locating org.eclipse.che.workspace.infrastructure.openshift.environment.OpenShiftEnvironmentFactory
  while locating org.eclipse.che.api.workspace.server.spi.environment.InternalEnvironmentFactory annotated with @com.google.inject.multibindings.Element(setName=,uniqueId=65, type=MAPBINDER, keyType=java.lang.String)
  at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftInfraModule.configure(OpenShiftInfraModule.java:73) (via modules: org.eclipse.che.api.deploy.WsMasterModule -> org.eclipse.che.workspace.infrastructure.openshift.OpenShiftInfraModule -> com.google.inject.multibindings.MapBinder$RealMapBinder)
  while locating java.util.Map<java.lang.String, org.eclipse.che.api.workspace.server.spi.environment.InternalEnvironmentFactory>
    for the 2nd parameter of org.eclipse.che.api.workspace.server.WorkspaceRuntimes.<init>(WorkspaceRuntimes.java:145)
  at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.class(WorkspaceRuntimes.java:93)
  while locating org.eclipse.che.api.workspace.server.WorkspaceRuntimes
    for the 1st parameter of org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator.<init>(WorkspaceLinksGenerator.java:49)
  at org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator.class(WorkspaceLinksGenerator.java:49)
  while locating org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator
Caused by: java.lang.RuntimeException (same stack trace as error #32)
147) Error injecting constructor, java.lang.RuntimeException: Exception while retrieving OpenId configuration from endpoint: http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration 
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:70)
  at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.class(KeycloakSettings.java:53)
  while locating org.eclipse.che.multiuser.keycloak.server.KeycloakSettings
    for the 1st parameter of org.eclipse.che.multiuser.keycloak.server.KeycloakProfileRetriever.<init>(KeycloakProfileRetriever.java:39)
  at org.eclipse.che.multiuser.keycloak.server.KeycloakProfileRetriever.class(KeycloakProfileRetriever.java:32)
  while locating org.eclipse.che.multiuser.keycloak.server.KeycloakProfileRetriever
    for the 1st parameter of org.eclipse.che.multiuser.keycloak.server.dao.KeycloakProfileDao.<init>(KeycloakProfileDao.java:38)
  while locating org.eclipse.che.multiuser.keycloak.server.dao.KeycloakProfileDao
  while locating org.eclipse.che.api.user.server.spi.ProfileDao
    for the 2nd parameter of org.eclipse.che.multiuser.keycloak.server.KeycloakUserManager.<init>(KeycloakUserManager.java:58)
  at org.eclipse.che.multiuser.keycloak.server.KeycloakUserManager.class(KeycloakUserManager.java:58)
  while locating org.eclipse.che.multiuser.keycloak.server.KeycloakUserManager
  while locating org.eclipse.che.multiuser.api.account.personal.PersonalAccountUserManager
  while locating org.eclipse.che.api.user.server.UserManager
    for the 2nd parameter of org.eclipse.che.multiuser.machine.authentication.server.MachineTokenRegistry.<init>(MachineTokenRegistry.java:57)
  at org.eclipse.che.multiuser.machine.authentication.server.MachineTokenRegistry.class(MachineTokenRegistry.java:57)
  while locating org.eclipse.che.multiuser.machine.authentication.server.MachineTokenRegistry
    for the 2nd parameter of org.eclipse.che.multiuser.machine.authentication.server.MachineTokenProviderImpl.<init>(MachineTokenProviderImpl.java:46)
  at org.eclipse.che.multiuser.machine.authentication.server.MachineTokenProviderImpl.class(MachineTokenProviderImpl.java:46)
  while locating org.eclipse.che.multiuser.machine.authentication.server.MachineTokenProviderImpl
  while locating org.eclipse.che.api.workspace.server.token.MachineTokenProvider
    for the 1st parameter of org.eclipse.che.api.workspace.server.spi.provision.env.MachineTokenEnvVarProvider.<init>(MachineTokenEnvVarProvider.java:34)
  while locating org.eclipse.che.api.workspace.server.spi.provision.env.MachineTokenEnvVarProvider
  while locating org.eclipse.che.api.workspace.server.spi.provision.env.EnvVarProvider annotated with @com.google.inject.multibindings.Element(setName=,uniqueId=16, type=MULTIBINDER, keyType=)
  at org.eclipse.che.api.deploy.WsMasterModule.configure(WsMasterModule.java:174) (via modules: org.eclipse.che.api.deploy.WsMasterModule -> com.google.inject.multibindings.Multibinder$RealMultibinder)
  while locating java.util.Set<org.eclipse.che.api.workspace.server.spi.provision.env.EnvVarProvider>
    for the 1st parameter of org.eclipse.che.api.workspace.server.spi.provision.env.EnvVarEnvironmentProvisioner.<init>(EnvVarEnvironmentProvisioner.java:37)
  while locating org.eclipse.che.api.workspace.server.spi.provision.env.EnvVarEnvironmentProvisioner
  while locating org.eclipse.che.api.workspace.server.spi.provision.InternalEnvironmentProvisioner annotated with @com.google.inject.multibindings.Element(setName=,uniqueId=10, type=MULTIBINDER, keyType=)
  at org.eclipse.che.api.deploy.WsMasterModule.configure(WsMasterModule.java:167) (via modules: org.eclipse.che.api.deploy.WsMasterModule -> com.google.inject.multibindings.Multibinder$RealMultibinder)
  while locating java.util.Set<org.eclipse.che.api.workspace.server.spi.provision.InternalEnvironmentProvisioner>
    for the 3rd parameter of org.eclipse.che.workspace.infrastructure.openshift.OpenShiftInfrastructure.<init>(OpenShiftInfrastructure.java:53)
  at org.eclipse.che.workspace.infrastructure.openshift.OpenShiftInfrastructure.class(OpenShiftInfrastructure.java:37)
  while locating org.eclipse.che.workspace.infrastructure.openshift.OpenShiftInfrastructure
  while locating org.eclipse.che.api.workspace.server.spi.RuntimeInfrastructure
    for the 3rd parameter of org.eclipse.che.api.workspace.server.WorkspaceRuntimes.<init>(WorkspaceRuntimes.java:145)
  at org.eclipse.che.api.workspace.server.WorkspaceRuntimes.class(WorkspaceRuntimes.java:93)
  while locating org.eclipse.che.api.workspace.server.WorkspaceRuntimes
    for the 1st parameter of org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator.<init>(WorkspaceLinksGenerator.java:49)
  at org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator.class(WorkspaceLinksGenerator.java:49)
  while locating org.eclipse.che.api.workspace.server.WorkspaceLinksGenerator
Caused by: java.lang.RuntimeException (same stack trace as error #33)
147 errors
	at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:470)
	at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
	at com.google.inject.Guice.createInjector(Guice.java:99)
	at org.everrest.guice.servlet.EverrestGuiceContextListener.getInjector(EverrestGuiceContextListener.java:140)
	at com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
	at org.everrest.guice.servlet.EverrestGuiceContextListener.contextInitialized(EverrestGuiceContextListener.java:85)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:752)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:988)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1860)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2018-11-23 02:35:26,943[ost-startStop-1]  [ERROR] [o.a.c.core.StandardContext 5208]     - One or more listeners failed to start. Full details will be found in the appropriate container log file
2018-11-23 02:35:26,947[ost-startStop-1]  [ERROR] [o.a.c.core.StandardContext 5259]     - Context [/api] startup failed due to previous errors
2018-11-23 02:35:27,085[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1023]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/api.war] has finished in [33,145] ms
2018-11-23 02:35:27,099[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 959]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/docs.war]
2018-11-23 02:35:27,740[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1023]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/docs.war] has finished in [641] ms
2018-11-23 02:35:27,757[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 959]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/swagger.war]
2018-11-23 02:35:28,015[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1023]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/swagger.war] has finished in [257] ms
2018-11-23 02:35:28,027[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 959]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/workspace-loader.war]
2018-11-23 02:35:28,164[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1023]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/workspace-loader.war] has finished in [137] ms
2018-11-23 02:35:28,222[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 632]  - Starting ProtocolHandler ["http-nio-8080"]
2018-11-23 02:35:28,311[main]             [INFO ] [o.a.catalina.startup.Catalina 683]   - Server startup in 37594 ms
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-1,EclipseLinkCommandChannel,che-10-ztvwf-1786"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-1,RemoteSubscriptionChannel,che-10-ztvwf-17696"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-1,WorkspaceStateCache,che-10-ztvwf-35788"
Received SIGTERM
Stopping Che server running on localhost:8080
2018-11-23 02:36:04,833[main]             [INFO ] [o.a.c.core.StandardServer 524]       - A valid shutdown command was received via the shutdown port. Stopping the Server instance.
2018-11-23 02:36:04,836[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 658]  - Pausing ProtocolHandler ["http-nio-8080"]
2018-11-23 02:36:04,889[main]             [INFO ] [o.a.c.core.StandardService 477]      - Stopping service [Catalina]
2018-11-23 02:36:04,963[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 687]  - Stopping ProtocolHandler ["http-nio-8080"]
2018-11-23 02:36:04,966[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 707]  - Destroying ProtocolHandler ["http-nio-8080"]
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-3,EclipseLinkCommandChannel,che-10-ztvwf-1786"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-2,WorkspaceLocks,che-10-ztvwf-23638"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-3,RemoteSubscriptionChannel,che-10-ztvwf-17696"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-4,EclipseLinkCommandChannel,che-10-ztvwf-1786"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-3,WorkspaceStateCache,che-10-ztvwf-35788"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-2,RemoteSubscriptionChannel,che-10-ztvwf-17696"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-2,EclipseLinkCommandChannel,che-10-ztvwf-1786"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-3,WorkspaceLocks,che-10-ztvwf-23638"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-4,WorkspaceStateCache,che-10-ztvwf-35788"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-4,WorkspaceLocks,che-10-ztvwf-23638"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-4,RemoteSubscriptionChannel,che-10-ztvwf-17696"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-5,EclipseLinkCommandChannel,che-10-ztvwf-1786"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-5,WorkspaceStateCache,che-10-ztvwf-35788"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-5,RemoteSubscriptionChannel,che-10-ztvwf-17696"
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "INT-6,WorkspaceStateCache,che-10-ztvwf-35788"

The content of http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration

Keycloak Openid Configuration
{"issuer":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che","authorization_endpoint":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/protocol/openid-connect/auth","token_endpoint":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/protocol/openid-connect/token","token_introspection_endpoint":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/protocol/openid-connect/token/introspect","userinfo_endpoint":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/protocol/openid-connect/userinfo","end_session_endpoint":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/protocol/openid-connect/logout","jwks_uri":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/protocol/openid-connect/certs","check_session_iframe":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/protocol/openid-connect/login-status-iframe.html","grant_types_supported":["authorization_code","implicit","refresh_token","password","client_credentials"],"response_types_supported":["code","none","id_token","token","id_token token","code id_token","code token","code id_token token"],"subject_types_supported":["public","pairwise"],"id_token_signing_alg_values_supported":["RS256"],"userinfo_signing_alg_values_supported":["RS256"],"request_object_signing_alg_values_supported":["none","RS256"],"response_modes_supported":["query","fragment","form_post"],"registration_endpoint":"http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/clients-registrations/openid-connect","token_endpoint_auth_methods_supported":["private_key_jwt","client_secret_basic","client_secret_post"],"token_endpoint_auth_signing_alg_values_supported":["RS256"],"claims_supported":["sub","iss","auth_time","name","given_name","family_name","preferred_username","email"],"claim_types_supported":["normal"],"claims_parameter_supported":false,"scopes_supported":["openid","offline_access"],"request_parameter_supported":true,"request_uri_parameter_supported":true}

OS and version:
openshift okd : 3.11
docker version : 1.13

@Caplost Caplost changed the title Deploy che server Pod failed on openshfit with mulit user deployment Deploy che server Pod failed on openshfit OKD 3.11 with mulit user deployment Nov 23, 2018
@sleshchenko sleshchenko added the kind/bug Outline of a bug - must adhere to the bug report template. label Nov 23, 2018
@ghost
Copy link

ghost commented Nov 23, 2018

@Caplost are you behind a proxy? Che server cannot reach this url. Namely, it cannot resolve it

@ghost ghost added kind/question Questions that haven't been identified as being feature requests or bugs. and removed kind/bug Outline of a bug - must adhere to the bug report template. labels Nov 23, 2018
@Caplost
Copy link
Author

Caplost commented Nov 24, 2018

@eivantsov Thank you for your reply !

I have not use proxy ,keycloak access just use OKD router .

another thing I found is when it creating Che server container , access it use okd terminal run command curl http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration. it shows cannot resolve the url. But useing Debug in Terminal of okd pod curl command works ok.

is that means when creating che server container it cannot access internet ? run curl http://www.google.com also shows cannot resolve the url

I checked the dns /etc/resolv.conf it's the same as other pod which can access keycloak;

another things I miss ?

@Caplost
Copy link
Author

Caplost commented Nov 26, 2018

anyone deploy Eclipse Che on OKD 3.11 ?

@Caplost
Copy link
Author

Caplost commented Nov 26, 2018

@eivantsov single-user eclipse/che-server image deploy success on OKD 3.11

@ghost
Copy link

ghost commented Nov 26, 2018

Yes, Che is ok with OKD 3.11

I'd try just deploy some image like nginx, expose a service and curl its route from within the pod.

There's a networking issue that you need to solve.

@skabashnyuk
Copy link
Contributor

@Caplost are you using minishift? OS of the host?

@ghost
Copy link

ghost commented Nov 26, 2018

@skabashnyuk it's a public instance with a real DNS http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration

@Caplost i highly recommend running the test I have suggested. Let's take Che out of equation for a moment. Just deploy nginx, expose service and try to curl its route from within the pod.

@Caplost
Copy link
Author

Caplost commented Nov 27, 2018

@eivantsov I have run the test your suggested which just deploy nginx, and curl its route from within pod is ok with using okd default nginx image and any other url also can accessed by curl;

By the way , the OKD 3.11 which just newly inserted with its default setting by ansible, other images which deployed ok,route also can accessed from within it 。

is my test catch the point ?

@ghost
Copy link

ghost commented Nov 27, 2018

@Caplost all deployments and pods that are created during installation, do not differ from any other pods that you create.

I don;t know why it works for nginx and does not work for Keycloak. Can you post screenshots of a working curl route?

@Caplost
Copy link
Author

Caplost commented Nov 27, 2018

@eivantsov
in Container: che

/ $ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
3: eth0@if181: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP
link/ether 0a:58:0a:81:00:ac brd ff:ff:ff:ff:ff:ff
inet 10.129.0.172/23 brd 10.129.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::1c64:b1ff:fe7f:2be5/64 scope link
valid_lft forever preferred_lft forever
/ $ ip route
default via 10.129.0.1 dev eth0
10.128.0.0/14 dev eth0
10.129.0.0/23 dev eth0 scope link src 10.129.0.172
224.0.0.0/4 dev eth0
/ $
+++++++++++++++++++++++++++++++++++++++++
on the host where container che run ;
[root@node1 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:16:3e:0b:f6:c1 brd ff:ff:ff:ff:ff:ff
inet 10.0.14.175/20 brd 10.0.15.255 scope global noprefixroute dynamic eth0
valid_lft 315335732sec preferred_lft 315335732sec
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:47:59:27:44 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether c6:7f:6f:cb:dc:5d brd ff:ff:ff:ff:ff:ff
12: br0: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000
link/ether 1e:a4:c8:37:23:4e brd ff:ff:ff:ff:ff:ff
13: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65520 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
link/ether 5a:4a:21:32:12:0b brd ff:ff:ff:ff:ff:ff
14: tun0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether d2:9a:87:f7:33:b5 brd ff:ff:ff:ff:ff:ff
inet 10.129.0.1/23 brd 10.129.1.255 scope global tun0
valid_lft forever preferred_lft forever
15: veth9aaf4622@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether 7a:d5:38:c6:9f:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0
29: veth69f50e4f@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether 0e:ca:31:3d:41:dc brd ff:ff:ff:ff:ff:ff link-netnsid 3
35: veth405b1534@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether be:32:22:fb:83:b9 brd ff:ff:ff:ff:ff:ff link-netnsid 4
104: veth2870d309@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether 92:f9:37:8b:d5:43 brd ff:ff:ff:ff:ff:ff link-netnsid 2
115: veth849fe124@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether f2:e4:df:95:e2:4a brd ff:ff:ff:ff:ff:ff link-netnsid 5
139: vethe37ed1c6@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether 7e:90:87:e5:4f:08 brd ff:ff:ff:ff:ff:ff link-netnsid 7
145: veth1975b40a@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether ba:a5:25:22:eb:6d brd ff:ff:ff:ff:ff:ff link-netnsid 8
157: veth81cae02a@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether de:30:09:b9:6d:4d brd ff:ff:ff:ff:ff:ff link-netnsid 9
169: veth11455279@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether 66:6f:df:30:68:95 brd ff:ff:ff:ff:ff:ff link-netnsid 10
181: vethb9489e4b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP group default
link/ether ea:bc:40:7e:0d:01 brd ff:ff:ff:ff:ff:ff link-netnsid 11
[root@node1 ~]# ip route
default via 10.0.15.253 dev eth0 proto dhcp metric 100
10.0.0.0/20 dev eth0 proto kernel scope link src 10.0.14.175 metric 100
10.128.0.0/14 dev tun0 scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
172.30.0.0/16 dev tun0

@ghost
Copy link

ghost commented Nov 27, 2018

@Caplost i ask just for a confirmation that you can run other deployments and routes are accessible from within the pod

@Caplost
Copy link
Author

Caplost commented Nov 28, 2018

@eivantsov che-server cannot access any public url ,
but I change the link http://keycloak-eclipse-che.openshift.qycloud.com.cn/auth/realms/che/.well-known/openid-configuration to http://keycloak-eclipse-che.keycloak:5050/auth/realms/che/.well-known/openid-configuration which use service name the Che-server successfully started!
it seems like a networking issue which che-serve podr Can't access public network。

@ghost
Copy link

ghost commented Nov 28, 2018

It's just a mere k8s deployment. You need to find out what the issue is.

By the way, having a url with a service name won't work, since this url should be accessible to clients, ie browsers that users will open to access Che.

@Caplost
Copy link
Author

Caplost commented Nov 28, 2018

@eivantsov Thank you for your patience.

@Caplost Caplost closed this as completed Nov 28, 2018
@Caplost Caplost reopened this Nov 28, 2018
@ghost
Copy link

ghost commented Nov 28, 2018

@Caplost service name won't work, when you try to log in, unless your browser is in cluster network.

Here's deployment yaml for Che server: https://github.com/eclipse/che/blob/master/deploy/openshift/templates/che-server-template.yaml

Che has even not come into play at this point.

@Caplost
Copy link
Author

Caplost commented Nov 28, 2018

@eivantsov Should it use the url with a service name just at che-server get auth from keycloak server ?
the keycloak pod works fine, browsers auth use public url of keycloak ruote ?

@ghost
Copy link

ghost commented Nov 28, 2018

Yes, I was thinking about it too. But you have got connectivity issues in your cluster anyway, it does not solve the initial problem

@Caplost
Copy link
Author

Caplost commented Nov 28, 2018

I don't know why it works for other pods which can curl public network , just che-server not 😩

@Caplost Caplost closed this as completed Nov 28, 2018
@Caplost
Copy link
Author

Caplost commented Jan 18, 2019

@eivantsov I try eclipse/che-server with tag nightly-centos it works ok ,but eclipse/che-server:nightly still not work.

@fishingwind
Copy link

fishingwind commented Mar 15, 2019

I don't know why it works for other pods which can curl public network , just che-server not

Hi, Caplost
Did you resolve your problem? I'm facing the same exception. The retrieving url has no port, which means 80 and is not accessable. If I curl with port 8080 it's OK.

@Caplost
Copy link
Author

Caplost commented Apr 28, 2019

@fishingwind you can change eclipse/che-server:nightly which is default docker image to eclipse/che-server:nightly-centos ,it may the docker image problem.

@SockenSalat
Copy link

I expierence the same problem on our test cluster which is version 17.3 and has been setup using kubeadm.

fetching the said URL with curl works fine from containers deployed on the same node.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/question Questions that haven't been identified as being feature requests or bugs.
Projects
None yet
Development

No branches or pull requests

5 participants