Tomcat is serving web page response content extremely slow

[sisqo]

After upgrading from spring-boot 2.2.7. to spring boot 2.3.3 we are experiencing extreme slowness in receiving web content in the production environment.

The document of our application is 30kb and is downloaded in over 8.9 sec. Normally it takes about 200 ms.

With version 2.3.3. every request experience this issue, going back to version 2.2.7. the problem disappears.

The problem occurs only in the production environment (Google Cloud VM), while in local we are unable to reproduce it.

I attach the chrome inspection page that verify the problem

[wilkinsona]

Thanks for the report.

Unfortunately, it's not clear why you believe that this is a problem with Spring Boot itself. You also haven't described what sort of content you're serving, whether it's generated by a controller, static content served by Spring MVC, static content served by Tomcat, or something else.

Given that we haven't heard of similar problems from anyone else, I think it's highly likely that the problem is specific to your application and/or the deployment environment. Perhaps you could spend some time describing exactly how the content is being served and narrowing down the possible causes of the problem by trying different versions of Tomcat, etc.

[sisqo]

Thanks Andy for your feedback, you are absolutely right I try to go into more detail.

The application where the problem occurs is a spring-boot MVC application with embedded Tomcat. The calls are standard get calls handled with @GetMapping by a @Controller.
The controller returns the name of the .jsp page we use to render the content.

The problem occurs in the download time of the document, and this happens with all controllers and with all calls of our application. For example, this is the home page eg: https://www.vino75.com (at the moment we are up with version 2.2.7 and the problem does not appear).

We are thinking that the problem is spring-boot because this is the only difference between deploy and with and without the problem: we have tried to deploy the application several times by changing only the version of spring-boot. With version 2.2.7 the problem does not arise while with 2.3.3 it occurs systematically.

I attach a screenshot of when the content is downloaded successfully.

Let me know if I can provide more information to better investigate the problem.

[philwebb]

Without a way to replicate things locally it's very hard to say what the problem might be. JSP rendering is usually handled directly by Tomcat, and Spring Boot doesn't really get involved that much. Can you try with Spring Boot v2.2.9.RELEASE just to see if it's a Tomcat upgrade that caused the issue?

If that works, we need to find a way to replicate the issue or get some more server-side logs that could shed some light on the cause.

[sisqo]

We have upgraded from Spring Boot v2.2.7.RELEASE to v2.2.9.RELEASE and the issue is not present.

[philwebb]

Are you able to create a minimal sample that you can share that fails in the same way when you deploy it to Google Cloud?

[sisqo]

We are unable to reproduce the problem, everything works fine locally.

[wilkinsona]

We understand that. We're interested in a minimal sample that reproduces the problem when deployed to Google Cloud.

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-projects-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

[glennvdv]

We are experiencing a similair problem. We upgraded from 2.2.5 to 2.3.4 and now in our production environment we are experiencing performance issues. Most Pages are 2 seconds slower then before the upgrade. We have Java melody for monitoring and there we can see that jsp are using high cpu and slower rendering. Unfortunately we haven't been able tot reproduce this locally.
Our app is packaged as a war (because of jsp) and are using embedded tomcat.

[asafbart]

same issue here also, upgrade 2.2.6 to 2.3.3, 10 times slower response time 5ms vs 50 ms for very simple configuration

[snicoll]

@asafbart we've requested a minimal sample that we can use to reproduce the problem. Can you please do that in the form of a sample project that works with 2.2.6 and is much slower by only changing the version to 2.3.3? You can do so by attaching a zip to this issue or share a link to a GitHub repository. Thank you.

[asafbart]

@snicoll found the issue. it was an import which used a library in class path which changes autoconfig and does not officially support this version of spring boot

[sisqo]

@snicoll found the issue. it was an import which used a library called jasypt-spring-boot and does not officially support this version spring

@asafbart thank you for your feedback, unfortunately it does not completely solve the case, in our project we don't have jasypt-spring-boot library

[asafbart]

@sisqo Probably unrelated, but generally my lesson is to see if some library on your classpath might affect the AutoConfiguration... but yeah maybe unrelated to you

[sisqo]

Issue finally solved!!! This is the complete explanation:
https://vivekvara.medium.com/jsp-performance-degrades-from-spring-boot-2-3-x-for-undefined-attributes-8cb434f743c2#_=_

[wilkinsona]

Thanks for following up, @sisqo. I don't understand why the EL implementation would only cause a problem on Google Cloud, but I'm glad that you got to the bottom of it. Your problem's a duplicate of #24744 which is the issue where we switched to the faster Apache EL implementation in 2.5.

[sisqo]

Thanks for following up, @sisqo. I don't understand why the EL implementation would only cause a problem on Google Cloud, but I'm glad that you got to the bottom of it. Your problem's a duplicate of #24744 which is the issue where we switched to the faster Apache EL implementation in 2.5.

It's not related to Google Cloud, now is clear. We were misled by the fact that this problem only occurred in production, but it was due to the fact that some parts of jsp were being processed in production that were obviously involved.
Thank you for your feedback and linking to the duplicate case.