How to make Tomcat automatically remove from nginx upstream before GC to avoid timeout?

Question

Tomcat will perform fullgc every once in a while. If nginx sends a request at this time, the request will time out.

One solution is to set a shorter timeout on the nginx side. If there is no response, change the machine to resend the request. However, the problem is that this request is a side effect and is not allowed to happen again.

I would like to ask, is there a mechanism whereby nginx automatically removes tomcat before gc, and then automatically adds it back after gc is completed?

In short, it is to avoid the timeout impact caused by tomcat gc~

PS:

nginx's upstream is set up with multiple tomcats. As long as one gc occurs, all requests falling on this machine will time out.

Answer 1

Tomcat has a listener to prevent memory overflow:

org.apache.catalina.core.JreMemoryLeakPreventionListener

I suspect it is caused by adjusting System.gc(), and this GC may be using the stop the world method, which means it is non-concurrent. You can try the following two options:

1.-XX:+ExplicitGCInvokesConcurrent concurrent GC
2. Disable the listener or disable direct calls from the JVM level -XX:+DisableExplicitGC

Then observe for a while.