本文介紹了Spring Boot隨機SSLException:在帶有JDK11的Kubernetes中重置連接的處理方法,對大家解決問題具有一定的參考價值,需要的朋友們下面隨著小編來一起學習吧!
問題描述
上下文:
我們有一個Spring Boot(2.3.1.RELEASE)Web應用程序
它是用Java 8編寫的,但在使用Java 11(openjdk:11.0.6-jre-stretch
)的容器中運行。
它有一個數據庫連接和一個通過HTTPS(簡單RestTemplate#交換方法)調用的上游服務(這很重要!)
部署在Kubernetes集群內(不確定這是否重要)
問題:
每天,我都會看到一小部分針對上游服務的請求失敗,錯誤為:I/O error on GET request for "https://upstream.xyz/path": Connection reset; nested exception is javax.net.ssl.SSLException: Connection reset
錯誤完全是隨機的,并且間歇性地發生。
我們遇到過與JRE11和TLS 1.3協商問題相關的類似錯誤(javax.net.ssl.SSLProtocolException: Connection reset
)。我們已將Docker映像更新為上述內容,并已修復該問題。
這是來自錯誤的堆棧跟蹤:
java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(Unknown Source)
at java.base/java.net.SocketInputStream.read(Unknown Source)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source)
at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source)
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:87)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:739)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:674)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:583)
....
配置:
public static RestTemplate create(final int maxTotal, final int defaultMaxPerRoute,
final int connectTimeout, final int readTimeout,
final String userAgent) {
final Registry<ConnectionSocketFactory> schemeRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", SSLConnectionSocketFactory.getSocketFactory())
.build();
final PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager(schemeRegistry);
connManager.setMaxTotal(maxTotal);
connManager.setDefaultMaxPerRoute(defaultMaxPerRoute);
final CloseableHttpClient httpClient = HttpClients.custom()
.setConnectionManager(connManager)
.setUserAgent(userAgent)
.setDefaultRequestConfig(RequestConfig.custom()
.setConnectTimeout(connectTimeout)
.setSocketTimeout(readTimeout)
.setExpectContinueEnabled(false).build())
.build();
return new RestTemplateBuilder()
.requestFactory(() -> new HttpComponentsClientHttpRequestFactory(httpClient))
.build();
}
有人遇到過這個問題嗎?
當我打開http客戶端上的調試日志時,它充滿了噪音,我無法辨別出任何有用的東西…
推薦答案
我們在遷移到aws/kubernetes時遇到了類似的問題。
我已經找到原因了。
您正在使用連接池。PoolingHttpClientConnectionManager的默認行為是它將重復使用連接。因此,當您的請求完成時,連接不會立即關閉。這將節省資源,因為不必一直重新連接。
Kubernetes集群使用NAT(網絡地址轉換)進行傳出連接。當某個連接在一段時間內未使用時,該連接將從NAT表中移除,并且該連接將被斷開。這會導致看似隨機的SSLExceptions。
在AWS上,當NAT表處于空閑狀態350秒時,連接將從NAT表中刪除。其他Kubernetes實例可能有其他設置。
參見https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway-troubleshooting.html
解決方案:
禁用連接重用:
final CloseableHttpClient closeableHttpClient = HttpClients.custom()
.setConnectionReuseStrategy(NoConnectionReuseStrategy.INSTANCE)
.setConnectionManager(poolingHttpClientConnectionManager)
.build();
或,讓httpClient驅逐空閑時間過長的連接:
return HttpClients.custom()
.evictIdleConnections(300, TimeUnit.SECONDS) //Read the javadocs, may not be used when the instance of HttpClient is created inside an EJB container.
.setConnectionManager(poolingHttpClientConnectionManager)
.build();
或使用永遠不返回-1或超時值超過300秒的自定義KeepAliveStrategy調用setConnectionKeepAliveStrategy(....)
。
這篇關于Spring Boot隨機SSLException:在帶有JDK11的Kubernetes中重置連接的文章就介紹到這了,希望我們推薦的答案對大家有所幫助,