OpenAI br encoding issue with Spring AI
· 阅读需 3 分钟
If you encountered JSON error when using Spring AI to interact with OpenAI, remove br from Accept-Encoding header.
This can be done by configuring RestClient and WebClient used by OpenAiApi, see the code below. Accept-Encoding header is set to gzip, deflate.
Update Accept-Encoding header
OpenAiApi.builder()
.apiKey("<api-key>")
.restClientBuilder(RestClient.builder().defaultHeader(HttpHeaders.ACCEPT_ENCODING, "gzip, deflate"))
.webClientBuilder(WebClient.builder().defaultHeader(HttpHeaders.ACCEPT_ENCODING, "gzip, deflate"))
.build()
The exception stacktrace may look like below:
Stacktrace
org.springframework.web.client.RestClientException: Error while extracting response for type [org.springframework.ai.openai.api.OpenAiApi$ChatCompletion] and content type [application/json]
at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:262)
at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.readBody(DefaultRestClient.java:819)
at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.lambda$toEntityInternal$2(DefaultRestClient.java:775)
at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchangeInternal(DefaultRestClient.java:579)
at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchange(DefaultRestClient.java:533)
at org.springframework.web.client.RestClient$RequestHeadersSpec.exchange(RestClient.java:680)
at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.executeAndExtract(DefaultRestClient.java:814)
at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntityInternal(DefaultRestClient.java:774)
at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.toEntity(DefaultRestClient.java:763)
at org.springframework.ai.openai.api.OpenAiApi.chatCompletionEntity(OpenAiApi.java:198)
at org.springframework.ai.openai.OpenAiChatModel.lambda$internalCall$1(OpenAiChatModel.java:199)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:357)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:230)
at org.springframework.ai.openai.OpenAiChatModel.lambda$internalCall$3(OpenAiChatModel.java:199)
at io.micrometer.observation.Observation.observe(Observation.java:564)
at org.springframework.ai.openai.OpenAiChatModel.internalCall(OpenAiChatModel.java:196)
at org.springframework.ai.openai.OpenAiChatModel.call(OpenAiChatModel.java:181)
at org.springframework.ai.chat.client.advisor.ChatModelCallAdvisor.adviseCall(ChatModelCallAdvisor.java:54)
at org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.lambda$nextCall$1(DefaultAroundAdvisorChain.java:110)
at io.micrometer.observation.Observation.observe(Observation.java:564)
at org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.nextCall(DefaultAroundAdvisorChain.java:110)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:480)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:443)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293)
at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Unexpected end-of-input: expected close marker for Object (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:408)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:356)
at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:229)
... 158 common frames omitted
Caused by: com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 1])
at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 2]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportInvalidEOF(ParserMinimalBase.java:641)
at com.fasterxml.jackson.core.base.ParserBase._handleEOF(ParserBase.java:530)
at com.fasterxml.jackson.core.base.ParserBase._eofAsNextChar(ParserBase.java:547)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._skipWSOrEnd(UTF8StreamJsonParser.java:3066)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:716)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:181)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2131)
at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1501)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:397)
... 160 common frames omitted
The JSON parser only receives { as the input, which is the first char of the OpenAI response stream.
When RestClient tries to read HTTP responses with message converters, readWithMessageConverters method in DefaultRestClient, the original ClientHttpResponse is wrapped in an IntrospectingClientHttpResponse. IntrospectingClientHttpResponse detects if the wrapped ClientHttpResponse has an empty body.
In the hasEmptyMessageBody method, when the InputStream doesn't support mark and reset methods, it creates a PushbackInputStream to wrap the InputStream, reads a byte, then unreads this byte.
Unfortunately, when br encoding is enabled and backend returns a compressed stream, the actual InputStream used is a BrotliInputStream. BrotliInputStream doesn't work well with PushbackInputStream.
When br encoding is disabled, BrotliInputStream won't be used.