Duplicate "Transfer-Encoding: chunked" header

[JanCizmar]

I am getting duplicate Transfer-Encoding: chunked headers, which causes my Nginx Ingress Controller to return 502. Weird thing about this: it starts happening randomly after the spring boot instance runs for some time. When the specific pod is restarted, it works fine again.

Full error message from Nginx Ingress controller:

2023/10/01 19:00:12 [error] 1275#1275: *631585 upstream sent duplicate header line: "Transfer-Encoding: chunked", previous value: "Transfer-Encoding: chunked" while reading response header from upstream, client: xxx, server: xxx, request: "GET /v2/projects/export?languages=de&format=JSON&zip=false HTTP/1.1", upstream: "http://10.224.1.48:8080/v2/projects/export?languages=de&format=JSON&zip=false", host: "app.tolgee.io"
194.74.103.210 - - [01/Oct/2023:18:59:47 +0000] "GET /v2/projects/xxx/translations/en HTTP/1.1" 502 559 "http://localhost:5173/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36" "-"   

The sensitive handler method 1 code. The sensitive handler method 2 code

I am not sure whether it only happens to these 2 endpoints. They are executed very frequently and have bigger responses.

SpringBoot version: 2.7.13
Nginx.com Ingress version: 0.18.1

(I don't think it's related to this), since I don't work with response entities returned from http client.

[wilkinsona]

Thanks for the report. Unfortunately, it's not clear to me why you believe this is a Spring (Boot) issue.

As explained in the Framework issue to which you have already linked, the Transfer-Encoding header is set by the underlying web server and Spring (Boot and Framework) never set the header as far as we know. Another possibility is that the header is being set by application code. You've linked to a couple of handler methods that look fine, but perhaps there is a filter or some other component in your application that is introducing the problem?

If you believe that Spring is the cause of the problem that you're seeing and you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue.

[JanCizmar]

I understand. I just hope someone can point me to right direction. I don't think we are touching the headers anywhere else in the app code, and further investigation is not trivial since the errors starts occurring for unknown reason only in the production environment.

Any tips, how to debug this or what can cause this are appreciated.

[wilkinsona]

Perhaps you could add a Filter, positioned first in the chain, that checks the response headers after calling doFilter. If there are no Transfer-Encoding headers in the response at this point, it would suggest that the web server is setting the header twice. If there are one or more Transfer-Encoding headers, that would suggest it's Spring or application code that is setting the header.

[JanCizmar]

So I've added such debug logging filter, and I can confirm, we are not adding the header in the application code. So it has to be added after the filter chain is done.

[wilkinsona]

Thanks for trying the filter. That points towards the web server being the most likely cause of the problem. I suspect you're using Tomcat so I think your next step should be to raise this with Tomcat's maintainers. I'd start with the users mailing list, asking for some further advice on how to narrow down the problem.

[itsayopapi]

@JanCizmar can you please elaborate more for me on what needs to be fixed please