Streaming dynamic server responses are buffered when testing with wrangler dev

[rmarscher]

It doesn't happen in production with a Cloudflare Worker or with waku dev. I found a related issue in the workers-sdk repo. It seems to be due to automatic compression of html and txt responses. It looks like there is a workaround - but I think we can wait for an upstream fix to #8004.

[rmarscher]

Another implementation of the workaround:
https://github.com/opennextjs/opennextjs-aws/pull/718/files#diff-1102925bd511dd8915dbfd18689c1a3351c17ceef436f0027a3caa0548edbc74R56-R61

[hamlim]

I remember seeing a similar workaround noted in the Hono docs for streaming as well, see callout below the code block here: https://hono.dev/docs/helpers/streaming#streamtext

[rmarscher]

There is more context in this comment.

I came up with a way to detect if the server is running on production Cloudflare Workers vs the wrangler dev command. It is tricky. There are only a few headers that are different so I picked the most reliable one. Another option could be to use an environment variable to decide if the Identity encoding header should be set.

We don't want to enable it in production. If we do, the responses aren't compressed. I verified in #1242 that production responses still have the zstd compression automatically added by Cloudflare but streaming works.