The Problem
We recently encountered a frustrating issue with HTTP response streaming at Mintlify. Our system uses the AI SDK with the Node stream API to forward streams, and suddenly things stopped working properly. The symptoms were confusing: streaming worked perfectly with cURL and Postman, but failed completely with node-fetch and browser fetch.
Initial Investigation
Our first hypothesis centered around stream compatibility issues. We suspected the problem might be related to how the AI SDK responds using web streams versus Node streams, particularly around HTTP/2 compliance. Web streams have known issues with HTTP/2, while Node streams are compliant.
The Cloudflare Worker Bandaid
While investigating, we discovered an odd workaround. We set up a Cloudflare worker as middleware between our server and clients:
// Cloudflare Worker that acts as a CORS-enabled streaming proxy export default { async fetch ( request, env, ctx ) { // Handle preflight CORS requests // .... // Extract request body for non-GET requests const body = [ "GET" , "HEAD" ].includes(request.method) ? null : request.body; // Create headers that mimic a real browser request // .... // Forward the request to the target URL const response = await fetch(request.url, { method : request.method, headers : proxyHeaders, body : body, }); // Copy response headers and enable CORS const responseHeaders = new Headers(response.headers); responseHeaders.set( "Access-Control-Allow-Origin" , "*" ); // Create streaming response to handle large payloads const { readable, writable } = new TransformStream(); response.body?.pipeTo(writable); return new Response(readable, { status : response.status, statusText : response.statusText, headers : responseHeaders, }); }, };
This worker simply received the stream and responded with it. Somehow, this completely fixed the issue, which made no sense to us at the time.
Ruling Out HTTP Versions
We quickly invalidated our HTTP/1 vs HTTP/2 suspicion by using cURL with the --http1.1 flag. Even when making HTTP/1 requests, things streamed properly. Throughout the entire debugging process, cURL consistently worked as expected.
... continue reading