ARTICLE AD BOX
I’m hitting a super weird issue with OpenAI’s streaming API and I’m lowkey losing my mind here so if anyone ran into this before pls tell me wtf is going on because I can’t tell anymore if it’s me or the API
basically I’m building a multi-message streaming thing in plain JS (no frameworks) where I need to pipe multiple openai responses one after another. looks fine on localhost, but in production (node 18) when two streams overlap even by a few ms, the whole thing starts mixing chunks between streams like they share the same reader or something. feels like some async iterator is leaking state?? idk
I’m using the official openai node sdk (the new one) and the .response.body.getReader() thing. sometimes the chunks arrive duplicated, sometimes out of order, sometimes a chunk from stream A literally shows up inside stream B. zero idea why. no shared variables, no global state, nothing.
I’m guessing something about the reader or WebStream polyfill inside node is borked when you run multiple concurrent reads? or maybe I’m doing something wrong with the loop. but it only happens on prod server under real traffic which makes this hell to debug.
any idea what’s the “correct” way to handle 2–3 parallel streaming completions without chunk bleed? is there like a known issue with the SDK or node streams?
here’s a stripped version of the code that still breaks randomly:
import OpenAI from "openai"; const client = new OpenAI({ apiKey: process.env.OPENAI_KEY }); async function runStream(prompt) { const res = await client.chat.completions.create({ model: "gpt-4.1", messages: [{ role: "user", content: prompt }], stream: true, }); const reader = res.response.body.getReader(); let buffer = ""; while (true) { const { done, value } = await reader.read(); if (done) break; const chunk = new TextDecoder().decode(value); // this is where random corruption happens: // sometimes chunk contains leftover bytes from another stream?? buffer += chunk; } return buffer; } // simulate parallel calls Promise.all([ runStream("msg A"), runStream("msg B"), runStream("msg C"), ]).then(console.log);If anyone knows why streams bleed into each other under concurrency, or what’s the correct approach to isolate them, I’d appreciate it because right now I’m just logging hex dumps and crying inside.
