Preventing Flash of Incomplete Markdown when streaming AI responses

You’ve heard of Flash of Unstyled Content (FOUC) before, where you see unstyled HTML appear, then the CSS loads and the browser suddenly updates to display the correct style. A similar problem exists when streaming responses generated by AI that I call “Flash of Incomplete Markdown” (FOIM). I’ve reproduced this within OpenAI’s playground by throttling my connection speed to dialup internet speeds: While this is greatly exaggerated due to the slow speed, you can see the incomplete markdown appear in chunks. This occurs because OpenAI’s streaming API returns an event stream where it builds up a response message. These chunks are provided by what are called output text deltas: You can read more about output text deltas in their API docs. I’ve seen this same behavior occur in a number of commercially launched products, so this isn’t some obscure effect. We experienced this in our product as well. One of the AI features Streak offers is the ability to ask a question about one of your de ... Read full article.