In this post we will review how to handle errors in an HTTP streaming response using GO and NDJSON. This is useful for AI chat responses, generated reports, and any operation that sends results before all its work is done.
Streaming enables the user to see output without waiting for the entire operation. However it introduces a problem: what happens if the server sends the result successfully, but fails to save it?
The client already received HTTP 200 and some text. Should it display success? Should it retry? Has the result been saved?
To answer these questions, the stream needs a completion contract.
The Streaming Problem
Let's assume our application generates a report. The server performs the following steps:
- Start an HTTP response.
- Send report sections as they become available.
- Save the completed report.
- Close the response.
Everything works until step 3 fails.
For a regular HTTP request, we could return an error status. However once the streaming response has started, the headers and status have already been sent. Calling WriteHeader with HTTP 500 at this point cannot replace the original HTTP 200.
Closing the response is also not enough. The client cannot tell whether the operation completed, the database failed, or the connection was interrupted.
Notice that even a clean end of the HTTP body only tells us the transport finished. It does not tell us our application completed all its work.
Create a Completion Contract
For this example we use NDJSON, which stands for newline-delimited JSON. Each line contains a complete JSON object. This makes it possible to process records as they arrive without waiting for a full JSON array.
We use two record types: chunk and final.
A successful response looks like this:
{"type":"chunk","text":"Report section one.\n"}
{"type":"chunk","text":"Report section two.\n"}
{"type":"final","outcome":"complete","saved":true}A response whose report was generated but not saved looks like this:
{"type":"chunk","text":"Report section one.\n"}
{"type":"chunk","text":"Report section two.\n"}
{"type":"final","outcome":"partial","saved":false,"error":"save_failed"}The report text is still useful. The client can display it with a notice that it was not saved.
Our completion rules are simple:
- A normally finished stream contains exactly one final record.
- The final record is the last record.
- Required work, including saving the report, happens before the final record.
- A stream that ends without a final record is incomplete.
The server should attempt to send a final record for handled failures. However no protocol can guarantee delivery after the connection breaks. The client must handle that case too.
GO Server Implementation
The following example uses only the GO standard library. Save it as server.go. The save function deliberately supports a failure so we can reproduce the problem without installing a database.
package main
import (
"encoding/json"
"errors"
"log"
"net/http"
)
type streamRecord struct {
RecordType string `json:"type"`
TextValue string `json:"text,omitempty"`
Outcome string `json:"outcome,omitempty"`
Saved *bool `json:"saved,omitempty"`
ErrorCode string `json:"error,omitempty"`
}
type reportStream struct {
encoder *json.Encoder
controller *http.ResponseController
}
func (s *reportStream) sendRecord(
record *streamRecord,
) error {
if err := s.encoder.Encode(record); err != nil {
return err
}
return s.controller.Flush()
}
func (s *reportStream) sendFinal(
saveError error,
) error {
saved := saveError == nil
record := streamRecord{
RecordType: "final",
Outcome: "complete",
Saved: &saved,
}
if saveError != nil {
record.Outcome = "partial"
record.ErrorCode = "save_failed"
}
return s.sendRecord(&record)
}
func saveReport(
failSave bool,
) error {
if failSave {
return errors.New("simulated storage failure")
}
// Replace with a database or object storage write.
return nil
}
func (s *reportStream) writeReport(
failSave bool,
) error {
sections := []string{
"Report section one.\n",
"Report section two.\n",
}
for _, section := range sections {
record := streamRecord{
RecordType: "chunk",
TextValue: section,
}
if err := s.sendRecord(&record); err != nil {
return err
}
}
saveError := saveReport(failSave)
if saveError != nil {
log.Printf("report save failed: %v", saveError)
}
return s.sendFinal(saveError)
}
func handleReport(
writer http.ResponseWriter,
request *http.Request,
) {
writer.Header().Set("Content-Type", "application/x-ndjson")
writer.Header().Set("Cache-Control", "no-store")
stream := reportStream{
encoder: json.NewEncoder(writer),
controller: http.NewResponseController(writer),
}
failSave := request.URL.Query().Get("failSave") == "true"
if err := stream.writeReport(failSave); err != nil {
log.Printf("report stream interrupted: %v", err)
}
}
func main() {
http.HandleFunc("/report", handleReport)
log.Fatal(http.ListenAndServe("127.0.0.1:8080", nil))
}The example requires GO 1.20 or later because it uses http.NewResponseController.
json.Encoder.Encode adds a newline after each JSON object. Flush requests that buffered output be sent immediately. A reverse proxy can still buffer the response, so its streaming configuration must also be checked when deploying the application.
Notice that Saved is a pointer. Chunk records omit this field, while final records include either true or false. Using a boolean with omitempty would hide false, which is an important result here.
Also notice the order: write the sections, save the report, then send the final record. Sending complete before saving would give the client a success indication that might become false a moment later.
The example logs the storage error on the server and sends a stable error code to the client. A real database error can contain internal details that should not appear in the response.
Run the Server
go run server.goIn another terminal, try both flows:
curl -N 'http://127.0.0.1:8080/report'
curl -N 'http://127.0.0.1:8080/report?failSave=true'The -N option disables curl's output buffering. Both requests return HTTP 200, but their final records describe different application outcomes.
GO Client Implementation
Now we need a client that understands the contract. Save the following as client.go and run it separately from server.go.
package main
import (
"encoding/json"
"errors"
"fmt"
"io"
"log"
"net/http"
"os"
"time"
)
type streamRecord struct {
RecordType string `json:"type"`
TextValue string `json:"text"`
Outcome string `json:"outcome"`
Saved *bool `json:"saved"`
ErrorCode string `json:"error"`
}
func readReport(
body io.Reader,
) error {
decoder := json.NewDecoder(body)
for {
var record streamRecord
err := decoder.Decode(&record)
if errors.Is(err, io.EOF) {
return errors.New("stream ended without a final record")
}
if err != nil {
return fmt.Errorf("cannot read stream: %w", err)
}
switch record.RecordType {
case "chunk":
if _, err := fmt.Print(record.TextValue); err != nil {
return err
}
case "final":
return checkFinal(&record)
default:
return fmt.Errorf("unknown record type: %q", record.RecordType)
}
}
}
func checkFinal(
record *streamRecord,
) error {
if record.Saved == nil {
return errors.New("final record is missing saved status")
}
if record.Outcome == "complete" && *record.Saved {
return nil
}
return fmt.Errorf(
"report did not fully complete: outcome=%q saved=%t error=%q",
record.Outcome, *record.Saved, record.ErrorCode,
)
}
func fetchReport(
reportUrl string,
) error {
client := http.Client{
Timeout: 30 * time.Second,
}
response, err := client.Get(reportUrl)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return fmt.Errorf("unexpected HTTP status: %s", response.Status)
}
return readReport(response.Body)
}
func main() {
if len(os.Args) != 2 {
log.Fatal("usage: go run client.go <report URL>")
}
if err := fetchReport(os.Args[1]); err != nil {
log.Fatal(err)
}
fmt.Println("Report generated and saved.")
}Try the client against both server flows:
go run client.go 'http://127.0.0.1:8080/report'
go run client.go 'http://127.0.0.1:8080/report?failSave=true'The first prints the report and a success message. The second prints the report and then reports that it was not saved.
The decoder reads JSON records rather than network packets. This is important because one record can arrive in multiple reads, and multiple records can arrive together. A network read is not an application message boundary.
The client treats the first final record as authoritative and stops reading. The server is responsible for sending it once, at the end. If you are validating an untrusted implementation, you can additionally read to EOF and reject duplicate final records or trailing records.
This example expects a trusted server and small records. For a public API, also bound record sizes and total output. Set the request timeout according to the expected operation duration; the example's 30 seconds includes reading the response body.
What Happens When the Connection Breaks?
If the server terminates after sending the sections but before sending final, the client reports an incomplete stream. It does not display a success message just because it received the report text.
However incomplete does not necessarily mean the report was not saved. The database write might have succeeded just before the connection was lost.
For operations where this matters, assign an operation ID before streaming starts. The client can query the stored operation status after reconnecting. If retries can create duplicate side effects, use an idempotency key and store the result of the operation.
The final record tells us what the server reported. The stored operation status helps us recover when that report never reaches the client.
Testing the Failure Paths
The most useful checks are the following:
- Successful generation and save produce one complete final record.
- A save failure after output produces one partial final record.
- EOF before final is reported as incomplete by the client.
- A malformed or truncated JSON record is reported as an error.
- A failed network write does not lead to another attempt to write success.
A generation failure can use a final outcome of failed with a stable error code. If some useful sections were already sent, define whether the client should keep them and show them as partial output.
These checks can run with an injected storage failure and an in-memory stream. They do not require a production database or a running AI model.
Using Server-Sent Events Instead of NDJSON
The same completion contract can be used with Server-Sent Events, also called SSE. The message framing changes, but the application still needs a final outcome:
event: chunk
data: {"text":"Report section one."}
event: final
data: {"outcome":"complete","saved":true}
Notice the blank line after each event. With EventSource, the client should close the connection when it receives final; otherwise the browser may reconnect after the server closes the stream.
Choose NDJSON or SSE according to the client and API requirements. Neither format automatically defines what successful completion means.
Final Note
An HTTP streaming response needs more than chunks of output. It needs an explicit completion result that is sent after the required work finishes.
This enables the client to distinguish a completed operation, a useful result that was not saved, and an interrupted request whose outcome is still unknown.
