Go ReadLine for Long Lines

 

In this post we present the correct method of reading lines from a long text in GO.

The naive method of reading lines in go is using bufio.Scanner:

scanner := bufio.NewScanner(file)
for scanner.Scan() {
  fmt.Println(scanner.Text())
}

This does not work in case of long lines. Instead, we should use bufio.Reader. However, the compiling of long lines is cumbersome, and a KISS wrapper is missing in the GO standard library, hence I've created it myself.

import (
  "bufio"
  "io"
  "strings"
)

type LineReader struct {
  reader *bufio.Reader
  isEof  bool
}

func ProduceLineReader(
  text string,
) *LineReader {
  reader := bufio.NewReader(strings.NewReader(text))

  return &LineReader{
   reader: reader,
   isEof:  false,
  }
}

func (r *LineReader) GetLine() string {
  var longLineBuffer *strings.Builder
  multiLines := false
  for {
   line, isPrefix, err := r.reader.ReadLine()
   if err == io.EOF {
    r.isEof = true
    return ""
   }

   if isPrefix {
    multiLines = true
   }

   if !multiLines {
    // simple single line
    return string(line)
   }

   if longLineBuffer == nil {
    // create only if needed - better performance
    longLineBuffer = &strings.Builder{}
   }

   longLineBuffer.Write(line)
   if !isPrefix {
    // end of long line
    return longLineBuffer.String()
   }
  }
}

func (r *LineReader) IsEof() bool {
  return r.isEof
}

An example of usage is:

reader := kitstring.ProduceLineReader(text)
for {
  line := reader.GetLine()
  if reader.IsEof() {
   break
  }
  t.Log("read line: %v", line)
}