Replacing a line within a file with Golang

Question

I'm new to Golang, starting out with some examples. Currently, what I'm trying to do is reading a file line by line and replace it with another string in case it meets a certain condition. The file is use for testing purposes contains four lines:

one
two
three
four

The code working on that file looks like this:

 func main() {

     file, err := os.OpenFile("test.txt", os.O_RDWR, 0666)

     if err != nil {
       panic(err)
     }

     reader := bufio.NewReader(file)

      for {

         fmt.Print("Try to read ...\n")

         pos,_ := file.Seek(0, 1)
         log.Printf("Position in file is: %d", pos)
         bytes, _, _ := reader.ReadLine()

         if (len(bytes) == 0) {
            break
         }

         lineString := string(bytes)

         if(lineString == "two") {
             file.Seek(int64(-(len(lineString))), 1)
             file.WriteString("This is a test.")
         }

         fmt.Printf(lineString + "\n")
   }

    file.Close()
 }

As you can see in the code snippet, I want to replace the string "two" with "This is a test" as soon as this string is read from the file. In order to get the current position within the file I use Go's Seek method. However, what happens is that always the last line gets replaced by This is a test, making the file looking like this:

one
two
three
This is a test

Examining the output of the print statement which writes the current file position to the terminal, I get that kind of output after the first line has been read:

2016/12/28 21:10:31 Try to read ...
2016/12/28 21:10:31 Position in file is: 19

So after the first read, the position cursor already points to the end of my file, which explains why the new string gets appended to the end. Does anyone understand what is happening here or rather what is causing that behavior?

Answer 1

The Reader is not controller by the file.Seek. You have declared the reader as: reader := bufio.NewReader(file) and then you read one line at a time bytes, _, _ := reader.ReadLine() however the file.Seek does not change the position that the reader is reading.

Suggest you read about the ReadSeeker in the docs and switch over to using that. Also there is an example using the SectionReader.

Answer 2

Aside from the incorrect seek usage, the difficulty is that the line you're replacing isn't the same length as the replacement. The standard approach is to create a new (temporary) file with the modifications. Assuming that is successful, replace the original file with the new one.

package main

import (
    "bufio"
    "io"
    "io/ioutil"
    "log"
    "os"
)

func main() {

    // file we're modifying
    name := "text.txt"

    // open original file
    f, err := os.Open(name)
    if err != nil {
        log.Fatal(err)
    }
    defer f.Close()

    // create temp file
    tmp, err := ioutil.TempFile("", "replace-*")
    if err != nil {
        log.Fatal(err)
    }
    defer tmp.Close()

    // replace while copying from f to tmp
    if err := replace(f, tmp); err != nil {
        log.Fatal(err)
    }

    // make sure the tmp file was successfully written to
    if err := tmp.Close(); err != nil {
        log.Fatal(err)
    }

    // close the file we're reading from
    if err := f.Close(); err != nil {
        log.Fatal(err)
    }

    // overwrite the original file with the temp file
    if err := os.Rename(tmp.Name(), name); err != nil {
        log.Fatal(err)
    }
}

func replace(r io.Reader, w io.Writer) error {
    // use scanner to read line by line
    sc := bufio.NewScanner(r)
    for sc.Scan() {
        line := sc.Text()
        if line == "two" {
            line = "This is a test."
        }
        if _, err := io.WriteString(w, line+"\n"); err != nil {
            return err
        }
    }
    return sc.Err()
}

For more complex replacements, I've implemented a package which can replace regular expression matches. https://github.com/icholy/replace

import (
    "io"
    "regexp"

    "github.com/icholy/replace"
    "golang.org/x/text/transform"
)

func replace2(r io.Reader, w io.Writer) error {
    // compile multi-line regular expression
    re := regexp.MustCompile(`(?m)^two$`)

    // create replace transformer
    tr := replace.RegexpString(re, "This is a test.")

    // copy while transforming
    _, err := io.Copy(w, transform.NewReader(r, tr))
    return err
}

Answer 3

OS package has Expand function which I believe can be used to solve similar problem.

Explanation:

file.txt

one
two
${num}
four

main.go

package main

import (
    "fmt"
    "os"
)

var FILENAME = "file.txt"

func main() {
    file, err := os.ReadFile(FILENAME)
    if err != nil {
        panic(err)
    }
    
    mapper := func(placeholderName string) string {
        switch placeholderName {
        case "num":
            return "three"
        }

        return ""
    }

    fmt.Println(os.Expand(string(file), mapper))
}

output

one
two
three
four

Additionally, you may create a config (yml or json) and populate that data in the map that can be used as a lookup table to store placeholders as well as their replacement strings and modify mapper part to use this table to lookup placeholders from input file.

e.g map will look like this,

table := map[string]string {
      "num": "three"
}

mapper := func(placeholderName string) string {
        if val, ok := table[placeholderName]; ok {
            return val
        }

        return ""
}

References:

• os.Expand documentation: https://pkg.go.dev/os#Expand

Playground