'Golang - How does golang memory profile count allocs/op?

I'm writing a custom JSON marshal function and comparing it to the built-in json.Marshal method.

My understanding is that when bytes.Buffer reaches its capacity, it needs to double its size and this costs 1 allocation.

However, benchmark result seems to indicate json.Marshal is doing this in a way where it does NOT need to allocate whenever it grows the underlying buffer, whereas my implementation seems to cost an extra allocation everytime the buffer doubles.

Why would MarshalCustom (code below) need to allocate more than json.Marshal?

$ go test -benchmem -run=^$ -bench ^BenchmarkMarshalText$ test
BenchmarkMarshalText/Marshal_JSON-10               79623             13545 ns/op            3123 B/op          2 allocs/op
BenchmarkMarshalText/Marshal_Custom-10        142296              8378 ns/op           12464 B/op          8 allocs/op
PASS
ok      test    2.356s

Full code.

type fakeStruct struct {
    Names []string `json:"names"`
}

var ResultBytes []byte

func BenchmarkMarshalText(b *testing.B) {
    names := randomNames(1000)

    b.Run("Marshal JSON", func(b *testing.B) {
        fs := fakeStruct{
            Names: names,
        }

        b.ReportAllocs()
        b.ResetTimer()
        for i := 0; i < b.N; i++ {
            ResultBytes, _ = json.Marshal(fs)
        }
    })

    b.Run("Marshal Custom", func(b *testing.B) {
        fs := fakeStruct{
            Names: names,
        }

        b.ReportAllocs()
        b.ResetTimer()
        for i := 0; i < b.N; i++ {
            ResultBytes = MarshalCustom(fs)
        }
    })
}

func MarshalCustom(fs fakeStruct) []byte {
    var b bytes.Buffer

    b.WriteByte('{')

    // Names
    b.WriteString(`,"names":[`)
    for i := 0; i < len(fs.Names); i++ {
        if i > 0 {
            b.WriteByte(',')
        }
        b.WriteByte('"')
        b.WriteString(fs.Names[i])
        b.WriteByte('"')
    }
    b.WriteByte(']')

    b.WriteByte('}')

    buf := append([]byte(nil), b.Bytes()...)
    return buf
}

func randomNames(num int) []string {
    const letters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
    const maxLen = 5

    rand.Seed(time.Now().UnixNano())
    res := make([]string, rand.Intn(num))
    for i := range res {
        l := rand.Intn(maxLen) + 1 // cannot be empty
        s := make([]byte, l)
        for j := range s {
            s[j] = letters[rand.Intn(len(letters))]
        }
        res[i] = string(s)
    }

    return res
}


Solution 1:[1]

@oakad is correct. If I force a GC run in every iteration of the benchmark, the allocs/op is much closer or even the same.

for i := 0; i < b.N; i++ {
    // marshal here

    runtime.GC()
}

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 ccnlui