Does Go have no real way to shrink a slice? Is that an issue?

Question

I've been trying out Go for some time and this question keeps bugging me. Say I build up a somewhat large dataset in a slice (say, 10 million int64s).

package main

import (
    "math"
    "fmt"
)

func main() {
    var a []int64
    var i int64;
    upto := int64(math.Pow10(7))
    for i = 0; i < upto; i++ {
        a = append(a, i)
    }
    fmt.Println(cap(a))
}

But then I decide I don't want most of them so I want to end up with a slice of just 10 of those. I've tried both slicing and delete techniques on Go's wiki but none of them seem to reduce the slice's capacity.

So that's my question: does Go has no real way of shrinking the capacity of a slice that would be similar to realloc()-ing with a smaller size argument than in your previous call on the same pointer in C? Is that an issue and how should one deal with it?

Answer 1

Go does not have a way of shrinking slices. This isn't a problem in most cases, but if you profile your memory use and find you're using too much, you can do something about it:

Firstly, you can just create a slice of the size you need and copy your data into it. The garbage collector will then free the large slice. Copy built-in

Secondly, you could re-use the big slice each time you wish to generate it, so you never allocate it more than once.

On a final note, you can use 1e7 instead of math.Pow10(7).

Answer 2

Additionally you can re-use most of the allocated memory during work of yours app, take a look at: bufs package

PS if you re-alocate new memory for smaller slice, old memory may not be freed in same time, it will be freed when garbage collector decides to.

Answer 3

Let's see this example:

func main() {
    s := []string{"A", "B", "C", "D", "E", "F", "G", "H"}
    fmt.Println(s, len(s), cap(s))  // slice, length, capacity

    t := s[2:4]
    fmt.Println(t, len(t), cap(t))

    u := make([]string, len(t))
    copy(u, t)
    fmt.Println(u, len(u), cap(u))
}

It produces the following output:

[A B C D E F G H] 8 8
[C D] 2 6
[C D] 2 2

s is a slice that holds 8 pieces of strings. t is a slice that keeps the part [C D]. The length of t is 2, but since it uses the same hidden array of s, its capacity is 6 (from "C" to "H"). The question is: how to have a slice of [C D] that is independent from the hidden array of s? Simply create a new slice of strings with length 2 (slice u) and copy the content of t to u. u's underlying hidden array is different from the hidden array of s.

The initial problem was this: you have a big slice and you create a new smaller slice on it. Since the smaller slice uses the same hidden array, the garbage collector won't delete the hidden array.

See the bottom of this post for more info: http://blog.golang.org/go-slices-usage-and-internals .

Answer 4

You can do that by re-assigning the slice's value to a portion of itself

  a := []int{1,2,3}
  fmt.Println(len(a), a) // 3 [1 2 3]

  a = a[:len(a)-1]
  fmt.Println(len(a), a) //2 [1 2]

Answer 5

There is a new feature called 3-index slice in Go 1.2, which means to get part of a slice in this way:

slice[a:b:c]

In which the len for the returned slice whould be b-a, and the cav for the new slice would be c-a.

Tips: no copy is down in the whole process, it only returns a new slice which points to &slice[a] and has the len as b-a and cav as c-a.

And that's the only thing you have to do:

slice= slice[0:len(slice):len(slice)];

Then the cav of the slice would be changed to len(slice) - 0, which is the same as the len of it, and no copy is done.