How do you convert a slice into an array?

Question

I am trying to write an application that reads RPM files. The start of each block has a Magic char of [4]byte.

Here is my struct

type Lead struct {
  Magic        [4]byte
  Major, Minor byte
  Type         uint16
  Arch         uint16
  Name         string
  OS           uint16
  SigType      uint16
}

I am trying to do the following:

lead := Lead{}
lead.Magic = buffer[0:4]

I am searching online and not sure how to go from a slice to an array (without copying). I can always make the Magic []byte (or even uint64), but I was more curious on how would I go from type []byte to [4]byte if needed to?

Answer 1

You have allocated four bytes inside that struct and want to assign a value to that four byte section. There is no conceptual way to do that without copying.

Look at the copy built-in for how to do that.

Answer 2

Try this:

copy(lead.Magic[:], buf[0:4])

Answer 3

Tapir Liui (auteur de Go101) twitte:

Go 1.18 1.19 will support conversions from slice to array: golang/go issues 46505.

So, since Go 1.18,the slice copy2 implementation could be written as:

*(*[N]T)(d) = [N]T(s)

or, even simpler if the conversion is allowed to present as L-values:

[N]T(d) = [N]T(s)

---

Without copy, you can convert, with the next Go 1.17 (Q3 2021) a slice to an array pointer.

This is called "un-slicing", giving you back a pointer to the underlying array of a slice, again, without any copy/allocation needed:

See golang/go issue 395: spec: convert slice x into array pointer, now implemented with CL 216424/, and commit 1c26843

Converting a slice to an array pointer yields a pointer to the underlying array of the slice.
If the length of the slice is less than the length of the array, a run-time panic occurs.

s := make([]byte, 2, 4)
s0 := (*[0]byte)(s)      // s0 != nil
s2 := (*[2]byte)(s)      // &s2[0] == &s[0]
s4 := (*[4]byte)(s)      // panics: len([4]byte) > len(s)

var t []string
t0 := (*[0]string)(t)    // t0 == nil
t1 := (*[1]string)(t)    // panics: len([1]string) > len(s)

So in your case, provided Magic type is *[4]byte:

lead.Magic = (*[4]byte)(buffer)

Note: type aliasing will work too:

type A [4]int
var s = (*A)([]int{1, 2, 3, 4})

---

Why convert to an array pointer? As explained in issue 395:

One motivation for doing this is that using an array pointer allows the compiler to range check constant indices at compile time.

A function like this:

func foo(a []int) int
{
   return a[0] + a[1] + a[2] + a[3];
}

could be turned into:

func foo(a []int) int
{
   b := (*[4]int)(a)
   return b[0] + b[1] + b[2] + b[3];
}

allowing the compiler to check all the bounds once only and give compile-time errors about out of range indices.

Also:

One well-used example is making classes as small as possible for tree nodes or linked list nodes so you can cram as many of them into L1 cache lines as possible.
This is done by each node having a single pointer to a left sub-node, and the right sub-node being accessed by the pointer to the left sub-node + 1.
This saves the 8-bytes for the right-node pointer.

To do this you have to pre-allocate all the nodes in a vector or array so they're laid out in memory sequentially, but it's worth it when you need it for performance.
(This also has the added benefit of the prefetchers being able to help things along performance-wise - at least in the linked list case)

You can almost do this in Go with:

  type node struct {
     value int
     children *[2]node
  }

except that there's no way of getting a *[2]node from the underlying slice.

Answer 4

I was more curious on how would I go from type []byte to [4]byte if needed to?

Starting from Go 1.17 you can directly convert a slice to an array pointer. With Go's type conversion syntax T(x) you can do this:

slice := make([]byte, 4)
arrptr := (*[4]byte)(slice)

Keep in mind that the length of the array must not be greater than the length of the slice, otherwise the conversion will panic.

bad := (*[5]byte)(slice) // panics: slice len < array len

This conversion has the advantage of not making any copy, because it simply yields a pointer to the underlying array.

Of course you can dereference the array pointer to obtain a non-pointer array variable, so the following also works:

slice := make([]byte, 4)
var arr [4]byte = *(*[4]byte)(slice)

However dereferencing and assigning will subtly make a copy, since the arr variable is now initialized to the value that results from the conversion expression. To be clear (using ints for simplicity):

v := []int{10,20}
a := (*[2]int)(v)

a[0] = 500
fmt.Println(v) // [500 20] (changed, both point to the same backing array)

w := []int{10,20}
b := *(*[2]int)(w)

b[0] = 500
fmt.Println(w) // [10 20] (unchanged, b holds a copy)

---

One might wonder why the conversion checks the slice length and not the capacity (I did). Consider the following program:

func main() {
    a := []int{1,2,3,4,5,6}
    fmt.Println(cap(a)) // 6

    b := a[:3]
    fmt.Println(cap(a)) // still 6

    c := (*[3]int)(b)

    ptr := uintptr(unsafe.Pointer(&c[0]))
    ptr += 3 * unsafe.Sizeof(int(0))

    i := (*int)(unsafe.Pointer(ptr))
    fmt.Println(*i) // 4
}

The program shows that the conversion might happen after reslicing. The original backing array with six elements is still there, so one might wonder why a runtime panic occurs with (*[6]int)(b) where cap(b) == 6.

This has actually been brought up. It's worth to remember that, unlike slices, an array has fixed size, therefore it needs no notion of capacity, only length:

a := [4]int{1,2,3,4}
fmt.Println(len(a) == cap(a)) // true

Answer 5

You might be able to do the whole thing with one read, instead of reading individually into each field. If the fields are fixed-length, then you can do:

lead := Lead{}

// make a reader to dispense bytes so you don't have to keep track of where you are in buffer
reader := bytes.NewReader(buffer)

// read into each field in Lead, so Magic becomes buffer[0:4],
// Major becomes buffer[5], Minor is buffer[6], and so on...
binary.Read(reader, binary.LittleEndian, &lead)