allocation - Can (the underlying array of) a slice with large starting index in Go be allocated memory-efficiently? -

i'm trying use slice, myslice, large starting index. rather explicitly subtracting starting index using myslice[index - myslicestartindex], tempted define slice in such way can use without such arithmetic myslice[index]. can done without allocating memory unused low indices?

the naive way of doing this, allocating slice , reslicing (e.g. myslice = myslice[3*1024*1024*1024:4*1024*1024*1024]) is memory inefficient because underlying array not needs allocated entire range, remains allocated. not work, because afterwards data formerly @ index 3*1024*1024*1024 @ index 0, whilst goal keep @ original index.

can allocate slice (or underlying array) in such way indices below slice's start not allocated, ideally not initially?

this not possible without /not/ allocating unused parts. way slice defined in go, through reflect.sliceheader

type sliceheader struct {     data uintptr     len  int     cap  int } 

it contains no starting index field. merely reference underlying, fixed size array. underlying array holds actual data. slice 'window' array, begins @ index 0. wherever 0 may in underlying array.

for instance, consider following code:

a := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9} b := a[2:8] c := a[8:] d := b[2:4] 

this yields memory layout follows:

fixed array: [ 0 1 2 3 4 5 6 7 8 9 ]  > [10]int @ address 273785072 slice    :   . . . . . . . . . .    > sliceheader{data:273785072 len:10 cap:10} slice b    :       . . . . . .        > sliceheader{data:273785080 len:6 cap:8} slice c    :                   . .    > sliceheader{data:273785104 len:2 cap:2} slice d    :           . .            > sliceheader{data:273785088 len:2 cap:6} 

the values data address offsets fixed array , 4 slices share underlying storage.

a =:= $273785072 b =:= $273785080 =:= $a + sizeof(int)*2 =:= $a + 8 c =:= $273785104 =:= $a + sizeof(int)*8 =:= $a + 32 d =:= $273785088 =:= $b + sizeof(int)*2 =:= $a + sizeof(int)*4 =:= $a + 16 

at whatever index re-slice existing slice, new slice indexed 0 len(s), because address in underlying fixed array points puts there.

memory mapping

if data loaded file on disk, can have different option: use syscall.mmap provide access data through slice, starting @ desired index. returned slice index 0 , covers range specified.

func mmap(fd *os.file, start, size int) ([]byte, error) {     _, err := fd.seek(0, 0)     if err != nil {         return nil, err     }      return syscall.mmap(int(fd.fd()), start, size,         syscall.prot_read, syscall.map_shared) } 

do not forget call syscall.munmap on returned slice, when done using it.