Go | Efficient and readable way to append slices and send to variadic function

Question content

Suppose I have the following functional pipeline:

func func3(opts ...functionobject) {
    for _, opt := range opts {
        opt()
    }
}

func func2(opts ...functionobject) {
   var functions []functionobject
   functions = append(functions, somefunction3)
   functions = append(functions, somefunction4)
...
...
...
    func3(append(functions, opts...)...)
}


func func1(opts ...functionobject) {
   var functions []functionobject
   functions = append(functions, somefunction)
   functions = append(functions, somefunction2)
...
...
...
    func2(append(functions, opts...)...)
}

Due to inheritance issues that I want to solve, the functions in functions should be called before the functions in opts, so I can't just append to opts But I have to prepend functions to opts (via append(functions, opts...)) and then use # again ##... sends it to the next function in the pipeline, so I get the weird expression:

func2(append(functions, opts...)...)

I don't know how efficient it is, but I'm sure it looks weird,

There must be a better way to do this, that's what I'm looking for.

But I would appreciate the accompanying explanation about efficiency :)

edit: I can't change the parameter type from

opts ...functionobject to opts []functionobject (as @dev.bmax suggested in the comments) because I'm in the existing code base so I can't change the function 1,2,3 that calls func{

By "looks weird" I don't just mean "appearance", I mean doing this twice (ellipses) looks weird and seems inefficient (am I wrong?)

Correct answer

Adding in front of a slice is basically inefficient because it requires a combination of:

Allocate a larger backing array
Move item to end of slice
...or both.

It would be more efficient if you could change the calling convention between functions to just append options and then process them in reverse. This avoids repeatedly moving items to the end of the slice and avoids all but the first allocation (if enough space is allocated ahead of time).

func func3(opts ...functionobject) {
    for i := len(opts) - 1; i >= 0; i-- {
        opts[i]()
    }
}

Note:

func3(opts ...functionobject) / func3(opts...) and func3(opts []functionobject) / func3(opts) are in terms of performance Equivalent. The former is effective syntax sugar for passing slices.

However, you mentioned that you need to preserve the calling convention...

Your example code will result in first, second, third, fifth... additional allocations within each function - allocations are needed to double the size of the backing array (for small slices).

append(functions, opts...) may also be allocated if earlier appends did not create enough spare capacity.

Auxiliary functions can make the code more readable. It can also be reused

opts Supports spare capacity in arrays:

func func2(opts ...functionobject) {
    // 1-2 allocations. always allocate the variadic slice containings 
    // prepend items. prepend reallocates the backing array for `opts`
    // if needed.
    opts = prepend(opts, somefunction3, somefunction4)
    func3(opts...)
}

// generics requires go1.18+. otherwise change t to functionobject.
func prepend[t any](base []t, items ...t) []t {
    if size := len(items) + len(base); size <= cap(base) {
        // extend base using spare slice capacity.
        out := base[:size]
        // move elements from the start to the end of the slice (handles overlaps).
        copy(out[len(items):], base)
        // copy prepended elements.
        copy(out, items)
        return out
    }
    return append(items, base...) // always re-allocate.
}

Some alternatives without helper functions that describe allocation in more detail:

// Directly allocate the items to prepend (2 allocations).
func func1(opts ...FunctionObject) {
    // Allocate slice to prepend with no spare capacity, then append re-allocates the backing array
    // since it is not large enough for the additional `opts`.
    // In future, Go could allocate enough space initially to avoid the
    // reallocation, but it doesn't do it yet (as of Go1.20rc1).
    functions := append([]FunctionObject{
        someFunction,
        someFunction2,
        ...
    }, opts...)
    // Does not allocate -- the slice is simply passed to the next function.
    func2(functions...)
}

// Minimise allocations (1 allocation).
func func2(opts ...FunctionObject) {
   // Pre-allocate the required space to avoid any further append
   // allocations within this function.
   functions := make([]FunctionObject, 0, 2 + len(opts))
   functions = append(functions, someFunction3)
   functions = append(functions, someFunction4)
   functions = append(functions, opts...)
   func3(functions...)
}

You can go one step further and reuse the spare capacity in

opts without allocating the slice containing the items to be prepended (0-1 allocation per function). However, this is complex and error-prone - I don't recommend it.

The above is the detailed content of Go | Efficient and readable way to append slices and send to variadic function. For more information, please follow other related articles on the PHP Chinese website!