Is Go\'s `encoding/gob` Encoding Deterministic, and How Can Deterministic Output Be Achieved?

Determinism in Encoding/Gob

Background

When encoding objects of type x and y into GOB format, a common expectation is that gob_encode(x) and gob_encode(y) will always produce identical byte sequences, given that the objects are equal.

Determinism of GOB Encoding

encoding/gob provides a non-deterministic implementation of GOB encoding when maps are involved. This uncertainty arises from the arbitrary iteration order of maps, leading to inconsistent serialization order during encoding.

Impact of Type Specifiers

However, for non-map values, encoding/gob implements a deterministic approach. This is because each data item in a GOB stream is prefixed with a type specifier. The first occurrence of a type in the stream includes the complete type definition. Subsequent occurrences of the same type reference the initial type specification using a reference or identifier.

Example

In the following example, we create a custom struct Int and encode it multiple times using the gob encoder.

type Int struct{ X int }
b := new(bytes.Buffer)
e := gob.NewEncoder(b)
e.Encode(Int{1})
fmt.Println(b.Bytes())
e.Encode(Int{1})
fmt.Println(b.Bytes())

Output:

[23 255 129 3 1 1 3 73 110 116 1 255 130 0 1 1 1 1 88 1 4 0 0 0 5 255 130 1 2 0]
[23 255 129 3 1 1 3 73 110 116 1 255 130 0 1 1 1 1 88 1 4 0 0 0 5 255 130 1 2 0 5 255 130 1 2 0]

While the initial encoding includes the full type definition, subsequent encodings only transmit a type reference, resulting in different byte sequences.

Implications

In general, unless you strictly require deterministic output, it is not necessary to be concerned about the aforementioned behavior.

However, it is important to note that deterministic output can be achieved by avoiding the use of maps and by using multiple encoders with identical ordering of encoded values.

Additionally, changes in the encoding/gob implementation between Go releases may affect the consistency of output. Compatibility is maintained, but the exact output may vary.

The above is the detailed content of Is Go\'s `encoding/gob` Encoding Deterministic, and How Can Deterministic Output Be Achieved?. For more information, please follow other related articles on the PHP Chinese website!