How to Unmarshal Unknown Protobuf Messages in Go?

Unmarshaling Unknown Protobuf Messages in Go

Unmarshalling protobuf messages into an interface{} to facilitate type casting is not feasible as the proto.Unmarshal function expects an argument of type proto.Message, an interface that concrete protobuf types implement.

Alternative Approach with Known Metadata

In cases where at least some contextual information (e.g., a string or number) accompanies the byte slice, you can use that information to map to the expected protobuf type and instantiate it before passing it to proto.Unmarshal:

func decodeWithMetadata(data []byte, identifier string) error {
    var message proto.Message
    switch identifier {
        case "foo":
            message = &mypb.Foo{}
        case "bar":
            message = &mypb.Bar{}
    }
    if err := proto.Unmarshal(data, message); err != nil {
        return err
    }
    log.Printf("%v\n", data)
    return nil
}

Parsing Completely Unknown Payloads

However, if the byte payload is entirely unknown, consider the protowire package, which allows extracting limited information from wire-formatted protobuf messages. Keep in mind that the wire representation of protobuf messages is ambiguous, leading to weak semantics.

Parser Implementation

The following code parses unknown protobuf messages:

type Field struct {
    Tag Tag
    Val Val
}

type Tag struct {
    Num int32
    Type protowire.Type
}

type Val struct {
    Payload interface{}
    Length int
}

func parseUnknown(b []byte) []Field {
    // ... implementation to parse unknown protobuf messages as described in the provided answer ...
}

Sample Usage

Given a protobuf schema like:

message Foo {
  string a = 1;
  string b = 2;
  Bar bar = 3;
}

message Bar {
  string c = 1;
}

The following code demonstrates how to use the parser:

// Parse the protobuf message as byte slice
b := []byte{0x0a, 0x01, 0x41, 0x12, 0x01, 0x42, 0x1a, 0x03, 0x43}
fields := parseUnknown(b)
for _, field := range fields {
    fmt.Printf("%#v\n", field)
}

Output:

main.Field{Tag:main.Tag{Num:1, Type:2}, Val:main.Val{Payload:[]uint8{0x41}, Length:1}}
main.Field{Tag:main.Tag{Num:2, Type:2}, Val:main.Val{Payload:[]uint8{0x42}, Length:1}}
main.Field{Tag:main.Tag{Num:1, Type:2}, Val:main.Val{Payload:[]uint8{0x43}, Length:1}}
main.Field{Tag:main.Tag{Num:3, Type:2}, Val:main.Val{Payload:[]main.Field{main.Field{Tag:main.Tag{Num:1, Type:2}, Val:main.Val{Payload:[]uint8{0x43}, Length:1}}}, Length:3}}

The above is the detailed content of How to Unmarshal Unknown Protobuf Messages in Go?. For more information, please follow other related articles on the PHP Chinese website!