How to convert bytes to float32 array in Go?

Question content

I am writing a float32 array of numbers in byte format from a Python script to an Elasticache Redis cluster, and then from Elasticache in a Go script Read bytes (as string). How to convert bytes-as-string back to original float32 array in Go script?

Python example:

import numpy as np
import redis
a = np.array([1.1, 2.2, 3.3], dtype=np.float32)
a_bytes = a.tobytes(order="C") #I have also tried order="F" with no luck
print(a_bytes) #Output: b'\xcd\xcc\x8c?\xcd\xcc\x0c@33S@'
redis_client = redis.cluster.RedisCluster(host=<elasticache config endpoint>, port=6379)
redis_client.mset_nonatomic({"key1": a_bytes})

Here is an example of what I tried in Go (playground)

package main

import (
    "fmt"
    "math"
    "strconv"
)

func main() {
    // aBytesStr is an example value retrieved from Elasticache
    // aBytesStr is type string, not raw bytes
    var aBytesStr string = "\xcd\xcc\x8c?\xcd\xcc\x0c@33S@"
    aHex := fmt.Sprintf("%X", aBytesStr)
    fmt.Println(aHex) // Output: CDCC8C3FCDCC0C4033335340
    var aArr [3]float32
    for i := 0; i < 3; i++ {
        aHex1 := aHex[i*8 : i*8+8]
        aParsed, err := strconv.ParseUint(aHex1, 16, 32)
        if err != nil {
            return
        }
        aArr[i] = math.Float32frombits(uint32(aParsed))
    }
    fmt.Println(aArr)
    // Expected output: [1.1 2.2 3.3]
    // Actual output [-4.289679e+08 -4.2791936e+08 4.17524e-08]
}

Correct answer

The example code you used is "convert hex to string"; you have raw bytes (I think based on aHex:CDCC8C3FCDCC0C4033335340), so it's simpler to convert directly (although you could convert the bytes to a hex string and then do the conversion, which would just add unnecessary work/complexity).

From this answer we get (Playground):

func GetFloatArray(aBytes []byte) []float32 {
    aArr := make([]float32, 3)
    for i := 0; i < 3; i++ {
        aArr[i] = BytesFloat32(aBytes[i*4:])
    }
    return aArr
}

func BytesFloat32(bytes []byte) float32 {
    bits := binary.LittleEndian.Uint32(bytes)
    float := math.Float32frombits(bits)
    return float
}

Updated reference comments:

I'm still a little confused by what you received, so let's try these two possibilities.

If the redis query returns raw data (bytes) as a go string (i.e. "\xcd\xcc\x8c?\xcd\xcc\x0c@33S@"), then you can Convert to []byte (Playground)

func main() {
    var aBytesStr string = "\xcd\xcc\x8c?\xcd\xcc\x0c@33S@"
    fmt.Println(GetFloatArray([]byte(aBytesStr)))
}

If Redis returns a string containing ASCII (/UTF-8) representation (i.e. CDCC = []byte{0x41, 0x44, 0x43, 0x43}), the final An easy way might be to decode this using encoding/hex( playground)

func main() {
    aHex := "CDCC8C3FCDCC0C4033335340"
    b, err := hex.DecodeString(aHex)
    if err != nil {
        panic(err)
    }
    fmt.Println(GetFloatArray(b))
}

Please note that your original approach may work, however, as pointed out in the comments above, you need to deal with Endianness so the following will work (playground - you can improve Efficiency, my purpose is for clarity):

byte1 := aHex[i*8 : i*8+2]
byte2 := aHex[i*8+2 : i*8+4]
byte3 := aHex[i*8+4 : i*8+6]
byte4 := aHex[i*8+6 : i*8+8]
aParsed, err := strconv.ParseUint(byte4+byte3+byte2+byte1, 16, 32)

However, this makes assumptions about the CPU running the code, which means the previous answer is preferable.

The above is the detailed content of How to convert bytes to float32 array in Go?. For more information, please follow other related articles on the PHP Chinese website!