golang http get garbled code

In recent years, with the popularity of Go language, more and more people have begun to use Go language to develop Web applications, including programs that use HTTP for network access. However, encountering garbled characters in HTTP GET requests is a common problem. This article will explore this problem, its possible causes, and provide some solutions.

1. Problem description

When using Go language to write HTTP GET requests, sometimes we encounter the problem of garbled text. The main symptom is that the response body returned by the request contains garbled characters instead of the expected results.

2. Cause of the problem

There may be many reasons for garbled HTTP GET requests. Here are some common reasons:

1. The correct character set is not used. . In the response header of the HTTP request, the server will return the character set of the document. If we do not parse this value correctly, it may cause encoding problems.

2. No character set specified. Sometimes the server does not provide character set information. If we do not specify a character set, it may cause encoding problems.

3. Character set does not match. Sometimes, the character sets in the request header and response header do not match, which may result in garbled characters.

4. When reading data from a file, the encoding specified is inconsistent with the actual encoding, which may also lead to garbled characters.

3. Solution

1. Check the character set of the server response

: In HTTP GET, the server's response header contains character set information. If we don't check and parse this value correctly, it may cause garbled characters. The correct way is to use the resp.Header.Get("Content-Type") method provided in the Go language's net/http library to obtain the Content-Type response header information and obtain the character set value from it. We then need to use this charset to convert the response body to the correct string. For example, if the character set in the response header is UTF-8, we can use the following method to convert the response body into a UTF-8 encoded string.

import (
    "io/ioutil"
    "net/http"
)

func main() {
    resp, err := http.Get("http://example.com/")
    if err != nil {
        // handle error
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        // handle error
    }

    contentType := resp.Header.Get("Content-Type")
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        // handle error
    }

    // convert body bytes to string
    var content string
    if strings.Contains(contentType, "UTF-8") {
        content = convertToString(string(body), "UTF-8", "UTF-8")
    } else {
        content = convertToString(string(body), contentType, "UTF-8")
    }
}

func convertToString(content string, srcEncoding string, destEncoding string) string {
    srcDecoder := charmap.Windows1252.NewDecoder()
    srcReader := strings.NewReader(content)
    srcReader.Reset(content)
    srcUTF8Reader := transform.NewReader(srcReader, srcDecoder)
    destDecoder := charmap.ISO8859_1.NewDecoder()
    destWriter := new(bytes.Buffer)
    destUTF8Writer := transform.NewWriter(destWriter, destDecoder)
    io.Copy(destUTF8Writer, srcUTF8Reader)
    return destWriter.String()
}

2. Specify the correct character set

When sending an HTTP GET request, we should specify the character set in the request header. In this case, we need to use the Req.Header.Set("Content-Type", "text/html; charset=UTF-8") method provided in the Go language's net/http library to specify the Content-Type. For example, if we wish to send UTF-8 text using UTF-8 encoding, we can use the following code:

import (
    "net/http"
)

func main() {
    client := http.Client{}
    req, err := http.NewRequest("GET", "http://example.com/", nil)
    if err != nil {
        // handle error
    }

    req.Header.Set("Content-Encoding", "gzip")
    req.Header.Set("Content-Type", "text/html; charset=UTF-8")

    resp, err := client.Do(req)
    if err != nil {
        // handle error
    }
    defer resp.Body.Close()
}

3.Character Set Conversion

If we specify the correct character set, but still If you encounter garbled characters, you may need to perform character set conversion on the returned content. We can use the transform.String() method provided in the golang.org/x/text/transform library of the Go language to convert strings. For example, suppose we read an ISO-8859-1 encoded text from the file, but the server returned UTF-8 encoded text, we can use the following code to convert:

import (
    "bytes"
    "io"
    "io/ioutil"
    "net/http"
    "golang.org/x/text/transform"
    "golang.org/x/text/encoding/charmap"
)

func main() {
    resp, err := http.Get("http://example.com/")
    if err != nil {
        // handle error
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        // handle error
    }

    // read body
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        // handle error
    }

    // convert body bytes to string
    s, _, err := transform.String(charmap.ISO8859_1.NewDecoder().Transformer(), string(body))
    if err != nil {
        // handle error
    }

    // do something with s
    ...
}

4. Conclusion

Garbled characters in HTTP GET requests may affect the results of your network requests. If you encounter this problem, first check the character set information, and then check that the character set is specified correctly. If none of the above solves your problem, you may need to perform a character set conversion. I hope the methods provided in this article can help you solve the garbled problem in HTTP GET requests.

The above is the detailed content of golang http get garbled code. For more information, please follow other related articles on the PHP Chinese website!