Why Does Decoding a String with Non-ASCII Characters Result in a \'UnicodeDecodeError: invalid continuation byte\'?

Decoding Error: "UnicodeDecodeError, invalid continuation byte"

When attempting to decode a string containing non-ASCII characters using the UTF-8 codec, you may encounter the "UnicodeDecodeError: invalid continuation byte" error. This error indicates that the string contains a byte sequence that is not a valid UTF-8 continuation byte.

---

In the specific example provided:

o = "a test of \xe9 char"
v = o.decode("utf-8")

The string o contains the non-ASCII character xe9 (é). When attempting to decode it using the UTF-8 codec, it fails with the aforementioned error because xe9 is an invalid UTF-8 continuation byte.

Solution

The solution is to use a different codec that can handle the non-ASCII characters in the string. In this case, you can use the latin-1 codec, which is designed to handle Western European characters including accented characters like é:

v = o.decode("latin-1")

Explanation

The latin-1 codec is a single-byte character set that includes 256 characters, including the letters of the English alphabet, accented characters, and some punctuation marks. It is commonly used to encode text in Western European languages.

By using the latin-1 codec, you can successfully decode the string o without encountering the "UnicodeDecodeError" exception.

The above is the detailed content of Why Does Decoding a String with Non-ASCII Characters Result in a \'UnicodeDecodeError: invalid continuation byte\'?. For more information, please follow other related articles on the PHP Chinese website!