How to Calculate the Actual Length of a UTF-8 String in C ?

Counting the Actual Length of UTF-8 Strings

While working with UTF-8 encoded std::string objects, developers often encounter a discrepancy between the value returned by str.length() and the actual number of characters in the string. This is because str.length() simply counts the number of bytes in the string, not taking into account the multibyte encoding used to represent UTF-8 characters.

The UTF-8 encoding defines a set of byte sequences used to represent Unicode characters. Each character can be represented by one to four bytes, depending on its Unicode code point. These byte sequences are structured as follows:

• 1-byte sequence (0x00-0x7F): Represents ASCII characters directly.
• 2-byte sequence (0x80-0xBF): Represents characters with code points from 0x0080 to 0x07FF.
• 3-byte sequence (0xC0-0xDF): Represents characters with code points from 0x0800 to 0xFFFF.
• 4-byte sequence (0xE0-0xFF): Represents characters with code points from 0x10000 to 0x10FFFF.

Calculating the Actual Length

The actual length of a UTF-8 encoded string can be determined by counting the number of first-bytes in the string, which are the bytes that do not match the pattern 10xxxxxx. This pattern represents continuation bytes, which are used to represent multiple-byte sequences.

Code Snippet

<code class="cpp">int len = 0;
while (*s) len += (*s++ & 0xc0) != 0x80;</code>

In this code, the while loop iterates over the string, incrementing the length len count by 1 for each first-byte it encounters.

The above is the detailed content of How to Calculate the Actual Length of a UTF-8 String in C ?. For more information, please follow other related articles on the PHP Chinese website!