PHP Master | Working with Multibyte Strings

Number language, whether in English, Japanese or any other language, consists of many characters. Therefore, when dealing with a numeric language, a basic question is how to represent each character numerically. In the past, we only had to represent English characters, but now things are very different, and the result is a dazzling character encoding scheme to represent characters in multiple different languages. How does PHP associate and process these different schemes?

Key points

• Multi-byte characters use one to four bytes to define characters, which is crucial for numeric representations of languages ​​with more than 256 unique characters. Unicode, especially UTF-8, is the most commonly used encoding scheme for these characters.
• PHP itself is not designed to handle multibyte characters. To process these characters, a special set of functions, the mbstring function, should be used. However, PHP's HTTP header also contains character set identifiers that can override the page's meta tags.
• Multi-byte support is not the default feature of PHP and requires reconfiguration. To enable the mb function, use the --enable-mbstring compile-time option and set the runtime configuration option mbstring-encoding_translation.
• Several multibyte string commands are available in PHP, such as mb_check_encoding, mb_strlen, and mb_ereg_search, which are used to check whether a specific encoding sequence is valid, find the number of characters in a multibyte string, and perform traditional character searches. Multibyte version.

Basics

We all know that "bits" can be 0 or 1, while "bytes" are a combination of eight consecutive bits. Since there are eight such double-valued bits in a byte, a byte can be configured in a total of 256 different modes (to the 8th power of 2). Different characters can be associated with each possible 8-bit mode. Put these bytes together in different orders and you have your own way of communicating. It's not necessarily smart, it depends on who is on both ends, but it's communication. As long as we can express characters in a language with 256 unique characters or less, we succeed. But what if we can't express a language with just 256 characters? Or what if we need to express multiple languages ​​in the same document? Today, as we digitize everything we can find, 256 characters are far from enough. Fortunately, character schemes that better meet this challenge have been designed. These new supercharacter sets use one to four bytes to define characters. Today, the big guy in the field of character encoding is Unicode, which is a solution that uses multiple bytes to represent characters. It was developed by Unicode Consortium and comes in several versions: UTF-32 (for Dreadnaught Class Starship), UTF-16 (for Enterprise in Star Trek: Dark Unbound) and UTF-8 (most of us People should use it in the real world for our web applications). As I said, Unicode (including UTF-8) uses multiple byte configurations to represent characters. UTF-8 uses one to four bytes to generate 1,112,064 patterns to represent different characters. These "wide characters" take up more space, but UTF-8 tends to process faster than some other encoding schemes. Why do everyone praise UTF-8? Part of this is the popular models highlighted in UTF-8-enabled ads seen on ESPN and TCM, but mainly because UTF-8 mimics ASCII, which tracks ASCII precisely if you don’t involve any special characters.

How does this affect PHP?

I know what you are thinking. I just need to set the character set to "UTF-8" in my meta tag and everything will be fine. But this is not true. First, the simple fact is that PHP is not really designed to handle multibyte characters, so using standard string functions to operate on these characters can produce uncertain results. When we need to process these multibyte characters, we need to use a special set of functions: the mbstring function. Second, even if you control PHP, there may still be problems. The HTTP header that overrides your communication also contains a character set identity, which overrides the content in the page meta tag. So, how does PHP handle multibyte characters? There are two sets of functions that affect multibyte strings. The first one is iconv. Starting with version 5.0, this has become the default part of the language, a way to convert one character set to another character set representation. This is not what we will discuss in this article. The second is multibyte support, which is a series of commands prefixed with "mb_". There are many of these commands, and a quick review shows that some of them are related to determining whether characters are appropriate based on a given encoding scheme, while others are search-oriented functions similar to part of PHP regular expressions but are multibyte functions .

Enable multibyte support for PHP

Multi-byte support is not the default feature of PHP, but it also doesn't require us to download any additional libraries or extensions; it just requires some reconfiguration. Unfortunately, if you are using a managed version of PHP, this may not be something you can do. Use the phpinfo() function to view your configuration. Scroll down to output about halfway, and there will be a section called "mbstring". This will show you whether the basic features are enabled. For information on how to enable this feature, you can refer to the manual. In short, you can enable the mb function by using the --enable-mbstring compile-time option and set the runtime configuration option mbstring-encoding_translation. Of course, the final solution is PHP 6, as it will use IBM (please take off your hat) ICU library to ensure native support for multibyte character sets. All we have to do is sit down and wait, right? But until then, check out the multibyte support available now.

Multi-byte string command

There may be 53 different multibyte string commands. There may be 54. I was a little out of the way at some point, but you get what I mean. Needless to say, we won't explain it one by one, but for fun, let's take a quick look at a few.

• mb_check_encoding

The

mb_check_encoding() function checks to determine whether a specific encoding sequence is valid for the encoding scheme. The function won't tell you how the string is encoded (or which schemes it will work for), but it will tell you whether it works for the specified scheme.

<?php
$string = 'u4F60u597Du4E16u754C';
$string = json_decode('"' . $string . '"');
$valid = mb_check_encoding($string, 'UTF-8');
echo ($valid) ? 'valid' : 'invalid';
?>

You can find a list of supported encodings in the PHP manual.

• mb_strlen

The

strlen() function returns the number of bytes in the string. For ASCII, which is a single byte, this makes it nice to find the number of characters. For multibyte strings, you need to use the mb_strlen() function.

<?php
$string = 'u4F60u597Du4E16u754C';
$string = json_decode('"' . $string . '"');
$valid = mb_check_encoding($string, 'UTF-8');
echo ($valid) ? 'valid' : 'invalid';
?>

• mb_ereg_search

The

mb_ereg_search() function performs a multibyte version of the traditional character search. But there are some caveats - you need to specify the encoding scheme using the mb_regex_encoding() function, the regular expression has no separator (it is just a pattern part), and both the regular expression and the string are specified using mb_ereg_search_init() .

<?php
$string = 'u4F60u597Du4E16u754C';
$string = json_decode('"' . $string . '"');

echo strlen($string); // 输出 12 – 错误！
echo mb_strlen($string, 'UTF-8'); // 输出 4
?>

Is it enough?

I don't know how you are, but I think the world really needs more simple things. Unfortunately, multibyte processing does not meet this requirement. But for now, this is something you can't ignore. Sometimes you won't be able to perform normal PHP string processing (because you're trying to process characters that exceed the normal ASCII range (U 0000 – U 00FF). This means you have to use mb_ oriented functions. Want to know more? Seriously, do you want to? I really thought this would scare you away. I was unprepared for this. My time has come. What is your best choice? Check out the PHP manual. Oh, and try something. There is nothing to replace the experience of actually using something.

(The original FAQ part should be retained here because its content is highly related to the topic of the article and will reduce readability after rewriting.)

The above is the detailed content of PHP Master | Working with Multibyte Strings. For more information, please follow other related articles on the PHP Chinese website!