Solve the problem of garbled characters when splitting gbk Chinese, split gbk Chinese garbled characters

Solve the problem of garbled characters when splitting gbk Chinese, split gbk Chinese garbled characters_PHP tutorial

WBOY

Release： 2016-07-12 08:58:37

Original

1057 people have browsed it

The problem of garbled characters in gbk Chinese is solved. Split gbk Chinese garbled characters

Recently I encountered a magical word "韬(tao)".

The specific process is as follows:

<span>1</span> <span>$list</span> = <span>explode</span>('|', 'abc弢|bc'<span>);
</span><span>2</span> <span>var_dump</span>(<span>$list</span>);

Copy after login

Get the result of this segmentation.

Contrary to imagination, the result is actually like this:

<span>array</span>(3<span>) {
  [</span>0]=>
  <span>string</span>(4) "<span>abc?
  [1]=>
  string(0) </span>""<span>
  [2]=>
  string(2) </span>"bc"<span>
}</span>

Copy after login

Garbled characters appeared, and an empty element appeared inexplicably.

The reason is that it turns out that the gbk encoding of the character "韬" is 8f7c, and the ASCII of | is 7c, so explode cuts the second ASCII of "韬" as |.

Since it is a double-byte problem, we solved it with mbstring.

Unfortunately, PHP does not have a function like mb_explode. After searching, I found a mb_split.

<span>array</span> mb_split ( <span>string</span> <span>$pattern</span> , <span>string</span> <span>$string</span> [, int <span>$limit</span> = -1 ] )

Copy after login

No place to declare encoding. Take a closer look and you will see that it is encoded through the mb_regex_encoding declaration.

So I wrote the following code:

<span>1</span> mb_regex_encoding('gbk'<span>);
</span><span>2</span> <span>$list</span> = mb_split('\|', 'abc弢|bc'<span>);
</span><span>3</span> <span>var_dump</span>(<span>$list</span>);

Copy after login

As a result, php reported an error, mb_regex_encoding does not recognize gbk, embarrassing.

Then use it to know:

<span>1</span> mb_regex_encoding('gb2312'<span>);
</span><span>2</span> <span>$list</span> = mb_split('\|', 'abc弢|bc'<span>);
</span><span>3</span> <span>var_dump</span>(<span>$list</span>);

Copy after login

Result:

<span>array</span>(3<span>) {
  [</span>0]=>
  <span>string</span>(4) "<span>abc?
  [1]=>
  string(0) </span>""<span>
  [2]=>
  string(2) </span>"bc"<span>
}</span>

Copy after login

I found that this method is of no use. ,

As for the reason? The word "Tao" is not in the code set of GB2312! ! ! ! ! However, this function does not support the encoding sets (GBK, GB18030) with this character! ! ! ! !

Since this is not easy to use, maybe a universal regular expression is OK. So we got the following code:

<span>1</span> <span>var_dump</span>(<span>preg_match_all</span>('/([^\|])*/', 'abc弢|bc', <span>$matches</span><span>));
</span><span>2</span> <span>var_dump</span>(<span>$matches</span>);

Copy after login

Result:

int(2<span>)
</span><span>array</span>(2<span>) {
  [</span>0]=>
  <span>array</span>(2<span>) {
    [</span>0]=>
    <span>string</span>(4) "<span>abc?
    [1]=>
    string(2) </span>"bc"<span>
  }
  [1]=>
  array(2) {
    [0]=>
    string(1) </span>"?<span>
    [</span>1]=>
    <span>string</span>(1) "c"<span>
  }
}</span>

Copy after login

Okay, I thought too much.

Now let’s study how to describe this scene using regular expressions.

For reference, Brother Bird’s blog: How to solve the problem of garbled characters when splitting GBK into Chinese. Unfortunately, I, who have relatively low regular expression ability, still can't think of a suitable regular expression (if there are masters who have come up with this regular expression, I hope they can tell me).

I have no choice but to use substr after much thought:

<span> 1</span> <span>function</span> mb_explode(<span>$delimiter</span>, <span>$string</span>, <span>$encoding</span> = <span>null</span><span>){
</span><span> 2</span>     <span>$list</span> = <span>array</span><span>();
</span><span> 3</span>     <span>is_null</span>(<span>$encoding</span>) && <span>$encoding</span> =<span> mb_internal_encoding();
</span><span> 4</span>     <span>$len</span> = mb_strlen(<span>$delimiter</span>, <span>$encoding</span><span>);
</span><span> 5</span>     <span>while</span>(<span>false</span> !== (<span>$idx</span> = mb_strpos(<span>$string</span>, <span>$delimiter</span>, 0, <span>$encoding</span><span>))){
</span><span> 6</span>         <span>$list</span>[] = mb_substr(<span>$string</span>, 0, <span>$idx</span>, <span>$encoding</span><span>);
</span><span> 7</span>         <span>$string</span> = mb_substr(<span>$string</span>, <span>$idx</span> + <span>$len</span>, <span>null</span>, <span>$encoding</span><span>);
</span><span> 8</span> <span>    }   
</span><span> 9</span>     <span>$list</span>[] = <span>$string</span><span>;
</span><span>10</span>     <span>return</span> <span>$list</span><span>; 
</span><span>11</span> }

Copy after login

Test code:

<span>1</span> <span>$a</span> = 'abc弢|bc'<span>;
</span><span>2</span> 
<span>3</span> <span>var_dump</span>(mb_explode('|', <span>$a</span>, 'gbk'<span>));
</span><span>4</span> <span>var_dump</span>(mb_explode('bc', <span>$a</span>, 'gbk'<span>));
</span><span>5</span> <span>var_dump</span>(mb_explode('弢', <span>$a</span>, 'gbk'));

Copy after login

Result:

<span>array</span>(2<span>) {
  [</span>0]=>
  <span>string</span>(5) "abc弢"<span>
  [</span>1]=>
  <span>string</span>(2) "bc"<span>
}
</span><span>array</span>(3<span>) {
  [</span>0]=>
  <span>string</span>(1) "a"<span>
  [</span>1]=>
  <span>string</span>(3) "弢|"<span>
  [</span>2]=>
  <span>string</span>(0) ""<span>
}
</span><span>array</span>(2<span>) {
  [</span>0]=>
  <span>string</span>(3) "abc"<span>
  [</span>1]=>
  <span>string</span>(3) "|bc"<span>
}</span>

Copy after login

This way you can get the correct results.