Detailed explanation of how PHP can quickly sort millions of data

In my daily work, I often receive notifications to send group messages to website members through in-site letters, mobile phone text messages, and emails. The user list is usually provided by other colleagues, and there will inevitably be duplication. In order to avoid repeated sending , so I need to deduplicate the user list they provide before sending information. Next, I will use the uid list to talk about how I use the php array to deduplicate. I hope to be helpful.

If you get a uid list with more than one million rows, the format is as follows:

10001000
10001001
10001002
......
10001000
......
10001111

In fact, using the characteristics of php arrays, it is easy to sort the duplicates. Let’s first take a look at the php arrays Definition: An array in PHP is actually an ordered map. A map is a type that associates values ​​to keys. This type is optimized in many ways, so it can be treated as a real array, or list (vector), hash table (which is an implementation of map), dictionary, set, stack, queue and many more possibilities. The value of an array element can also be another array. Tree structures and multidimensional arrays are also allowed.
In PHP arrays, keys are also called indexes and are unique. We can use this feature to perform deduplication. The sample code is as follows:

<?php
//定义一个数组，用于存放排重后的结果
$result = array();
//读取uid列表文件
$fp = fopen(&#39;test.txt&#39;, &#39;r&#39;);
while(!feof($fp))
{
$uid = fgets($fp);
$uid = trim($uid);
$uid = trim($uid, "\r");
$uid = trim($uid, "\n");
if($uid == &#39;&#39;)
{
continue;
}
//以uid为key去看该值是否存在
if(empty($result[$uid]))
{
$result[$uid] = 1;
}
}
fclose($fp);
//将结果保存到文件
$content = &#39;&#39;;
foreach($result as $k => $v)
{
$content .= $k."\n";
}
$fp = fopen(&#39;result.txt&#39;, &#39;w&#39;);
fwrite($fp, $content);
fclose($fp);
?>

With more than 20 lines of code, just It can deduplicate more than one million data, the efficiency is also good, and it is very practical. Mobile phone numbers and emails can also be deduplicated in this way.
Also, this method can also be used to deduplicate two files. If you have two uid list files, the format is the same as the uid list above. The sample program is as follows:

<?php
//定义数组，用于存放排重后的结果
$result = array();
//读取第一个uid列表文件，放入$result_1
$fp = fopen(&#39;test_1.txt&#39;, &#39;r&#39;);
while(!feof($fp))
{
$uid = fgets($fp);
$uid = trim($uid);
$uid = trim($uid, "\r");
$uid = trim($uid, "\n");
if($uid == &#39;&#39;)
{
continue;
}
//以uid为key写入$result，如有重复就会覆盖
$result[$uid] = 1;
}
fclose($fp);
//读取第二个uid列表文件，并进行排重操作
$fp = fopen(&#39;test_2.txt&#39;, &#39;r&#39;);
while(!feof($fp))
{
$uid = fgets($fp);
$uid = trim($uid);
$uid = trim($uid, "\r");
$uid = trim($uid, "\n");
if($uid == &#39;&#39;)
{
continue;
}
//以uid为key去看该值是否存在
if(empty($result[$uid]))
{
$result[$uid] = 1;
}
}
fclose($fp);
//$result里保存的就排重以后的结果，可以输出到文件，代码省略
?>

If you think about it carefully, it is not difficult to find that using this feature of arrays can solve more problems in our work.

The above is the detailed content of Detailed explanation of how PHP can quickly sort millions of data. For more information, please follow other related articles on the PHP Chinese website!