Implementation code for excluding duplicate data from millions of data in PHP array_PHP tutorial

If you get a uid list with more than one million rows, the format is as follows:

Copy the code The code is as follows:

10001000
10001001
10001002
......
10001000
......
10001111　To sort out the duplication, let’s first take a look at the definition of a PHP array: an array in PHP is actually an ordered map. A map is a type that associates values ​​to keys. This type is optimized in many ways, so it can be treated as a real array, or a list (vector), a hash table (an implementation of a map), a dictionary, a set, a stack, a queue, and many more possibilities. The value of an array element can also be another array. Tree structures and multidimensional arrays are also allowed.

In PHP arrays, keys are also called indexes and are unique. We can use this feature to perform deduplication. The sample code is as follows:



Copy the code

The code is as follows: //Define an array to store the results after deduplication

$result = array ();
//Read uid list file
$fp = fopen('test.txt', 'r');

while(!feof($fp))
{
$uid = fgets($fp);
$uid = trim($uid);
$uid = trim($uid, "r");
$uid = trim($ uid, "n");

if($uid == '')
{
continue;
}
//Us uid as the key to see if the value exists
if(empty($result[$uid]))
{
$result[$uid] = 1;
}
}

fclose($fp) ;

//Save the result to file
$content = '';
foreach($result as $k => $v)
{
$content .= $k."n";
}
$fp = fopen('result.txt', 'w');
fwrite($fp, $content);
fclose($fp) ;
?>　


With more than 20 lines of code, more than one million data can be deduplicated. The efficiency is also good and very practical. Mobile phone numbers and emails can also be deduplicated in this way.

Also, this method can also be used to deduplicate two files. If you have two uid list files, the format is the same as the uid list above. The sample program is as follows:



Copy the code

The code is as follows: //Define an array to store the results after deduplication

$result = array();
//Read the first uid list file and put $result_1
$fp = fopen('test_1.txt', 'r');
while( !feof($fp))
{
$uid = fgets($fp);
$uid = trim($uid);
$uid = trim($uid, "r") ;
$uid = trim($uid, "n");
if($uid == '')
{
continue;
}
//Uid is The key is written to $result. If there is a duplicate, it will be overwritten.
$result[$uid] = 1;
}
fclose($fp);
//Read the second uid list file , and perform deduplication operation
$fp = fopen('test_2.txt', 'r');
while(!feof($fp))
{
$uid = fgets($ fp);
$uid = trim($uid);
$uid = trim($uid, "r");
$uid = trim($uid, "n");
if($uid == '')
{
continue;
}
//Us uid as the key to see if the value exists
if(empty($result[$uid] ))
{
$result[$uid] = 1;
}
}
fclose($fp);
//What is saved in $result is after deduplication The result can be output to a file, and the code is omitted
?>


If you think about it carefully, it is not difficult to find that using this feature of arrays can solve more problems in our work.

http://www.bkjia.com/PHPjc/321981.html

www.bkjia.comtruehttp: //www.bkjia.com/PHPjc/321981.htmlTechArticleIf you get a uid list with more than one million rows, the format is as follows: Copy the code as follows: 10001000 10001001 10001002 ...... 10001000 ...... 10001111 In fact, using php arrays...

Statement of this Website

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress images for free

Clothoff.io

AI clothes remover

AI Hentai Generator

Generate AI Hentai for free.

Show More

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

R.E.P.O. Best Graphic Settings

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Assassin's Creed Shadows: Seashell Riddle Solution

2 weeks ago By DDD

R.E.P.O. How to Fix Audio if You Can't Hear Anyone

4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

WWE 2K25: How To Unlock Everything In MyRise

1 months ago By 尊渡假赌尊渡假赌尊渡假赌

Show More

Hot Tools

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Show More

Hot Topics

Where is the login entrance for gmail email?

7500

15

CakePHP Tutorial

1377

52

What is the format of the account name of steam

78

11

win11 activation key permanent

52

19

nyt connections hints and answers

19

54

Show More