How to read a large CSV file in php and import it into the database example

The following editor will bring you an example of reading a large CSV file into the database with PHP. The editor thinks it’s pretty good, so I’ll share it with you now and give it as a reference. Let’s follow the editor and take a look

How does PHP read large CSV files and import them into the database?

For CSV files with millions of data items, the file size may reach hundreds of MB. If it is simply read, it is likely to time out or freeze.

In order to successfully import the data in the CSV file into the database, batch processing is very necessary.

The following function reads certain rows of data specified in the CSV file:

##

/**
 * csv_get_lines 读取CSV文件中的某几行数据
 * @param $csvfile csv文件路径
 * @param $lines 读取行数
 * @param $offset 起始行数
 * @return array
 * */
function csv_get_lines($csvfile, $lines, $offset = 0) {
 if(!$fp = fopen($csvfile, 'r')) {
 	return false;
 }
 $i = $j = 0;
	while (false !== ($line = fgets($fp))) {
		if($i++ < $offset) {
			continue; 
		}
		break;
	}
	$data = array();
	while(($j++ < $lines) && !feof($fp)) {
		$data[] = fgetcsv($fp);
	}
	fclose($fp);
 return $data;
}

Calling method:

$data = csv_get_lines('path/bigfile.csv', 10, 2000000);
print_r($data);

The function mainly uses the line positioning idea to realize the file pointer by skipping the starting line number position.

As for how the data is stored in the database, this article will not go into detail.

The above function has been tested on files within 500M and runs smoothly. It has not been tested on larger files. Please consider using it or improving it.

The above is the detailed content of How to read a large CSV file in php and import it into the database example. For more information, please follow other related articles on the PHP Chinese website!