The reading of large CSV files has been described before (code example of PHP reading and processing large CSV files line by line), but there are still some problems in how to quickly and completely operate large files.
1. How to quickly get the total number of lines of a large CSV file?
Method 1: Get the file content directly, use newline characters to split it to get the total number of lines. This method is feasible for small files, but not feasible when processing large files;
Method 2: Use fgets to traverse line by line, and get Get the total number of lines. This method is better than method one, but there is still the possibility of timeout for large files;
Method three: Use the SplFileObject class to directly position the pointer to the end of the file, and get the total number of lines through the SplFileObject::key method. This method is feasible and efficient.
Specific implementation method:
Still using PHP's SplFileObject class to achieve quick positioning through the seek method.
3. Based on the above two points, organize it into a class for reading csv files:
public function get_data($length = 0, $start = 0) {
if(!$this->_open_file()) {
return false;
}
$length = $length ? $length : $this->get_lines();
$start = $start - 1;
$start = ($start < 0) ? 0 : $start;
$data = array();
$this->spl_object->seek($start);
while ($length-- && !$this->spl_object->eof()) {
$data[] = $this->spl_object->fgetcsv();
$this->spl_object->next();
}
return $data;
}
public function get_lines() {
if(!$this->_open_file()) {
return false;
}
$this->spl_object-> seek(filesize($this->csv_file));
return $this->spl_object->key();
}
public function get_error() {
return $this->error;
}
}
$csv_file = 'path/bigfile.csv';
$csvreader = new CsvReader($csv_file);
$line_number = $csvreader->get_lines();
$data = $csvreader->get_data(10);
echo $line_number, chr(10);
print_r($data);
In fact, the above-mentioned CsvReader class is not only for CSV large files, it can also be used for large or very large files of other text types, provided that the fgetcsv method in the class is slightly changed to current.