Technical Guide to Uploading and Downloading Large Files in PHP

With the advent of the mobile Internet era, the need to transfer large files is becoming more and more common. Among them, PHP, as a popular programming language, has good performance in uploading and downloading large files. In this article, we will learn the large file upload and download technology in PHP, including important technologies such as how to handle large files, chunked uploads, resumed uploads, and asynchronous downloads.

1. Methods for processing large files in PHP

The most important step when processing large files in PHP is to plan a suitable memory usage plan. We cannot read the entire large file directly into memory because this will cause memory overflow and possibly crash the entire program. So, what is the method for processing large files in PHP?

First of all, we can use the concept of "file pointer" to solve this problem. A "file pointer" is a pointer located in a file through which we can read a certain part of the file instead of reading the entire file at once. In PHP, we can use the fopen() and fread() functions to open and read files.

Secondly, we can use the "streaming" method. First split large files into multiple parts and then upload or download them piece by piece. This avoids reading the entire large file at once and ensures data integrity.

2. Large file upload

For HTTP communication, PHP’s default upload file size is 2MB, so we need to change the upload_max_filesize option in the php.ini configuration file to allow larger files to be uploaded. .

But even after changing the php.ini file, we are still facing issues while uploading large files. How can we avoid memory leaks and program crashes during uploading? The answer is to use chunked upload technology.

1. Chunked upload

The so-called chunked upload is to divide a large file into fixed-size chunks, and then upload them piece by piece until the upload is completed. This method avoids reading the entire file at once and reduces server memory pressure.

The specific implementation is as follows:

Front-end code:

<input type="file" name="file" id="file">

var chunkSize = 1 * 1024 * 1024; //每一块的大小为1MB

var file = document.getElementById('file').files[0];
var name = file.name;
var size = file.size;

var chunkNum = Math.ceil(size / chunkSize); //总块数

for(var i=0;i<chunkNum;i++){

    var start = i*chunkSize;
    var end = Math.min(size,start+chunkSize);

    var chunk = file.slice(start,end); //将文件切成每个chunkSize大小的块

    //使用FormData上传
    var formData = new FormData();
    formData.append('chunk', chunk);
    formData.append('filename', i+'_'+name); //在文件名前加上块的编号

    //发起已经分割好的块的上传请求
    send(chunkUrl,formData);

}

Back-end code:

$chunk = $_FILES['chunk']; //获取文件块
$filename = $_POST['filename']; //获取文件名

//上传文件块
move_uploaded_file( $chunk['tmp_name'] , $upload.$filename);

echo "success";

The above code implements dividing a large file into several 1MB sized files upload in chunks to spread the pressure.

2. Breakpoint Resume

Another issue worth noting is that if the upload of a large file is accidentally canceled, how do we continue uploading where it was interrupted?

The answer is to use breakpoint resume technology. We can record the progress of file upload while uploading the file, and then when the user uploads again, cache the unfinished file blocks of the last upload so that they can be uploaded again later.

The specific implementation is as follows:

Front-end code:

var chunkSize = 1 * 1024 * 1024; //每一块的大小为1MB

var file = document.getElementById('file').files[0];
var name = file.name;
var size = file.size;

var chunkNum = Math.ceil(size / chunkSize); //总块数

for(var i=0;i<chunkNum;i++){

    var start = i*chunkSize;
    var end = Math.min(size,start+chunkSize);

    var chunk = file.slice(start,end); //将文件切成每个chunkSize大小的块

    //使用FormData上传
    var formData = new FormData();
    formData.append('chunk', chunk);
    formData.append('filename', i+'_'+name); //在文件名前加上块的编号
    formData.append('chunkNum',chunkNum); //总块数
    formData.append('index',i); //当前块的编号

    //将文件分块上传
    send(chunkUrl,formData,function(){ 

        uploaded[i] = 1; //标记该块已上传
        var uploadedAll = uploaded.every(v => v == 1); //检查是否所有块都已上传

        //如果所有块都已上传，则在服务端组装文件
        if(uploadedAll){

            var formData = new FormData();
            formData.append('name', name);
            formData.append('chunkNum',chunkNum); //总块数

            //发起文件组装请求
            send(mergeUrl,formData,function(response){

                console.log(response);

            });

        }

    });

}

Back-end code:

$chunk = $_FILES['chunk']; //获取文件块
$filename = $_POST['filename']; //获取文件名
$chunkNum = $_POST['chunkNum']; //获取文件总块数
$index = $_POST['index']; //获取当前块的编号

//上传文件块
move_uploaded_file( $chunk['tmp_name'] , $upload.$filename);

//上传完成后检查所有块是否已上传
$uploaded = $this->isUploaded($upload,$chunkNum);
if($uploaded){
    $this->mergeFile($upload,$chunkNum,$name); //组装文件
}

echo "success";

The above code realizes the uploading of files in pieces and uploads them to the client Mark the progress of the upload on the server side, and check whether the upload is completed on the server side. When the upload is complete, all uploaded file chunks are spliced.

3. Large file download

When downloading large files in PHP, we also need to solve the problems of memory leaks and program crashes. For this we can use asynchronous download technology.

Asynchronous downloading means placing the download request in a separate thread to avoid blocking the main thread of the server. In PHP, we can use the exec() function to start a separate thread.

The specific implementation is as follows:

Front-end code:

<a href="#" onclick="download()">下载</a>

function download(){

    var filename = 'test.zip'; //准备下载文件名

    //发起异步下载请求
    var ajax = new XMLHttpRequest();
    ajax.open('GET', '/download.php?filename='+filename, true);
    ajax.send();

}

Back-end code:

$action = isset($_GET['action']) ? $_GET['action'] : 'download'; //获取下载动作
$filename = $_GET['filename']; //获取文件名

if($action == 'download'){

    //启动异步下载
    exec('php download.php '.$filename.' > /dev/null &'); 

}else{

    //文件下载
    header("Content-type: application/octet-stream");
    header("Content-Disposition: attachment; filename=".$filename);
    header("Accept-Ranges: bytes");

    $file = fopen($download.$filename,"r");
    if($file){

        while(!feof($file)){
            print(fread($file, 1024*8 ));
            flush();
            ob_flush();
        }

    }
    fclose($file);

}

The above code implements an asynchronous download request initiated on the client. The end uses the exec() function to start a separate thread for downloading, thus avoiding the problem of the main thread being blocked. At the same time, the memory problem is also solved when downloading files, using chunked download and output to the buffer.

4. Summary

In this article, we learned the large file upload and download technology in PHP, mainly including how to handle large files, chunked upload, breakpoint resume and asynchronous download and other technologies. Through the guidance of this article, we can better master the techniques for processing large files in PHP, and apply these techniques in actual projects to achieve efficient transmission of large files.

The above is the detailed content of Technical Guide to Uploading and Downloading Large Files in PHP. For more information, please follow other related articles on the PHP Chinese website!