A brief analysis of how vue implements file slicing upload

In the process of actual development projects, sometimes it is necessary to upload relatively large files. Then, the upload will be relatively slow, so the background may require the front-end to process the files. Slice upload is very simple. It is to cut a file stream of, for example, 1 G into several small file streams, and then request the interface to deliver these small file streams respectively.

Process Brief

To implement file slice import, first we use elementUI or the native upload tag to obtain the file file stream, and then we need to do One thing is to know whether this file has been uploaded before. We need to submit a unique identifier of this file to the backend, and then let the backend tell us whether the problem exists in the backend. At this time, the backend may return us three statuses :

• There are no files, all need to be uploaded.

• This file exists and the front end does not need to upload it again.

• Part of this file has been uploaded, and you need to continue uploading the parts that have not been uploaded.

There are only these three states. An important parameter to implement this step is the unique identifier of the file. This identifier uses the MD5 identification code of the file. So we first need to get the MD5 unique identification code of this file. [Related recommendations: vuejs video tutorial, web front-end development]

We know that when a file needs to be uploaded or needs to be partially uploaded, we need to perform a slicing operation . This is very simple. For example, the binary byte stream of a file is 1 G in total, and then it is usually cut into one piece of 5M. So if it is a 1G file, it needs to be cut into 205 pieces. Of course, the last piece is not necessarily 5M, and then we go The interface for uploading files in parts, initiating requests to import files into parts.

First of all, let’s make it clear that after we submit these 205 fragmented data streams, the background needs to be merged, that is, the 205 file streams submitted by our fragments are synthesized into a large file and saved. In this way, the files merged in the background are the files we will eventually submit for import. Since the backend needs to be merged, we upload them piece by piece. We have to let the backend know which part of the file each piece is, right? So when we submit a sharding request, we generally need to pass these parameters:

• chunk: The current shard number, usually starting from 0.
• chunkSize: The size of the fragment, usually 5M, which is 5242880 bytes.
• chunks: How many shards are there in total.
• file: The file stream of the current fragment.
• md5: MD5 unique identification code of the entire file, not fragmented.
• name: The name of the current file.
• size: Current fragment size (if the last block is not necessarily 5M).

This way the backend knows how to merge the final files. Of course, specific fields need to be defined by the backend, and then we can connect accordingly according to this idea.

OK, after all our 205 shard data are submitted, we need to go through another interface, which is to notify the backend. Okay, my front-end shards have been transmitted, and you can merge the files.

Then the backend merge is completed and the file is imported successfully!

Get the MD5 unique identification code of the file

Many people say that MD5 is not encryption. In fact, this idea is not very correct here. MD5 cannot encrypt files. Yes, it will only generate a unique code. So, oh, just think of it as encryption, but don’t think that submitting an MD5 code is just like submitting an encrypted file to the backend. First of all, this is a string, not Encrypted files, secondly, this thing cannot decrypt. We just generate a unique identifier for the file here, so that the backend can determine whether the file has existed on the server before. If there is a unique identifier, it means it has been uploaded before. You can just use the previous one without uploading it again. , after all, if the file is modified, the MD5 identification code will change.

How to get the MD5 encoding of a file is very simple. You need to use a library spark-md5 in vue.

One command to install

npm install --save spark-md5

Then we can write a method to encapsulate it.

Create a fileMd5Sum.js file:

import SparkMD5 from 'spark-md5'

export default {
  // md5值计算
  fileMd5Sum(file) {
    return new Promise(resolve => {
      let fileReader = new FileReader()
      let Spark = new SparkMD5.ArrayBuffer()
      fileReader.readAsArrayBuffer(file)
      fileReader.onload = function (e) {
        Spark.append(e.target.result)
        let md5 = Spark.end()
        resolve(md5)
      }
    });
  }
}

Then you can use it when you need to use it. Of course, what is returned here is a Promise, just .then Got it.

Or use async, await.

let md5Str = await this.fileMd5Sum.fileMd5Sum(file.raw)

文件切片

获取了文件MD5标识码，后台说需要提交，我们就需要把文件切片，从头提交或者是提交部分操作了，如果不需要的话直接走合并接口就可以了，走合并接口其实是告诉后台合并，我们要做的就是把其他除了文件相关的其他参数传递给后台罢了。

文件切片就是类似于字符串截取，这里是截取字节。获取需要的参数我们自己些就可以了。假设我们使用 elementUI 文件上传组件获取到了文件 file。

获取文件名 name

let fileName = file.name  // 获取文件名

分片文件大小 chunkSize

let chunkSize = 5 * 1024 * 1024   // 一般是 5M，具体多少看后端想要多少

文件切片 chunkList 列表

            let chunkList = []  // 创建一个数组用来存储每一片文件流数据
            if (file.size < chunkSize) {  // 如果文件大小小于5M就只有一片，不用切都
              chunkList.push(file.raw.slice(0))  // 文件流从第一个字节直接截取到最后就可以了
            } else {  // 如果文件大小大于5M 就需要切片了
              var start = 0, end = 0  // 创建两个变量 开始位置 结束位置
              while (true) {  // 循环
                end += chunkSize  // 结束为止 = 结束位置 + 每片大小
                let blob = file.raw.slice(start, end)  // 文件流从开始位置截取到结束位置
                start += chunkSize  // 截取完，开始位置后移
                if (!blob.size) {  // 如果截取不到了就退出
                  break;
                }
                chunkList.push(blob)  // 把截取的每一片数据保存到数组
              }
            }

切片总数 chunks

我们上一步已经获取到每片文件流的数组了，所以说呢，直接获取就可以了。

let chunks = chunkList.length

切片大小 size

我们是按照 5 M 切割的，所以说每片大小应该是 5 * 1024 * 1024 但是呢，不对，因为最后一片不一定就是正好的 5 M，所以说我们可直接 .size 获取一下大小。比如：

chunkList[0].size  // 获取第1片大小

参数都找齐了，然后就走切片提交接口开始提交数据就可以了。

合并

当我们把分片数据全部提交成功，后台返回说切片文件保存成功之后，我们就可以走最后一个接口，提交就可以了。

好了，就这样！完成！！！

（学习视频分享：vuejs入门教程、编程基础视频）

The above is the detailed content of A brief analysis of how vue implements file slicing upload. For more information, please follow other related articles on the PHP Chinese website!