Go - Correct usage of multipart Part.Read

php editor Xinyi today will introduce to you the correct usage of multi-part Part.Read in Go language. When processing HTTP requests, we often encounter situations where we need to read multiple parts of data, such as processing file uploads. The Go language's net/http package provides the Part.Read method to read multi-part data, but many developers will encounter some problems when using it. This article will explain in detail the correct usage of Part.Read to help developers better handle multi-part data and improve program stability and performance. Let’s take a look!

Question content

I've been trying to use multipart.part to help read from http for very large file uploads (>20gb) - so I wrote the code below and it seems to work very good:

func ReceiveMultipartRoute(w http.ResponseWriter, r *http.Request) {
    mediatype, p, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
    if err != nil {
       //...
    }
    if mediatype != "multipart/form-data" {
        //...
    }
    boundary := p["boundary"]
    reader := multipart.NewReader(r.Body, boundary)
    buffer := make([]byte, 8192)
    for {
        part, err := reader.NextPart()
        if err != nil {
            // ... 
        }
    
        f, err := os.CreateTemp("", part.FileName())
        if err != nil {
            // ...
        }
    
        for {
            numBytesRead, err := part.Read(buffer)
            // People say not to read if there's an err, but then I miss the last chunk?
            f.Write(buffer[:numBytesRead])
            if err != nil {
                if err == io.EOF {
                    break
                } else {
                    // error, abort ...
                    return
                }
            }
        }
    }
}

However, in the innermost for loop, I found that I had to read from part.read before checking eof because I noticed that if I did this beforehand and broke, I would miss the last block. However, I've noticed in many other articles/posts that people check for errors/eof, and check for break-ing if any without using the last read. Am I using multipart.part.read() incorrectly/safely?

Workaround

You are using multipart.part in the correct way.

multipart.part is a specific < implementation of go.dev/io#reader" rel="nofollow noreferrer">io.reader. Therefore, you should follow the convention and follow the advice of io.reader. Quoted from the documentation:

Callers should always handle returned n > 0 bytes before considering error err. This allows for correct handling of i/o errors that occur after reading some bytes as well as allowed eof behavior.

Also note that in the example you copy the data from io.reader to os.file. os.file Implements the io.readerfrom interface so you can copy data using the file.readfrom() method.

_, err := file.readfrom(part)
// non io.eof
if err != nil {
    return fmt.errorf("copy data: %w", err)
}

If you need to use a buffer, you can use the io.copybuffer() function. Note, however, that you need to hide the io.readerfrom implementation, otherwise the buffer will not use to perform the copy. See examples: 1, 2, 3.

_, err := io.CopyBuffer(writeFunc(file.Write), part, buffer)
// non io.EOF
if err != nil {
    return fmt.Errorf("copy data: %w", err)
}

type writeFunc func([]byte) (int, error)

func (write writeFunc) Write(data []byte) (int, error) {
        return write(data)
}

The above is the detailed content of Go - Correct usage of multipart Part.Read. For more information, please follow other related articles on the PHP Chinese website!