


Golang uses multipart to upload large files to external API. How to avoid `io.Copy(io.Writer, io.Reader)` problems
Feb 06, 2024 am 09:45 AMMy goal is to use golang's built-in net/http package to upload a large file to POST https://somehost /media
.
HTTP format for Api call
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
In golang, this is the code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
|
How to avoidio.Copy(io.Writer, io.Reader)
Problems
The above code works fine, but on the line _, errFile3 = io.Copy(part3, file)
. This essentially copies everything in the file into main memory.
How to avoid this situation?
Is there any way I can stream large files to the api via multipart-formdata
?
The program will be run on the remote server. May crash if opening a very large file.
Correct Answer
Use io.Pipe and a goroutine to copy the file to the request without loading the entire file in memory.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
|
The above is the detailed content of Golang uses multipart to upload large files to external API. How to avoid `io.Copy(io.Writer, io.Reader)` problems. For more information, please follow other related articles on the PHP Chinese website!

Hot tools Tags

Hot Article

Hot tools Tags

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

How to use PHP to call web services and APIs?

Exploring the boundaries of agents: AgentQuest, a modular benchmark framework for comprehensively measuring and improving the performance of large language model agents

Can software compiled by Mingw be used in a Linux environment?

Let Siri no longer be mentally retarded! Apple defines a new client-side model, which is 'much better than GPT-4. It gets rid of text and visually simulates screen information. The minimum parameter model is still 5% better than the baseline system.

Metis: The ReGenesis project we launched is a rebirth of the blockchain ecosystem

How to solve external resource calls and access in PHP development
