Parallelize File Downloads in Golang Using Goroutines
Question:
Can we leverage goroutines to download multiple files concurrently?
Code Context:
<code class="go">package main import ( "fmt" "io" "io/ioutil" "net/http" "net/url" "os" "path/filepath" "sync" ) // ... (existing Dropbox access token handling code) var wg sync.WaitGroup func downloadFile(file File, token TokenResponse) { // Acquire WaitGroup counter wg.Add(1) defer wg.Done() // Release counter when function returns downloadURL := fmt.Sprintf("https://api-content.dropbox.com/1/files/dropbox/%s?access_token=%s", file.Path, token.AccessToken) resp, err := http.Get(downloadURL) if err != nil { panic(err) } defer resp.Body.Close() filename := filepath.Base(file.Path) outFile, err := os.Create(filename) if err != nil { panic(err) } defer outFile.Close() io.Copy(outFile, resp.Body) } func main() { // ... (existing Dropbox authorization and file list code) // Spawn goroutines for file downloads for _, file := range flr.FileList { go downloadFile(file, tr) if count >= 2 { break } } // Wait for all goroutines to complete before exiting wg.Wait() }</code>
This modified code uses a sync.WaitGroup to ensure that the main goroutine doesn't exit until all file downloads are complete. This allows goroutines to download files in parallel, improving performance.
The above is the detailed content of ## Can Goroutines Speed Up File Downloads in Golang?. For more information, please follow other related articles on the PHP Chinese website!