Golang network request concurrency

With the development of the Internet, network requests are a very important link, and many applications need to obtain data through network requests. For high-concurrency scenarios, network request concurrency is particularly important. This article will introduce how to use Golang for concurrent processing of network requests.

1. Golang’s concurrency model

Golang is a language that supports concurrent programming. Its concurrency model is based on goroutine and channel.

Goroutine is a lightweight thread that can run multiple goroutines simultaneously in a process. In golang, you can easily create a goroutine through the go keyword, for example:

func test() {
    fmt.Println("hello, world!")
}

func main() {
    go test() // 创建一个goroutine
    fmt.Println("main")
}

In the above code, the test function is created into a goroutine and runs at the same time when the main function is executed.

Channel is a communication mechanism that can be used to pass data between goroutines. The creation and use of channels is very simple, for example:

ch := make(chan int)
go func() {
    ch <- 1 // 发送数据到通道
}()

x := <-ch // 从通道中读取数据

In this code, we create an integer channel ch, and use an anonymous function to send an integer 1 to channel ch in another goroutine. Then, an integer is read from channel ch in the main goroutine.

2. Golang network request concurrency

In golang, the net/http package in the standard library provides support for the http protocol. We can use this package to easily initiate http requests, for example:

resp, err := http.Get("http://www.google.com")

This code can initiate an http GET request to Google and get the response. However, if you need to initiate requests for multiple URLs at the same time, each request will need to wait for the previous request to complete before it can be initiated, which is inefficient.

In this case, we can use goroutine and channels for concurrent processing. For example:

func main() {
    urls := []string{"http://www.google.com", "http://www.sina.com.cn", "http://www.baidu.com"}

    ch := make(chan string)
    for _, url := range urls {
        go func(u string) {
            resp, err := http.Get(u)
            if err != nil {
                ch <- fmt.Sprintf("error: %s", err)
                return
            }
            defer resp.Body.Close()

            ch <- fmt.Sprintf("url: %s, status: %s", u, resp.Status)
        }(url)
    }

    for range urls {
        fmt.Println(<-ch)
    }
}

In the above code, we created a string type channel ch to receive information about each request result. We use a for loop to iterate through the list of URLs and use a goroutine to make http GET requests to each URL individually.

In goroutine, we first initiate a request and close the response body after the request is completed. Then use channel ch to send the requested result information. The outer for loop is responsible for receiving the result information returned by the channel and printing it to the console.

By using goroutine and channels, we can initiate multiple network requests at the same time to improve concurrency performance.

3. Golang network request timeout processing

When concurrently requesting the network, we need to use a timeout mechanism to control the time range of the request in the face of network anomalies or long server response times.

In golang, we can use the context package to add a timeout mechanism, for example:

func main() {
    urls := []string{"http://www.google.com", "http://www.sina.com.cn", "http://www.baidu.com"}

    ch := make(chan string)
    ctx, cancel := context.WithTimeout(context.Background(), time.Second*2)
    defer cancel()
    
    for _, url := range urls {
        go func(ctx context.Context, u string) {
            req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
            if err != nil {
                ch <- fmt.Sprintf("error: %s", err)
                return
            }
            
            resp, err := http.DefaultClient.Do(req)
            if err != nil {
                ch <- fmt.Sprintf("error: %s", err)
                return
            }
            defer resp.Body.Close()

            ch <- fmt.Sprintf("url: %s, status: %s", u, resp.Status)
        }(ctx, url)
    }

    for range urls {
        fmt.Println(<-ch)
    }
}

In the above code, we use the WithTimeout function of the context package to create a timeout of 2 seconds. context. Then, in the goroutine, an http request with context is created using the http.NewRequestWithContext function and the request is sent. During the request process, we use the ctx.Done() channel to listen for the timeout signal. If the timeout signal is triggered, the requested operation is interrupted.

By using the timeout mechanism of the context package, we can control the time of the network request and handle exceptions that may occur during the request process.

4. Summary

In this article, we introduced Golang’s concurrency model, how to use goroutine and channels for concurrent processing of network requests, and how to use the context package to handle the timeout mechanism of network requests. .

As a common operation in Internet applications, network requests have a crucial impact on improving system performance. By processing network requests concurrently and using the timeout mechanism, we can improve the concurrency performance and stability of the application and improve the user experience.

The above is the detailed content of Golang network request concurrency. For more information, please follow other related articles on the PHP Chinese website!