How to use context to implement request result merging in Go

How to use context to implement request result merging in Go

In modern distributed systems, it is often necessary to initiate multiple concurrent requests at the same time and merge the results of these requests. The context package in the Go language provides an elegant way to manage concurrent requests in such scenarios and ensures that invalid requests can be terminated as early as possible when the request needs to be canceled.

This article will introduce how to use context to merge request results and provide relevant code examples.

First, let us understand some key concepts and usage methods in the context package.

1. Context: context is a context object that can be used to control goroutine. In concurrent requests, we can pass the context object to each request and use the context object to manage and control these requests.
2. WithCancel: Use the WithCancel(parent) function to create a new child context and return a cancel function. When we want to cancel the context, we only need to call the cancel function.
3. WithTimeout: Use the WithTimeout(parent, timeout) function to create a context with a timeout. After the specified time, the context will be automatically canceled.

After understanding these basic concepts, we can start to implement the merging of request results.

First, let us assume that we have a service that needs to make requests to multiple external APIs at the same time and merge their results. We can use context to achieve the following functions:

1. Create a parent context and create a child context for each request.
2. In each goroutine, use these subcontexts to send requests and wait for the results.
3. When all requests are completed, the results are returned through the channel or other methods.

Next, let us look at a sample code that uses context to implement request result merging:

package main

import (
    "context"
    "fmt"
    "net/http"
    "sync"
    "time"
)

func fetchData(ctx context.Context, url string, wg *sync.WaitGroup, ch chan<- string) {
    defer wg.Done()

    req, err := http.NewRequest("GET", url, nil)
    if err != nil {
        ch <- fmt.Sprintf("%s failed: %v", url, err)
        return
    }

    select {
    case <-ctx.Done():
        ch <- fmt.Sprintf("%s cancelled", url)
        return
    default:
    }

    client := http.DefaultClient
    resp, err := client.Do(req)
    if err != nil {
        ch <- fmt.Sprintf("%s failed: %v", url, err)
        return
    }

    select {
    case <-ctx.Done():
        ch <- fmt.Sprintf("%s cancelled", url)
    case <-time.After(1 * time.Second):
        body := make([]byte, 1024)
        _, _ = resp.Body.Read(body)
        ch <- fmt.Sprintf("%s fetched: %s", url, body)
    }

    resp.Body.Close()
}

func main() {
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()

    urls := []string{"https://www.google.com", "https://www.bing.com", "https://www.baidu.com"}

    var wg sync.WaitGroup
    results := make(chan string, len(urls))

    for _, url := range urls {
        wg.Add(1)
        go fetchData(ctx, url, &wg, results)
    }

    go func() {
        wg.Wait()
        close(results)
    }()

    for res := range results {
        fmt.Println(res)
    }
}

In the above sample code, we first create a parent context, and then A subcontext is created for each request.

In the fetchData function, we use the select statement to check whether the context has been cancelled. If canceled, the request is terminated immediately. If it is not canceled, send the request and wait for the result.

Finally, we start multiple goroutines in the main function to process the request and return the results through the channel. We use sync.WaitGroup to wait for all requests to complete, and can cancel the entire request process at any time through the cancel function.

Summary:

By using the context package, we can manage concurrent requests gracefully and cancel invalid requests in a timely manner when needed. The above sample code shows how to use context to merge request results. By rationally using context, we can improve the system's concurrent processing capabilities while maintaining the clarity and readability of the code.

The key to using context is to correctly use functions such as WithCancel and WithTimeout to create sub-contexts, and use select statements in goroutine to check whether to cancel or time out. In this way, we can terminate invalid requests in time when needed and merge the valid request results.

By in-depth understanding and flexible use of the context package, we can better build concurrent and reliable distributed systems.

The above is the detailed content of How to use context to implement request result merging in Go. For more information, please follow other related articles on the PHP Chinese website!