Why doesn't net/http consider timeout durations longer than 30 seconds?

#php editor Banana discovered when discussing the timeout duration in network requests, why did net/http not consider a timeout limit exceeding 30 seconds in the design? The timeout period means that after sending a request, if no response is received within the specified time, the request will be considered failed. In network requests, the setting of the timeout is very important. If it is too short, the request may fail, and if it is too long, resources will be wasted. Through analysis, the main reason is that the balance of performance and resources, as well as the uncertainty of the network environment, were taken into consideration during the design. Next, we will answer this question in detail.

Question content

Use golang 1.20.1.

In golang's net/http and context packages, I cannot set a timeout exceeding thirty seconds. Setting a shorter timeout works well, like

Code:

log.infof("elasticsearch url is %v", elasticsearchurl)
    client := &http.client{timeout: time.duration(time.second * 60)}
    req, err := http.newrequest("get", listbackupsurl, nil)
    if err != nil {
        internalerror(w, fmt.sprintf("error creating request: %v", err))
        return
    }
    req.setbasicauth(username, password)
    resp, err := client.do(req)
    if err != nil {
        // handle error
        internalerror(w, fmt.sprintf("error accessing elasticsearch: %v", err))
        return
    }

log:

i0503 23:01:55.973821       1 somecode.go:85] url is http://elasticsearch.example.ingest:9200
e0503 23:02:25.976345       1 caller_handler.go:63] 500 internal server error: error accessing elasticsearch: get "http://elasticsearch.example.ingest:9200/_cat/snapshots/object-store-repo?v&s=id": dial tcp 1.2.3.4:9200: i/o timeout

The timeout is thirty seconds, not sixty seconds.

If I use http.newrequestwithcontext(...) and use a context with the same timeout set, I get the same behavior:

Code:

log.infof("elasticsearch url is %v", elasticsearchurl)
    ctx, cancel := context.withtimeout(context.background(), time.duration(time.second * 60)) 
    defer cancel()
    req, err := http.newrequestwithcontext(ctx, "get", listbackupsurl, nil)
    if err != nil {
        internalerror(w, fmt.sprintf("error creating request: %v", err))
        return
    }
    req.setbasicauth(username, password)
    resp, err := client.do(req)
    if err != nil {
        // handle error
        internalerror(w, fmt.sprintf("error accessing elasticsearch: %v", err))
        return
    }

log:

i0503 23:31:10.941169       1 somecode.go:85] elasticsearch url is http://elasticsearch.example.ingest:9200
e0503 23:31:40.941642       1 caller_handler.go:63] 500 internal server error: error accessing elasticsearch: get "http://elasticsearch.example.ingest:9200/_cat/snapshots/object-store-repo?v&s=id": dial tcp 1.2.3.4:9200: i/o timeout

However, if I change the timeout to three seconds (time.duration(time.second * 3)) in either method, it works as expected:

I0503 23:44:17.622121       1 somecode.go:85] Elasticsearch URL is http://elasticsearch.example.ingest:9200
E0503 23:44:20.624795       1 caller_handler.go:63] 500 Internal Server Error: error accessing elasticsearch: Get "http://elasticsearch.example.ingest:9200/_cat/snapshots/object-store-repo?v&s=id": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Solution

I would isolate the problem to eliminate possibilities to help narrow down what is causing the problem. If possible, use a small snippet of code so you can process only what you want.

To test your infrastructure, httpstat will help you simulate remote timeouts. Example:

func main() {

    client := &http.Client{Timeout: time.Duration(time.Second * 200)}
    req, err := http.NewRequest("GET", "https://www.php.cn/link/1a59ef90d1ea801448e1567d0896a99f/504?sleep=120000", nil)
    if err != nil {
        log.Fatal(err)
        return
    }

    resp, err := client.Do(req)
    if err != nil {
        log.Fatal(err)
        return
    }

    fmt.Println(resp)
}

If you receive dial tcp ip:port: i/o timeout timeout at this time, then you need to check your operating system and firewall. Whatever timeouts you set in go should override the OS defaults, if you're getting timeouts this way it's probably a firewall (local or remote) that's causing this.

Alternatively, es may timeout if you are able to connect externally, although according to the documentation you should expect an error message directly from es. You can set the timeout for es directly in the url and test it.

Hope this helps.

The above is the detailed content of Why doesn't net/http consider timeout durations longer than 30 seconds?. For more information, please follow other related articles on the PHP Chinese website!