Multithreaded requests in go and not getting high RPS

php editor Xigua noticed that when using Go language to make multi-threaded requests, sometimes it is not possible to obtain high request per second (RPS) speed. Although the Go language is excellent at concurrency processing, in some cases, multi-threaded requests are not very efficient. This may be due to network latency, resource competition, etc. In this article, we will explore this problem and provide some possible solutions to improve the RPS of multi-threaded requests in Go language.

Question content

I'm trying to write a multi-threaded client to test my server. When I use 2 goroutines, everything is fine, I get 50k rps and my cpu load is normal, but when I create more than 2 goroutines, the rps drops to 3k but my cpu load exceeds. Although when I run the client code multiple times (like running the same code on 3 consoles at the same time) I get more rps like 80k rps.

This is my client code

package main

import (
    "fmt"
    "net/http"
    "os"
    "sync"
    "time"
)

func main() {
    requesturl := fmt.sprintf("http://localhost:%d/home", 3333)
    var wg sync.waitgroup
    wg.add(4)
    req, err := http.newrequest(http.methodget, requesturl, nil)
    if err != nil {
        fmt.printf("client: could not create request: %s\n", err)
        os.exit(1)
    }
    for i := 0; i < 4; i++ {
        go func() {
            defer wg.done()
            client := http.client{
                timeout: 30 * time.second,
            }
            for {
                client.do(req)
            }
        }()
    }
    wg.wait()
}

This is my server side code

package main

import (
    "errors"
    "fmt"
    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promhttp"
    "log"
    "net/http"
    "os"
    "sync"
)

// log handling
func openlogfile(path string) (*os.file, error) {
    logfile, err := os.openfile(path, os.o_wronly|os.o_append|os.o_create, 0644)
    if err != nil {
        return nil, err
    }
    return logfile, nil
}

// variables of counter in metric
var okstatuscounter = prometheus.newcounter(
    prometheus.counteropts{
        name: "ok_request_count",
        help: "number of 200",
    },
)

func listener(serverlog *log.logger) http.handlerfunc {
    return func(w http.responsewriter, r *http.request) {
        //metric 
        okstatuscounter.inc()
        
        w.writeheader(http.statusok)
    }
}

func main() {
    //metric
    prometheus.mustregister(okstatuscounter)

    //log handling
    filesimpleserverlog, err := openlogfile("simpleserver/simpleserverlog.log")
    if err != nil {
        log.fatal(err)
    }
    serverlog := log.new(filesimpleserverlog, "[simple server]", log.lstdflags|log.lshortfile|log.lmicroseconds)

    var wg sync.waitgroup
    wg.add(1)

    //server:
    go func() {
        defer wg.done()
        mux := http.newservemux()
        mux.handlefunc("/home", listener(serverlog))
        mux.handle("/metrics", promhttp.handler())
        server := http.server{
            addr:    fmt.sprintf(":%d", 3333),
            handler: mux,
        }
        if err := server.listenandserve(); err != nil {
            if !errors.is(err, http.errserverclosed) {
                serverlog.printf("error running http server: %s\n", err)
            }
        }
    }()
    wg.wait()
}

At first I thought go might be using one port for all client connections, but when I checked it with netstat, it was using multiple ports. I tried searching but couldn't find a suitable answer

I tried sync.mutex:

var mu sync.Mutex
...
for i := 0; i < 1000; i++ {
        go func() {
            defer wg.Done()
            client := http.Client{
                //Timeout: 30 * time.Second,
            }
            for {
                mu.Lock()
                _, err := client.Do(req)
                if err != nil {
                    clientLog.Printf("client: error making http request: %s\n", err)
                    os.Exit(1)
                }
                mu.Unlock()
            }
        }()
    }
    wg.Wait()
...

With the above changes, I got 13k rps and my cpu load was normal, but it was simply not enough

Workaround

Since you are only sending requests to one host, http The default value of the transport is not suitable for you. It is best to set the parameters manually according to your situation:

t := http.DefaultTransport.(*http.Transport).Clone()
t.MaxIdleConns = 100
t.MaxConnsPerHost = 100
t.MaxIdleConnsPerHost = 100
    
httpClient = &http.Client{
  Timeout:   10 * time.Second,
  Transport: t,
}

For more information you can read here.

The above is the detailed content of Multithreaded requests in go and not getting high RPS. For more information, please follow other related articles on the PHP Chinese website!