How to implement a high-concurrency network programming framework in Go language

In realizing high-concurrency network programming, the Workerman framework of PHP language has always been known for its excellent performance and simplicity and ease of use. However, compared to the PHP language, Golang is more suitable for high concurrency and distributed system development, so implementing the Golang version of the workerman framework has become the pursuit of many developers. In this article, we will introduce how to use the Golang language to implement a high-concurrency network programming framework, similar to Workerman.

1. Prerequisite knowledge

Before we start, we need to master some basic knowledge:

1.Golang language basics: variables, functions, structures, interfaces, etc. concept.

2. Network programming basics: basic knowledge of TCP/UDP, HTTP and other protocols.

3.Goroutine: The coroutine of Golang language can greatly improve the efficiency of concurrent programming.

4.Channel: A communication mechanism provided by the Golang language, which can be used for data transmission and synchronization between different coroutines.

5.Select: A multiplexing mechanism provided by the Golang language, which can monitor the status of multiple Channels and improve program efficiency.

2. Framework architecture

According to the implementation of the Workerman framework, we can divide it into three parts:

1. Receive the connection and generate the client.

2. Business process used to process client requests.

3. Monitor the client connection status and recycle it.

In the Golang language, we can use goroutine to implement the above three parts respectively.

1. Receive the connection and generate the client

We can use the "net" package that comes with the Golang language to create a TCP server, and at the same time open a goroutine to monitor the client connection status.

import (
  "fmt"
  "net"
)

func main() {
  listener, err := net.Listen("tcp", "127.0.0.1:8080")
  if err != nil {
    fmt.Println("failed to listen:", err)
    return
  }

  go func() {
    for {
      conn, err := listener.Accept()
      if err != nil {
        fmt.Println("failed to accept:", err)
        continue
      }
      // 生成客户端
    }
  }()

  // 等待进程退出
  select {}
}

After receiving the client connection, we need to encapsulate a Client object to handle all requests and responses for the connection.

type Client struct {
  Conn   net.Conn
  RespCh chan []byte
}

func NewClient(conn net.Conn) *Client {
  return &Client {
    Conn:   conn,
    RespCh: make(chan []byte, 10),
  }
}

2. Business process used to process client requests

The client's requests and responses are delivered directly through the Channel. When a new connection is received, we need to encapsulate it into a Client object and open a goroutine to handle the connection. This goroutine will listen to all requests sent by the client through the Channel and respond accordingly.

We encapsulate the business process into a Handler interface.

type Handler interface {
  OnConnect(*Client) error
  OnMessage(*Client, []byte) error
  OnClose(*Client) error
}

The client's request and response are passed through the RespCh attribute of the Client object. Therefore, in the Handler interface, we need to define a RespCh property to receive the response from the client.

type Handler interface {
  OnConnect(*Client) error
  OnMessage(*Client, []byte) error
  OnClose(*Client) error
  RespCh() chan []byte
}

We can create an EchoHandler to implement the Handler interface.

type EchoHandler struct {
  clients   []*Client
  respChan  chan []byte
}

func NewEchoHandler() *EchoHandler {
  return &EchoHandler{
    clients:  make([]*Client, 0),
    respChan: make(chan []byte, 10),
  }
}

func (h *EchoHandler) OnConnect(c *Client) error {
  h.clients = append(h.clients, c)
  return nil
}

func (h *EchoHandler) OnMessage(c *Client, data []byte) error {
  // 将客户端发送的数据广播给所有其他客户端，并将其存入respChan中
  for _, client := range h.clients {
    if client == c {
      continue
    }
    client.RespCh <- data
  }
  return nil
}

func (h *EchoHandler) OnClose(c *Client) error {
  for index, client := range h.clients {
    if client == c {
      h.clients = append(h.clients[:index], h.clients[index+1:]...)
    }
  }
  return nil
}

func (h *EchoHandler) RespCh() chan []byte {
  return h.respChan
}

When the Client objects of each connected client are stored in the clients array, we can receive the data sent by each client through the RespCh attribute and broadcast the information sent by the client to other clients. Client functionality.

3. Monitor the client connection status and recycle it

For the old version of the Workerman framework, Workerman will recycle idle connections within a certain period of time. The new version of Workerman implements this function through TCP keepalive.

When implementing the Golang version of workererman, we can also solve the idle connection problem through TCP keepalive. We can monitor the status of its socket in the goroutine of each client. If a client does not send data after 10 seconds of idle time, it will be regarded as an illegal connection and its socket will be closed.

func (c *Client) Process() {
  defer func() {
    c.Conn.Close()
    c.handler.OnClose(c)
  }()
  // 设置 socket keepalive
  tcpConn, ok := c.Conn.(*net.TCPConn)
  if ok {
    tcpConn.SetKeepAlive(true)
    tcpConn.SetKeepAlivePeriod(10 * time.Second)
  }
  // 进入读协程，接收客户端发送的所有数据
  go func() {
    for {
      buf := make([]byte, 1024)
      n, err := c.Conn.Read(buf)
      if err != nil {
        if err != io.EOF {
          fmt.Println("failed to read:", err)
        }
        break
      }
      // 将客户端发送的消息交给Handler处理
      c.handler.OnMessage(c, buf[:n])
    }
  }()
  // 进入写协程，将respChan中的所有响应发送给当前客户端
  go func() {
    for resp := range c.handler.RespCh() {
      _, err := c.Conn.Write(resp)
      if err != nil {
        fmt.Println("failed to write:", err)
        break
      }
    }
  }()
  // OnConnect
  err := c.handler.OnConnect(c)
  if err != nil {
    fmt.Println("failed to on connect:", err)
    return
  }
  // 在Worker进程退出时进行清理
  select {}
}

3. Implement the Worker process

After completing the above three steps, we need to create a Worker process to manage all client connections. One or more Handlers need to be loaded in the Worker process to handle all data requests sent by clients.

type Worker struct {
  listener  net.Listener
  handlers  map[string]Handler
}

func NewWorker(addr string) (*Worker, error) {
  listener, err := net.Listen("tcp", addr)
  if err != nil {
    fmt.Println("failed to listen:", err)
    return nil, err
  }
  return &Worker{
    listener: listener,
    handlers: make(map[string]Handler),
  }, nil
}

func (w *Worker) Register(name string, handler Handler) {
  w.handlers[name] = handler
}

func (w *Worker) Start() {
  go func() {
    for {
      conn, err := w.listener.Accept()
      if err != nil {
        fmt.Println("failed to accept:", err)
        continue
      }
      // 封装连接客户端为Client对象，用于后续的处理
      client := NewClient(conn)
      client.handler = w.handlers["Echo"]
      // 开启客户端goroutine来处理该连接
      go client.Process()
    }
  }()
  // 等待进程退出
  select {}
}

In the Worker process, we need to define a handlers attribute to store instances of different Handlers, and monitor client connections in the Start() function and start new goroutines to handle client requests.

4. Test

We can use the following code to create a Worker process and register an EchoHandler in it to handle all client requests.

func main() {
  server, _ := NewWorker("127.0.0.1:8080")
  handler := NewEchoHandler()
  server.Register("Echo", handler)
  server.Start()
}

We can use the telnet tool to simulate multiple clients sending messages to the server and view their reception.

We use the following command to connect to the server:

telnet 127.0.0.1 8080

We can enter the following text in telnet:

Hello workerman!

We can open multiple telnet windows at the same time to simulate multiple Client requests in parallel.

On the server, we can see the output:

$ go run worker.go 
服务器已启动...
failed to read: read tcp 127.0.0.1:8080->127.0.0.1:56182: use of closed network connection

This is because when we close the client connection, we cancel its listening operation, resulting in a read error.

After the telnet input is completed, we can see that each telnet window will receive the text returned by the server.

5. Summary

In this article, we introduced how to use the Golang language to implement a highly concurrent network programming framework, similar to the workerman in the PHP language. During the implementation process, we used the coroutine, communication mechanism and multiplexing mechanism in the Golang language, and successfully implemented a high-concurrency network programming framework similar to Workerman by encapsulating the Client object and Handler interface.

In fact, in daily programming, we recommend directly using the net/http package provided by the Golang language to implement high-concurrency network programming, which is more concise and has better performance than the workerman framework. We only need to open an http server and use goroutine to process each request concurrently to easily implement high-concurrency network programming.

The above is the detailed content of How to implement a high-concurrency network programming framework in Go language. For more information, please follow other related articles on the PHP Chinese website!