How to build a cluster in golang

1. Foreword

With the development of the Internet, the concurrent performance of applications has become a topic of increasing concern. As a highly concurrent programming language, golang is becoming more and more popular among developers. It comes with GC mechanism, coroutine and channel support, which greatly reduces the complexity and development difficulty of the program.

This article will introduce how to use golang to build a simple cluster to better allocate concurrent requests and improve the performance and reliability of the program.

2. The principle of building a cluster

Before introducing the specific operations, let’s first understand the principle of building a cluster. Generally speaking, a cluster can be simply understood as multiple servers running the same or different applications. Multiple servers communicate through the network to complete functions such as load balancing and data sharing.

In golang, there is a package called net/http, which can easily build an http server. In addition to the http server, we also need to support server service discovery, load balancing and other functions in the cluster. At this time, you can use third-party components like zookeeper to implement it.

In this article, we will use etcd as the service registration center in the cluster to complete the functions of load balancing and service discovery.

3. Environment preparation

Before starting the configuration, we need to install the corresponding tools and environment.

1. golang environment

Download and configure the golang environment on the official website. You can download the corresponding installation package from "https://golang.org/dl/" .

2. etcd

etcd is a distributed key-value storage system open sourced by coreos company, which can easily realize functions such as load balancing and service registration. The corresponding version can be downloaded from "https://github.com/etcd-io/etcd/releases".

4. Specific operations

1. Write http service program

First, we need to write an http service program to handle client requests. Here we can use the net/http package built into the golang system. The most basic operation is the ListenAndServe function, which is used to start the http service.

Next, we write a program to listen to a local http request and return a sentence to the client.

The code is as follows:

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprint(w, "Hello World")
    })

    http.ListenAndServe(":8080", nil)
}

2. Configuring etcd

In etcd, we need to first create a key-value pair to register the service. In this example, we use /services/httpServer as the service path and the localhost IP address and port number 8080 as the node value.

Execute the following command under etcd client to complete the registration:

curl -L http://127.0.0.1:2379/v2/keys/services/httpServer -XPUT -d value='{"host":"localhost", "port":"8080"}'

Configure multiple service nodes in etcd to achieve load balancing and high availability.

3. Writing etcd secure access module

In etcd cluster, we need to achieve more secure service access and load balancing. Here we will use the etcd_sdk package, which can be conveniently used to connect to the etcd registration center and read service nodes.

It is recommended that when writing a service program, read etcd registration information and constantly monitor registration changes to maintain synchronization with the cluster registration center.

The code is as follows:

package main

import (
    "context"
    "fmt"
    "github.com/coreos/etcd/clientv3"
    "net/http"
    "strings"
    "sync"
    "time"
)

var (
    endpoints []string
    currentConfig clientv3.Config
    etcdConfigLocker sync.Mutex
)

func getConfig()(clientv3.Config, error) {
    etcdConfigLocker.Lock()
    defer etcdConfigLocker.Unlock()

    if endpoints == nil {
        return clientv3.Config{}, fmt.Errorf("no endpoints available")
    }

    return clientv3.Config{
        Endpoints: endpoints,
        DialTimeout: 5 * time.Second,
    }, nil
}

type ServiceInfo struct {
    Key    string `json:"key"`
    Value  string `json:"value"`
}

func main() {
    endpoints = append(endpoints, "127.0.0.1:2379")
    readServices()

    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        url := fmt.Sprintf("http://%s%s", getService(), r.URL.Path)
        fmt.Printf("Forward %s => %s\n", r.URL.Path, url)
        http.Redirect(w, r, url, 307)
    })
    err := http.ListenAndServe(":8080", nil)
    if err != nil {
        panic(err)
    }
    readServices()
}

func getService() string {
    config, err := getConfig()
    if err != nil {
        panic(err)
    }

    client, err := clientv3.New(config)
    if err != nil {
        panic(err)
    }

    defer client.Close()

    prefix := "services/httpServer"

    ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
    resp, err := client.Get(ctx, prefix, clientv3.WithPrefix())
    cancel()
    if err != nil {
        panic(err)
    }

    services := make([]*ServiceInfo, 0)
    for _, kv := range resp.Kvs {
        services = append(services, &ServiceInfo{
            Key: string(kv.Key),
            Value: string(kv.Value),
        })
    }

    if len(services) == 0 {
        panic(fmt.Errorf("no endpoint available"))
    }

    return strings.Replace(services[0].Value, "\"", "", -1)
}

func readServices() {
    go func() {
        for {
            getConfigFromEtcd()
            time.Sleep(5 * time.Second)
        }
    }()
}

func getConfigFromEtcd() {
    client, err := clientv3.New(currentConfig)
    if err != nil {
        fmt.Printf("ERROR: create etcd client failed: %s\n", err.Error())
        return
    }

    defer client.Close()

    key := "services/httpServer"
    ctx, cancel := context.WithTimeout(context.Background(), 10 * time.Second)
    resp, err := client.Get(ctx, key, clientv3.WithPrefix())
    cancel()

    if err != nil {
        fmt.Printf("ERROR: get etcd key(%s) failed: %s\n", key, err.Error())
        return
    }

    tempEndpoints := make([]string, 0, len(resp.Kvs))
    for _, itm := range resp.Kvs {
        value := string(itm.Value)
        tempEndpoints = append(tempEndpoints, value)
    }

    fmt.Printf("INFO: get endpoints from etcd(%s) success: %v\n", currentConfig.Endpoints, tempEndpoints)

    currentConfig = clientv3.Config{
        Endpoints: tempEndpoints,
        DialTimeout: 5 * time.Second,
    }
}

In the code, we use the clientv3 package in etcd sdk to connect to etcd registration center and obtain service node information from it. The getConfig() and getConfigFromEtcd() functions are used to read etcd registry information.

4. Run the service program

After configuring the above steps, we can run the program. Open the terminal, switch to the project directory, and execute the following command:

go run main.go

After successful operation, open the browser and visit http://127.0.0.1:8080, you will see the program print out "Hello World" , indicating that the program has been successfully run.

In this example, we use the http service. In actual projects, we can also use high-performance protocols like grpc to improve program performance.

5. Summary

In this article, we introduced the principles and specific operations of using golang to build a cluster. By using etcd registration center and corresponding sdk package, we realize the registration, reading and dynamic maintenance of service nodes. This helps improve the performance and reliability of the program and leads to a better user experience.

In practical applications, we also need to pay attention to the security and fault tolerance of the program to ensure the reliability of the program.

The above is the detailed content of How to build a cluster in golang. For more information, please follow other related articles on the PHP Chinese website!