Data concurrency processing in Golang and Go WaitGroup

Data concurrency processing in Golang and Go WaitGroup

Introduction:
In modern software development, data concurrency processing is a very important technology. When processing large amounts of data, using concurrency techniques can significantly improve program performance and response time. As a concurrency-friendly programming language, Golang provides a variety of ways to implement concurrent data processing, the most commonly used of which is using Go WaitGroup. This article will introduce in detail data concurrency processing in Golang and how to use Go WaitGroup to manage concurrent tasks.

1. Basics of concurrent processing
   In Golang, goroutine is mainly used to implement concurrent processing. Goroutine is a lightweight thread that can execute concurrently with other goroutines. By using goroutines, multiple functions or methods can be executed simultaneously in the same program, taking full advantage of the power of multi-core processors. The following is a simple sample code:

package main

import (
    "fmt"
    "time"
)

func main() {
    go printNumbers()
    go printLetters()
    time.Sleep(2 * time.Second)
}

func printNumbers() {
    for i := 1; i <= 5; i++ {
        fmt.Println(i)
        time.Sleep(500 * time.Millisecond)
    }
}

func printLetters() {
    for i := 'a'; i <= 'e'; i++ {
        fmt.Printf("%c
", i)
        time.Sleep(500 * time.Millisecond)
    }
}

In the above code, we created two goroutines to concurrently execute the printNumbers and printLetters functions. The printNumbers function prints the numbers 1 to 5, and the printLetters function prints the lowercase letters a to e. By using time.Sleep, let the main program wait long enough to ensure that the program exits after the two goroutines are completed.

2. Go WaitGroup usage
   Although waiting for the goroutine to complete through time.Sleep is one way, this method is not reliable and flexible in actual development. Golang provides sync.WaitGroup to better manage the completion status of goroutine. WaitGroup is a counting semaphore used to wait for the completion of a group of goroutines. The following is a sample code using WaitGroup:

package main

import (
    "fmt"
    "sync"
    "time"
)

func main() {
    var wg sync.WaitGroup
    wg.Add(2) // 添加两个任务

    go printNumbers(&wg)
    go printLetters(&wg)

    wg.Wait() // 等待所有任务完成
}

func printNumbers(wg *sync.WaitGroup) {
    defer wg.Done() // 减少计数器

    for i := 1; i <= 5; i++ {
        fmt.Println(i)
        time.Sleep(500 * time.Millisecond)
    }
}

func printLetters(wg *sync.WaitGroup) {
    defer wg.Done() // 减少计数器

    for i := 'a'; i <= 'e'; i++ {
        fmt.Printf("%c
", i)
        time.Sleep(500 * time.Millisecond)
    }
}

In the above code, we first create a WaitGroup object wg, And inform WaitGroup that there are two tasks to wait for through the wg.Add(2) method. Then, we call the wg.Done() method in the printNumbers and printLetters functions respectively to decrement the counter. Finally, by calling the wg.Wait() method, the program will block until all tasks are completed, and then continue to execute the following code.

3. Advanced usage of WaitGroup
   In addition to basic usage, WaitGroup also provides some advanced usage, such as limiting the number of concurrencies, timeout control, etc. The following is a sample code that uses WaitGroup for concurrent task limiting:

package main

import (
    "fmt"
    "sync"
    "time"
)

func main() {
    var (
        wg     sync.WaitGroup
        maxCon = 2 // 最大并发数
        tasks  = 10 // 总任务数
    )

    // 创建一个带有最大并发数限制的通道
    semaphore := make(chan struct{}, maxCon)

    for i := 0; i < tasks; i++ {
        wg.Add(1)
        go process(i, &wg, semaphore)
    }

    wg.Wait()
}

func process(id int, wg *sync.WaitGroup, semaphore chan struct{}) {
    defer wg.Done()

    semaphore <- struct{}{} // 每个任务开始前获取信号量
    defer func() {
        <-semaphore // 每个任务结束时释放信号量
    }()

    fmt.Printf("Task %d start
", id)
    time.Sleep(500 * time.Millisecond)
    fmt.Printf("Task %d finish
", id)
}

In the above code, we first create a semaphore channel with a capacity of is maxCon, which is the maximum number of concurrencies. Then, we create goroutine for tasks tasks through a loop. Before each goroutine starts, a semaphore is obtained from the semaphore channel, indicating that there is still available concurrency. After the task is executed, the occupied semaphore will be released. In this way, we can limit the number of concurrencies and avoid resource exhaustion caused by executing too many goroutines at the same time.

4. Conclusion
   This article introduces how to implement concurrent data processing in Golang and use WaitGroup to manage concurrent tasks. By using goroutine and WaitGroup, we can easily implement concurrent processing, fully utilize the capabilities of multi-core processors, and improve program performance. I hope this article will help you understand data concurrency processing and the use of WaitGroup.

The above is the detailed content of Data concurrency processing in Golang and Go WaitGroup. For more information, please follow other related articles on the PHP Chinese website!