What is the best practice in Golang for tracking the completion status of two Goroutines within a third Goroutine?

What is the best practice in Golang for tracking the completion status of two Goroutines in a third Goroutine? In Golang, to track the completion status of two Goroutines and process their results in a third Goroutine, the best practice is to use WaitGroup from the sync package. WaitGroup allows us to wait in the main Goroutine for the completion of other Goroutines. First, we need to create a WaitGroup object and call the Add method in the main Goroutine to set the number of waiting Goroutines. Then, the Done method is called at the end of each Goroutine to signal the completion of that Goroutine. Finally, the Wait method is called in the third Goroutine to wait for all Goroutines to complete. This way, we can safely track and process the results of both Goroutines. This is the best practice in Golang for tracking the completion status of multiple Goroutines.

Question content

I have three goroutines running concurrently. Two of them do some processing and send their results to the result channel. The third goroutine "counts" the results by reading the result channels. I could use a waitgroup to wait for the two computation goroutines to complete and then iterate over the result channels to tally the results, but this doesn't scale and requires me to create a buffered result channel with a huge buffer size, which is unacceptable in production code.

I want to count the results as processing occurs, but I don't want to exit the program before all statistics are completed. What are the best practices for achieving this in Go?

This is my current method and it works great. I'm wondering if there's a better way as this seems a bit clunky?

package main

import (
    "fmt"
    "sync"
)

type T struct{}

func main() {
    var widgetInventory int = 1000
    transactions := make(chan int, 100)
    salesDone := make(chan T)
    purchasesDone := make(chan T)
    var wg sync.WaitGroup
    fmt.Println("Starting inventory count = ", widgetInventory)

    go makeSales(transactions, salesDone)
    go newPurchases(transactions, purchasesDone)

    wg.Add(1)

    go func() {
        salesAreDone := false
        purchasesAreDone := false

        for {
            select {
            case transaction := <-transactions:
                widgetInventory += transaction
            case <-salesDone:
                salesAreDone = true
            case <-purchasesDone:
                purchasesAreDone = true
            default:
                if salesAreDone && purchasesAreDone {
                    wg.Done()
                    return
                }
            }
        }
    }()

    wg.Wait()
    fmt.Println("Ending inventory count = ", widgetInventory)
}

func makeSales(transactions chan int, salesDone chan T) {
    for i := 0; i < 3000; i++ {
        transactions <- -100
    }

    salesDone <- struct{}{}
}

func newPurchases(transactions chan int, purchasesDone chan T) {
    for i := 0; i < 3000; i++ {
        transactions <- 100
    }

    purchasesDone <- struct{}{}
}

Solution

Doesn't fit any reasonable definitionGood. You have a hot for loop here:

for {
            select {
            case transaction := <-transactions:
                widgetInventory += transaction
            case <-salesDone:
                salesAreDone = true
            case <-purchasesDone:
                purchasesAreDone = true
            default:
                if salesAreDone && purchasesAreDone {
                    wg.Done()
                    return
                }
            }
        }

As long as there is no channel to read from, the default case will be executed. This happens a lot because of the way channels work.

This slightly adjusted version of the code illustrates the "heat" of this loop. Exact results will vary and may be quite high.

Default case ran 27305 times

You don't want a default situation when selecting from a channel, unless that default also blocks something in it. Otherwise you'll get thermal cycling like this.

Better approach: Use nilable channels for selection

Typically in a select you want to identify a closed channel and set the channel variable to nil; select will never succeed from nil channel reads content, so this effectively "disables" the selection.

Consider this modified version of the code :

go func(transactions chan int, salesDone <-chan T, purchasesDone <-chan T) {
        defer wg.Done()
        for transactions != nil {
            select {
            case transaction, ok := <-transactions:
                if ok {
                    widgetInventory += transaction
                } else {
                    transactions = nil
                }
            case <-salesDone:
                salesDone = nil
                if purchasesDone == nil {
                    close(transactions)
                }
            case <-purchasesDone:
                purchasesDone = nil
                if salesDone == nil {
                    close(transactions)
                }

            }
        }
    }(transactions, salesDone, purchasesDone)

With these adjustments to the consumer, we no longer have hot loops; we always block until data is read from the channel. Once both salesDone and purchasesDone are "signaled", we close(transactions). Once we exhaust transactions and it is closed, we set transactions to nil. We loop when transactions is not nil, which in this code means all channels are nil.

Subtle but important point: I am passing a channel to this function so its reference does not share scope with main. Otherwise, setting transactions to nil will write to a variable shared between goroutines. In this case however, it doesn't matter anyway because we "know" we are the last to read from transactions.

Simpler option: multiple wait groups

If you think about what you're doing here, you need to wait until both producers have finished producing transactions. Then you want to drain transactions. Once the channel is closed and drained, main knows the summation is complete.

You don't need select to perform this operation. And select having a case for each "worker" is arguably rather inelegant; you have to hardcode multiple workers and handle the "completion" channel individually.

What you need to do is:

• 除了为生产者使用一个 var resultswgsync.WaitGroup 之外，还为消费者添加一个。
• 生产者 defer wg.Done()
• 消费者 defer resultswg.Done() 在遍历 transactions 之前：

  go func() {
      defer resultswg.Done()
      for transaction := range transactions {
          widgetInventory += transaction
      }
  }()

• main 处理等待生产者、关闭事务以结束范围，然后等待消费者：

  wg.Wait()
  close(transactions)
  resultswg.Wait()

以这种方式编码，最终会变得简短而甜蜜：

package main

import (
    "fmt"
    "sync"
)

func main() {
    var widgetInventory int = 1000
    transactions := make(chan int, 100)

    var wg, resultswg sync.WaitGroup
    fmt.Println("Starting inventory count = ", widgetInventory)
    wg.Add(2)

    go makeSales(transactions, &wg)
    go newPurchases(transactions, &wg)
    resultswg.Add(1)
    go func() {
        defer resultswg.Done()
        for transaction := range transactions {
            widgetInventory += transaction
        }
    }()

    wg.Wait()
    close(transactions)
    resultswg.Wait()
    fmt.Println("Ending inventory count = ", widgetInventory)
}

func makeSales(transactions chan int, wg *sync.WaitGroup) {
    defer wg.Done()
    for i := 0; i < 3000; i++ {
        transactions <- -100
    }

}

func newPurchases(transactions chan int, wg *sync.WaitGroup) {
    defer wg.Done()
    for i := 0; i < 3000; i++ {
        transactions <- 100
    }

}

您可以在这里看到，在此模式中可以有任意数量的生产者；您只需为每个生产者添加 wg.Add(1) 即可。

当我不知道每个工作人员会返回多少结果时，我一直使用这种模式来并行化工作。我发现它很容易理解，并且比尝试 select 多个通道简单得多。事实上，我什至想说，如果您发现自己从多个渠道进行 selecting，您应该退后一步，确保它对您来说确实有意义。我使用 select 的频率远远低于使用等待组的频率。

The above is the detailed content of What is the best practice in Golang for tracking the completion status of two Goroutines within a third Goroutine?. For more information, please follow other related articles on the PHP Chinese website!