The Go program is too big. Can we use lazy initialization?

#In the continuous development of the company, most of them were large single entities at the beginning, and the transformation was slow. A warehouse will be used for more than ten years. Scale is basically a process of increasing.

One of the impacts is that the size of the packaged application is getting larger and larger, and I don’t know where it is used... The proposal to be discussed today "proposal: language: lazy init imports to possibly import without side effects^[1]》, is related to this.

Proposal

Background

Let’s observe a very simple Go code and study it . The following code:

package main

import _ "crypto/x509"

func main() {}

This Go program has only 3 lines of code and seems to have nothing. Is this actually the case?

We can execute the following command to see the initialization process:

$ go build --ldflags=--dumpdep main.go 2>&1 | grep inittask

Output result:

runtime.main -> runtime..inittask
runtime.main -> main..inittask
main..inittask -> crypto/x509..inittask
crypto/x509..inittask -> bytes..inittask
crypto/x509..inittask -> crypto/sha256..inittask
crypto/x509..inittask -> encoding/pem..inittask
crypto/x509..inittask -> errors..inittask
crypto/x509..inittask -> sync..inittask
crypto/x509..inittask -> crypto/aes..inittask
crypto/x509..inittask -> crypto/cipher..inittask
crypto/x509..inittask -> crypto/des..inittask
...
context..inittask -> context.init.0
vendor/golang.org/x/net/dns/dnsmessage..inittask -> vendor/golang.org/x/net/dns/dnsmessage.init
vendor/golang.org/x/net/route..inittask -> vendor/golang.org/x/net/route.init
vendor/golang.org/x/net/route..inittask -> vendor/golang.org/x/net/route.init.0
...

This program actually initializes a lot of software packages (standard libraries, third-party package, etc.). This changes the package size from the standard 1.3 MB to 2.3 MB.

At a certain scale, the impact is considered to be very expensive. Because you can see that the 3-line Go program doesn't do anything substantial.

Programs that are sensitive to startup performance will be more uncomfortable. Ordinary programs will also enter a vicious cycle over time, and startup will be slower than normal.

Proposal

In terms of solution, we combined it with another proposal "proposal: spec: Go 2: allow manual control over imported package initialization^[2]》Let’s take a look together.

The core idea is to introduce lazy initialization (lazy init), which is also often called lazy loading in the industry. That is to say, the actual import is done when necessary, and the initialization is completed when the package is not introduced.

Optimization direction: mainly adding lazy initialization statements after importing the package path, such as the go:lazyinit or go:deferred annotations mentioned below. Wait until the program is actually used before formally initializing it.

1、go:lazyinit 的例子：

package main

import (
      "crypto/x509" // go:lazyinit
      "fmt"
)

func main() {...}

2、go:deferred 的例子：

package main

import (
    _ "github.com/eddycjy/core" // go:deferred
    _ "github.com/eddycjy/util" // go:deferred
)

func main() {
    if os.Args[1] != "util" {
        // 现在要使用这个包，开始初始化
        core, err := runtime.InitDeferredImport("github.com/some/module/core")
        ...
    }
    ...
}

以此来实现，可以大大提高启动性能。

讨论

实际上在大多数的社区讨论中，对这个提案是又爱又恨。因为它似乎又有合理的诉求，但细思似乎又会发现完全不对劲。

这个提案的背景和解决方案，是治标不治本的。因为根本原因是：许多库滥用了 init 函数，让许多不必要的东西都初始化了。

Go 核心开发团队认为让库作者去修复这些库，而不是让 Go 来 “解决” 这些问题。如果支持惰性初始化，也会为这些低质量库的作者提供继续这样做的借口。

似曾相识的感觉

在写这篇文章时，我想起了 Go 的依赖管理（Go modules），其有一个设计是基于语义化版本的规范。

如下图

版本格式为 “主版本号.次版本号.修订号”，版本号的递增规则如下：

• 主版本号：当你做了不兼容的 API 修改。
• 次版本号：当你做了向下兼容的功能性新增。
• 修订号：当你做了向下兼容的问题修正。

Go modules 的原意是软件库都遵守这个规范，因此内部会有最小版本选择的逻辑。

也就是一个模块往往依赖着许多其它许许多多的模块，并且不同的模块在依赖时很有可能会出现依赖同一个模块的不同版本，Go 会把版本清单都整理出来，最终得到一个构建清单。

如下图：

You will find that the final dependency version built is likely to be inconsistent with the expected, leading to many business problems. The most classic one is the multi-version compatibility problem of grpc-go, protoc-go, etcd, which makes many people suffer.

The design of the Go team in this area is relatively ideal, and Cao Da also classified it as one of the seven deadly sins of Go modules. The init function of the software package has a lot of problems with random initialization, which is also a bit familiar.

Summary

The solution (proposal) to this problem is still under discussion. Obviously the Go team hopes that the authors of the software library can constrain their own code and not mess around. initialization.

How about introducing lazy initialization, what do you think? Welcome to leave messages and discuss in the comment area.

The above is the detailed content of The Go program is too big. Can we use lazy initialization?. For more information, please follow other related articles on the PHP Chinese website!