php小编小新带来了一篇关于使用goroutine构建Zip的技巧文章。在这篇文章中,我们将了解如何利用goroutine并发执行Zip操作,以提高程序的效率。此外,我们还将讨论切片超出范围的问题,并介绍如何使用容量为4096的切片来解决这个问题。无论你是初学者还是有经验的开发者,本文都将帮助你更好地理解和应用goroutine和切片相关的知识。让我们一起来探索吧!
我正在尝试通过添加 goroutine 来处理每个必须存档的文件来改进构建 zip 的函数。
但最终却以恐慌告终
panic:运行时错误:切片超出范围[4126:4096
]
目标目录包含 190 个文件(500 mo)。 我实在不明白出了什么问题 预先感谢您的帮助
功能:
func buildarchive() error { var files []string err := filepath.walk("/tmp/dir-to-zip", func(filepath string, info os.fileinfo, err error) error { if info.isdir() { return nil } if err != nil { fmt.println(err) return err } files = append(files, filepath) return nil }) if err != nil { return err } bundle, err := os.create("/tmp/archive.zip") if err != nil { return err } bundlewriter := zip.newwriter(bundle) var wg sync.waitgroup wg.add(len(files)) for _, filepath := range files { go func(filepath string) { defer wg.done() relpath := strings.trimprefix(filepath, fmt.sprintf("%v/", filepath.dir("/tmp/dir-to-zip"))) bundlefile, err := bundlewriter.create(relpath) if err != nil { fmt.println(err) } fsfile, err := os.open(filepath) if err != nil { fmt.println(err) } _, err = io.copy(bundlefile, fsfile) if err != nil { fmt.println(err) } }(filepath) } wg.wait() err = bundlewriter.close() if err != nil { return err } return nil }
错误发生在这里:
_, err = io.copy(bundlefile, fsfile) if err != nil { fmt.println(err) }
堆栈跟踪:
goroutine 48 [running]: bufio.(*Writer).Write(0xc00002a100, {0xc00041a400?, 0x3d?, 0xc00041a400?}) /usr/local/go/src/bufio/bufio.go:670 +0x1c8 archive/zip.(*countWriter).Write(0xc00000c138, {0xc00041a400?, 0x3d?, 0x4afa20?}) /usr/local/go/src/archive/zip/writer.go:601 +0x2e io.WriteString({0x4e7538, 0xc00000c138}, {0xc0000212c9, 0x3d}) /usr/local/go/src/io/io.go:314 +0x91 archive/zip.writeHeader({0x4e7538, 0xc00000c138}, 0xc000220090) /usr/local/go/src/archive/zip/writer.go:422 +0x5ec archive/zip.(*Writer).CreateHeader(0xc0000760a0, 0xc00021e1b0) /usr/local/go/src/archive/zip/writer.go:378 +0x797 archive/zip.(*Writer).Create(0x4e7698?, {0xc0000212c9, 0x3d}) /usr/local/go/src/archive/zip/writer.go:223 +0x6c main.BuildArchive.func2({0xc0000212c0, 0x46}) /home/simba/go/src/foobar/main.go:79 +0x1c5 created by main.BuildArchive /home/simba/go/src/foobar/main.go:73 +0x5aa panic: runtime error: slice bounds out of range [:4126] with capacity 4096 goroutine 6 [running]: bufio.(*Writer).Flush(0xc00002a100) /usr/local/go/src/bufio/bufio.go:634 +0x171 bufio.(*Writer).Write(0xc00002a100, {0xc0001b4200?, 0xc000199b20?, 0xc000199b20?}) /usr/local/go/src/bufio/bufio.go:672 +0xd8 archive/zip.(*countWriter).Write(0xc00000c138, {0xc0001b4200?, 0x0?, 0xc000199b40?}) /usr/local/go/src/archive/zip/writer.go:601 +0x2e archive/zip.(*countWriter).Write(0xc000220018, {0xc0001b4200?, 0xc0001b02c0?, 0xc0001b02f0?}) /usr/local/go/src/archive/zip/writer.go:601 +0x2e compress/flate.(*huffmanBitWriter).write(...) /usr/local/go/src/compress/flate/huffman_bit_writer.go:136 compress/flate.(*huffmanBitWriter).writeCode(0xc0001b41e0?, {0x6000?, 0x22?}) /usr/local/go/src/compress/flate/huffman_bit_writer.go:347 +0xe5 compress/flate.(*huffmanBitWriter).writeTokens(0xc0001b41e0, {0xc002558000, 0x4001, 0x403f800000403f?}, {0xc0001aa900, 0x11e, 0x108129000000000f?}, {0xc0001ac100, 0x1e, 0x1e}) /usr/local/go/src/compress/flate/huffman_bit_writer.go:583 +0xb9 compress/flate.(*huffmanBitWriter).writeBlock(0xc0001b41e0, {0xc002558000?, 0x20?, 0xd79?}, 0x0, {0x0, 0x0, 0x0}) /usr/local/go/src/compress/flate/huffman_bit_writer.go:495 +0x490 compress/flate.(*compressor).writeBlock(0xc0005a2000, {0xc002558000?, 0xc000032f00?, 0xc000199d28?}, 0x47739b?) /usr/local/go/src/compress/flate/deflate.go:170 +0x9c compress/flate.(*compressor).deflate(0xc0005a2000) /usr/local/go/src/compress/flate/deflate.go:509 +0x59b compress/flate.(*compressor).write(0xc0005a2000, {0xc00256a000?, 0x8000, 0xf311b6fd?}) /usr/local/go/src/compress/flate/deflate.go:554 +0x82 compress/flate.(*Writer).Write(...) /usr/local/go/src/compress/flate/deflate.go:712 archive/zip.(*pooledFlateWriter).Write(0xc00020c040?, {0xc00256a000?, 0x8000?, 0x4af140?}) /usr/local/go/src/archive/zip/register.go:51 +0xc5 archive/zip.(*countWriter).Write(...) /usr/local/go/src/archive/zip/writer.go:601 archive/zip.(*fileWriter).Write(0xc000222000, {0xc00256a000, 0x8000, 0x8000}) /usr/local/go/src/archive/zip/writer.go:533 +0x97 io.copyBuffer({0x4e7558, 0xc000222000}, {0x4e7678, 0xc0001f8008}, {0x0, 0x0, 0x0}) /usr/local/go/src/io/io.go:428 +0x204 io.Copy(...) /usr/local/go/src/io/io.go:385 main.BuildArchive.func2({0xc00001c0c0, 0x35}) /home/simba/go/src/foobar/main.go:89 +0x385 created by main.BuildArchive /home/simba/go/src/foobar/main.go:73 +0x5aa exit status 2
zip.Writer
并发使用不安全。您启动多个 goroutine,每个 goroutine 创建并写入 zip 条目(文件)。
Writer.Create()
记录:
创建使用提供的名称将文件添加到 zip 文件中。它返回一个 Writer,文件内容应写入其中。
[...] 在下次调用 Create、CreateHeader 或 Close 之前,必须将文件内容写入 io.Writer。
您不能同时创建 zip。每个 zip 条目的 io.Writer
s 写入相同的底层文件(或一般情况下相同的 io.Writer
),即使您的应用程序没有出现恐慌或崩溃,生成的 zip 存档也可能无效。
以上是使用 goroutine 构建 Zip,切片超出范围,容量为 4096的详细内容。更多信息请关注PHP中文网其他相关文章!