Is there a concept of reading the output of a program as a stream line by line?

Question content

I have a shell command (e.g. journalctl -f -o json) that continuously streams the lines Transfer to standard output.

I want to line by line retrieve this output and process it further.

The documentation for

os/exec addresses how to read the output of a command and io handle stream buffering.

It seems to me that processing all goes through a fixed buffer which is read in, processed and further written to. My problem is that the size of this buffer is fixed and independent of the content.

Is there a way to read the incoming stream (in my case the output of the shell command) line by line? Maybe there is a more advanced library than io reader?

Correct answer

Use cmd.stdoutpipe() Get (pipe) using cmd.start() Output before starting the process ( start() starts the command but does not wait for it to complete).

And use bufio.scanner to read the input (io.reader) line-by-line.

For example, I would use this bash script to print the current time 3 times and sleep 1 second between each time:

for i in {1..3}; do date; sleep 1; done

Example of executing this command and reading its output line by line:

cmd := exec.command("bash", "-c", "for i in {1..3}; do date; sleep 1; done")
out, err := cmd.stdoutpipe()
if err != nil {
    log.fatal(err)
}
defer out.close()

err = cmd.start()
if err != nil {
    log.fatal(err)
}

scanner := bufio.newscanner(out)
for scanner.scan() {
    line := scanner.text()
    fmt.println("output:", line)
}

Example output:

2022/11/29 14:38:48 Output: Tue Nov 29 02:38:48 PM CET 2022
2022/11/29 14:38:49 Output: Tue Nov 29 02:38:49 PM CET 2022
2022/11/29 14:38:50 Output: Tue Nov 29 02:38:50 PM CET 2022

(The first datetime at the beginning of each line comes from the log package to verify that each line is printed after the second delay, the other timestamp is the output of the date command .)

The above is the detailed content of Is there a concept of reading the output of a program as a stream line by line?. For more information, please follow other related articles on the PHP Chinese website!