How Can I Efficiently Read a File Backwards in Python?

How to Efficiently Read a File in Reverse Order Using Python

To read a file in reverse order, where you start from the last line and move towards the beginning, Python provides a robust solution: a generator function that iterates through the lines in reverse order.

Efficient Generator Solution:

The following generator function, reverse_readline, achieves this task with high efficiency:

import os

def reverse_readline(filename, buf_size=8192):
    """A generator that returns the lines of a file in reverse order"""
    with open(filename, 'rb') as fh:
        segment = None
        offset = 0
        fh.seek(0, os.SEEK_END)
        file_size = remaining_size = fh.tell()
        while remaining_size > 0:
            offset = min(file_size, offset + buf_size)
            fh.seek(file_size - offset)
            buffer = fh.read(min(remaining_size, buf_size))
            # remove file's last "\n" if it exists, only for the first buffer
            if remaining_size == file_size and buffer[-1] == ord('\n'):
                buffer = buffer[:-1]
            remaining_size -= buf_size
            lines = buffer.split('\n'.encode())
            # append last chunk's segment to this chunk's last line
            if segment is not None:
                lines[-1] += segment
            segment = lines[0]
            lines = lines[1:]
            # yield lines in this chunk except the segment
            for line in reversed(lines):
                # only decode on a parsed line, to avoid utf-8 decode error
                yield line.decode()
        # Don't yield None if the file was empty
        if segment is not None:
            yield segment.decode()

This function maintains an offset and reads the file in reverse chunks, starting from the end. It yields the lines in reverse order while handling the special case of the last line that may have a trailing newline.

To use this generator, simply open the file in binary mode and iterate through the lines in reverse order:

for line in reverse_readline("my_file.txt"):
    # Process line in reverse order

The above is the detailed content of How Can I Efficiently Read a File Backwards in Python?. For more information, please follow other related articles on the PHP Chinese website!