How Can I Efficiently Read a Large Text File in .NET?

Reading Large Text Files Efficiently in .NET

When handling massive text files, reading them line by line performance becomes critical. This article discusses the optimal approach to read a 1 GB text file in .NET.

Using StreamReader

A widely used method to read text files in .NET is StreamReader.ReadLine. However, it's not the most optimal approach for large files as it reads the entire file into memory, which can be resource-intensive.

MemoryMappedFiles for Large Files

In .NET 4.0 and later, consider using MemoryMappedFile. This class is specifically designed for efficiently handling large files. It allows you to create a memory-mapped file, which makes a portion of the file accessible from memory. This avoids the overhead of reading the entire file into memory, significantly improving performance.

Code Sample

The following code demonstrates how to read a large text file using MemoryMappedFile:

using System.IO.MemoryMappedFiles;

// Create a memory-mapped file object
MemoryMappedFile mmf = MemoryMappedFile.CreateFromFile(filePath, FileMode.Open, "MyMappedFile");

// Create a memory-mapped view to the file contents
MemoryMappedViewAccessor view = mmf.CreateViewAccessor();

// Read the file line by line using a byte array
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = view.Read(buffer, 0, buffer.Length)) != 0)
{
    // Convert the buffer to a string
    string line = System.Text.Encoding.UTF8.GetString(buffer, 0, bytesRead);

    // Process the line as needed
}

Conclusion

While StreamReader.ReadLine can suffice for smaller text files, MemoryMappedFiles offer significantly better performance for large files. By avoiding the overhead of loading the entire file into memory, it enables faster and more efficient processing.

The above is the detailed content of How Can I Efficiently Read a Large Text File in .NET?. For more information, please follow other related articles on the PHP Chinese website!