How Can I Efficiently Read Text Files Line by Line in C#?

C#efficiently read text files

In the .NET C#environment, reading text files one by one is a common task. This article will explore several different methods and analyze its efficiency.

Use streamReader.readline

Your current method uses and the buffer size is set to 128. However, the benchmark test shows that increasing the size of the buffer to 1024 (default) or 4096 (NTFS cluster) can significantly improve performance. The following code fragment demonstrates this:

StreamReader.ReadLine Use File.readlines

const Int32 BufferSize = 128;
using (var fileStream = File.OpenRead(fileName))
using (var streamReader = new StreamReader(fileStream, Encoding.UTF8, true, BufferSize)) {
    String line;
    while ((line = streamReader.ReadLine()) != null)
    {
      // 处理每一行
    }
}

This method uses internally, compared with the smaller in the buffer area, the performance is improved. It uses iterators and reduces the memory consumption.

StreamReader Use File.readalllines StreamReader.ReadLine

var lines = File.ReadLines(fileName);
foreach (var line in lines)
  // 处理每一行

is similar to <似>, but returns a string array instead of

. This method requires more memory, but allows random access.

Use string.split

File.ReadLines IEnumerable<string> Although <方> is very convenient, when reading large files, its speed is significantly slower, as shown in the following example:

var lines = File.ReadAllLines(fileName);
for (var i = 0; i < lines.Length; i++)
  // 处理每一行

Conclusion

In short, for reading text files one by one,

is recommended for its simplicity and efficiency. If you need to customize the sharing option, consider using the String.Split with an appropriate buffer size. However, the benchmark test must be performed to determine the solution that is most suitable for your specific needs.

The above is the detailed content of How Can I Efficiently Read Text Files Line by Line in C#?. For more information, please follow other related articles on the PHP Chinese website!