How Can I Optimize Entity Framework Inserts for Maximum Performance When Dealing with Large Datasets?

Boosting Entity Framework Insert Performance with Large Datasets

Inserting large datasets into Entity Framework (EF) can be slow. This article explores techniques to significantly speed up this process, especially when using TransactionScopes and handling a substantial number of records.

Optimizing SaveChanges() Calls

The common practice of calling SaveChanges() for each record is inefficient for bulk inserts. Instead, consider these alternatives:

• Batch SaveChanges(): Call SaveChanges() only once after all records are added to the context.
• Interval SaveChanges(): Call SaveChanges() after a set number of records (e.g., 100 or 1000).
• Context Recycling: Call SaveChanges(), dispose the context, and create a new one after a certain number of records.

These methods reduce database round trips and minimize overhead from change tracking.

Efficient Bulk Insert Implementation

The following code demonstrates optimized bulk insertion:

using (TransactionScope scope = new TransactionScope())
{
    MyDbContext context = null;
    try
    {
        context = new MyDbContext();
        context.Configuration.AutoDetectChangesEnabled = false;

        int count = 0;
        foreach (var entityToInsert in someCollectionOfEntitiesToInsert)
        {
            ++count;
            context = AddToContext(context, entityToInsert, count, 1000, true);
        }

        context.SaveChanges();
    }
    finally
    {
        if (context != null)
            context.Dispose();
    }

    scope.Complete();
}

private MyDbContext AddToContext(MyDbContext context, Entity entity, int count, int commitCount, bool recreateContext)
{
    context.Set<Entity>().Add(entity);

    if (count % commitCount == 0)
    {
        context.SaveChanges();
        if (recreateContext)
        {
            context.Dispose();
            context = new MyDbContext();
            context.Configuration.AutoDetectChangesEnabled = false;
        }
    }

    return context;
}

This example disables automatic change detection (AutoDetectChangesEnabled = false) for performance gains. The AddToContext method handles committing changes and optionally recreating the context after a specified number of records.

Benchmark Results

Testing with 560,000 records showed dramatic improvements:

• Per-record SaveChanges() : Hours
• SaveChanges() after 100 records: Over 20 minutes
• SaveChanges() after 1000 records (no context disposal): 242 seconds
• SaveChanges() after 1000 records (with context disposal): 191 seconds

Conclusion

These strategies dramatically improve Entity Framework bulk insert performance. Disabling change tracking, using batch SaveChanges(), and managing context effectively are key to efficiently handling large datasets.

The above is the detailed content of How Can I Optimize Entity Framework Inserts for Maximum Performance When Dealing with Large Datasets?. For more information, please follow other related articles on the PHP Chinese website!