


Detailed explanation on using Lazy Collections to improve the performance of Laravel Excel reading (easily supports millions of data)
A new type of collection has been added in Laravel 6: Lazy Collections. They are great if you need to process very large data sets (thousands or millions of rows) without running into memory constraints.
Recommended: laravel tutorial
My most recent task was to refactor Excel export in a project at work. The problem is that the export can no longer be created because the dataset is too large for Laravel to handle. The database query returned approximately 300,000 results! The application times out or consistently runs out of memory.
A naive approach is to increase the timeout or memory limit and hope that the next time something goes wrong, someone else will handle it. But that's not how I work. I don't like Band-Aids. I like concrete, long-term solutions.
Laravel Excel extension pack is already quite flexible. It does a great job of reducing database load by using "chunks" when using FromQuery-concerns. However, our export still struggles with large data sets.
My colleagues and I discussed whether we should completely rewrite this feature: push the export to a queue and send a notification to the user when the export is complete. However, this feature is just a minor thing in this app. It doesn't make sense to us to add so much overhead just for a simple export.
Later that day, I had a little "Eureka" moment because I remembered that Laravel had LazyCollections.
I rewrote the export: it now uses FromCollection-concern instead of FromQuery. The only change I had to make to the collection() method was to replace the get() method at the end of the query builder chain with a cursor().
Below is a simplified version of our export functionality. The Request object is passed through the constructor so we can adapt the query based on what the user selects in the UI.
<?php namespace App\Exports; use App\User; use Maatwebsite\Excel\Concerns\Exportable; use Maatwebsite\Excel\Concerns\FromCollection; use Illuminate\Http\Request; class UsersExport implements FromCollection { use Exportable; protected Request $request; public function __construct(Request $request) { $this->request = $request; } public function collection() { return User::query() ->when($this->request->get('include_subscribed'), function ($q) { return $q->where('is_subscribed', true); }) ->cursor(); // ← 重要的一点 } }
I believe you are experiencing memory issues in your project. You increased the memory limit and hopefully that solved the problem (I've done this countless times myself).
If it were in a Laravel project, I hope, I could have you revisit that code and rewrite it using LazyCollections.
Fixing this issue was very interesting, so I did a little benchmark: our export can now easily export millions of rows without hitting memory limits. so cool!
The above is the detailed content of Detailed explanation on using Lazy Collections to improve the performance of Laravel Excel reading (easily supports millions of data). For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics



Method for obtaining the return code when Laravel email sending fails. When using Laravel to develop applications, you often encounter situations where you need to send verification codes. And in reality...

Laravel schedule task run unresponsive troubleshooting When using Laravel's schedule task scheduling, many developers will encounter this problem: schedule:run...

The method of handling Laravel's email failure to send verification code is to use Laravel...

How to implement the table function of custom click to add data in dcatadmin (laravel-admin) When using dcat...

Laravel - Dump Server - Laravel dump server comes with the version of Laravel 5.7. The previous versions do not include any dump server. Dump server will be a development dependency in laravel/laravel composer file.

The impact of sharing of Redis connections in Laravel framework and select methods When using Laravel framework and Redis, developers may encounter a problem: through configuration...

Custom tenant database connection in Laravel multi-tenant extension package stancl/tenancy When building multi-tenant applications using Laravel multi-tenant extension package stancl/tenancy,...

Laravel - Action URL - Laravel 5.7 introduces a new feature called “callable action URL”. This feature is similar to the one in Laravel 5.6 which accepts string in action method. The main purpose of the new syntax introduced Laravel 5.7 is to directl
