I need to export or update about 1.5 million rows and Model::all() runs out of memory. I see chunk, chunkById, lazy, lazyById and cursor in the docs. Which one should I pick and what are the pitfalls?
DEV3 Developer Forum
chunk(), lazy() or cursor() for processing a million rows in Laravel?
Every post is reviewed first. Usually reviewed within 24 hours.
1 reply
all() loads every model into memory. Pick one of the batching tools.
chunkById (default choice for updates)
User::where('active', false)->chunkById(1000, function ($users) {
foreach ($users as $user) {
$user->update(['archived_at' => now()]);
}
});
chunk() uses offset/limit, so if your callback changes the column used in the where clause, rows get skipped. chunkById() paginates by primary key and is safe for updates and deletes.
lazyById (nice API, constant memory)
foreach (Order::lazyById(2000) as $order) {
fputcsv($out, [$order->id, $order->total]);
}
Returns a LazyCollection that loads chunks behind the scenes, so you can use map, filter and each as if it were one collection.
cursor()
Runs one query and hydrates one model at a time. Lowest memory in PHP but the DB driver may still buffer the full result set (the MySQL PDO driver uses buffered queries by default), and you cannot eager load relationships, so it invites N+1. Prefer lazy/lazyById unless you know why.
Bulk SQL when possible
User::where('active', false)->update(['archived_at' => now()]);
One query beats a million model updates, but skips model events.
Pitfalls
- Eager loading works with
chunkandlazy(->with('items')->lazyById()), not withcursor. - Run big jobs in a queue or console command with
--memorymonitoring; disable query log:DB::disableQueryLog(). - Always order deterministically; chunkById does it for you.