Browse documentationOpen

byrcsc/laravel-data-sync · 1.x

Failures and retries.

Tell a failed run from a run with failed rows, read the failure records, and choose between reprocessing a whole file and replaying only its failed rows.

Two kinds of failure are worth keeping apart.

A failed run means the file did not finish: a schema mismatch, an unreachable disk, a dead worker, maxErrors() exceeded. Nothing is marked processed and the staged file is preserved for retry.

Failed rows mean the file finished and some of its rows did not qualify: a validation failure, a transform that threw, a missing key. The run status is completed_with_errors, the checksum is marked processed, and the file is archived. Each bad row is recorded in sync_row_failures.

Both are visible in sync:status.

Reading failed rows

php artisan sync:status products --failures
01JQ8Z3M0K2N4P6R8T0V2X4Y6A row 412: {"price_cents":["The price cents must be an integer."]} {"SKU":"DS-1041","Price":"n\/a"}

Each record carries the row number as the reader counts it, the errors keyed by attribute, the exception message when one was thrown, and the raw payload — the values as the reader produced them, before mapping — when payload capture is on.

use ByRcsc\LaravelDataSync\Models\SyncRowFailure;

SyncRowFailure::query()
    ->whereHas('run', fn ($q) => $q->where('definition', 'products'))
    ->latest('id')
    ->get()
    ->each(function (SyncRowFailure $failure): void {
        $failure->row_number;
        $failure->errors;      // ['price_cents' => ['The price cents must be an integer.']]
        $failure->exception;   // exception message, when the row threw
        $failure->payload;     // raw values, or null when capture was off
    });

The run's error_summary holds a JSON histogram of the same errors — how many rows failed for each reason — which is usually the faster read.

Retrying a run

php artisan sync:retry 01JQ8Z3M0K2N4P6R8T0V2X4Y6A
php artisan sync:retry 412

Runs are addressed by ULID or numeric id, and must have finished. Retrying restores the preserved copy of the file to staging, verifies its checksum, creates a new run, and dispatches it to the queue. Which copy is restored depends on how the original ended:

Original run statusRestored from
completed, completed_with_errorsThe archive disk
failed, cancelledThe failed disk

This is the path for a run that failed outright, and it works after the source file is gone — which is the point of preserving the copy in the first place.

A retry always creates a new run, so history keeps both attempts.

Retrying reprocesses every row in the file, not just the bad ones. For an Upsert definition that is harmless. For CreateOnly, rows that already exist are skipped by the write mode, which is also harmless — but check the mode before retrying a definition with side effects in afterSave().

When the run record is gone

Runs are pruned; archived files usually outlive them by a shorter window, but either can disappear first. sync:retry falls back to searching the archive and failed disks for a stored file whose name begins with the reference you gave, and reprocesses it if the ledger still knows its checksum. When neither is found, the command says so and exits non-zero.

Replaying only the failed rows

php artisan sync:retry 01JQ8Z3M0K2N4P6R8T0V2X4Y6A --failed-rows

--failed-rows replays only the rows recorded in sync_row_failures for that run, using their captured payloads. It is the cheaper path when a run finished completed_with_errors and the cause was something outside the file — a reference table that had not been seeded, a rate that had not been loaded.

It differs from a whole-file retry in four ways:

  • It processes in the command, synchronously, not on the queue.
  • It maps and validates the stored payloads exactly as the original run did, including prepare() and duplicate resolution across the replayed set.
  • It needs payload capture to have been on for the original run. Without it the command refuses, naming data-sync.store_failed_payloads.
  • It does not apply to transfers, which have no rows.

Payloads whose captured values are not plain text — a run whose rows carried structured values — cannot be replayed this way either; the command tells you to retry the whole archived file instead.

Rows that fail again are recorded against the new run. The original run's failure records stay where they are.

Choosing between them

SituationUse
The run failed and nothing was writtensync:retry <run>
A worker died mid-batchsync:retry <run>
maxErrors() aborted the run after fixing the feedsync:retry <run>
Most rows imported; a few failed on missing reference datasync:retry <run> --failed-rows
The source file itself was wrongFix it and drop it again — a new checksum is a new file
A completed file must be imported again as-issync:run <name> --force

When retrying will not help

  • The definition is wrong. A schema mismatch, a bad matchOn(), a missing model attribute — fix the class first; the same file will fail the same way.
  • Payload capture was off and you need --failed-rows. Retry the whole file.
  • Retention has removed everything. Once both the run and its stored file are pruned, the only path left is a fresh copy from the supplier. Widen retention.failed_files_days if that keeps happening.