Matching and writing.

Once a row is mapped, three questions decide what happens to it: which existing record it is, whether that record may be created or updated, and which write path carries it to the database.

matchOn

public function matchOn(): array
{
    return ['sku'];
}

matchOn() names the mapped attributes that identify a row. Composite keys are a list:

return ['warehouse_code', 'sku'];

Every entry must be a mapped field; the definition fails validation otherwise. The identity is the JSON encoding of those attributes in matchOn() order, so a row with null in a match column has an identity too — it just matches every other row with null there.

Back matchOn() with a unique index. Without one, concurrent chunk jobs can insert the same logical row twice, and upsert() has nothing to conflict on. sync:doctor warns when a definition's match columns have no matching unique or primary index on the destination table.

Write modes

use ByRcsc\LaravelDataSync\Enums\WriteMode;

public function writeMode(): WriteMode
{
    return WriteMode::CreateOnly;
}

Skipped rows are counted, not failed. A CreateOnly feed that re-sends its whole catalogue every night therefore reports mostly skips, which is the signal you want.

Duplicates inside one file

Two rows in the same file can match each other. onDuplicate() decides which one counts:

use ByRcsc\LaravelDataSync\Enums\DuplicatePolicy;

public function onDuplicate(): DuplicatePolicy
{
    return DuplicatePolicy::FirstWins;
}

Resolution happens across the whole file before any chunk is written, which is why a queued run plans the file first. Losing rows are counted as skipped, not failed.

Rows that could not be mapped at all — a transform that threw, for example — are never treated as duplicates of each other. Each one is recorded as its own failure.

Chunks and transactions

public function chunkSize(): int
{
    return 500;
}

chunkSize() is the number of rows in one transaction, and — on the queued path — in one job. It defaults to 1000, and data-sync.overrides.<name>.chunk_size overrides it in config without touching the class.

Each chunk is written in its own transaction. A chunk that fails rolls back its own rows and leaves earlier chunks committed, which is what lets a large file make progress instead of losing an hour of work to row 900,000.

There is no cross-chunk atomicity. A run that fails part way through has already written the chunks that succeeded. The ledger records the checksum as failed, so the next sync:run picks the file up again and reprocesses it from the start — which is safe for Upsert and UpdateOnly definitions, and worth thinking through for CreateOnly ones.

Sizing it is a memory-versus-overhead trade: a chunk is held in memory as mapped rows, and each chunk costs a transaction and, when queued, a job. A few hundred to a few thousand rows suits most feeds. Rows with large captured JSON want smaller chunks.

Atomic files

public function atomic(): bool
{
    return true;
}

atomic() processes the whole file as a single job inside a single transaction: no batch, no parallelism, and the transaction held open for the duration. Use it for small files where partial application is worse than failure — a chart-of-accounts replacement, a rate table — and never for large ones, where it converts a resumable import into one long lock.

An atomic run persists no intermediate progress, so sync:status shows its counters only once it finishes.

Stopping a bad feed

public function maxErrors(): ?int
{
    return 50;
}

Failed rows do not stop a run; they are recorded and processing continues. maxErrors() is the guard against importing a mangled feed in full before anyone notices: once the run's failed count reaches the threshold, the run aborts with a MaxErrorsExceededException. On the queued path the chunk job cancels the batch, so sibling chunks stop too and the run finishes failed.

The threshold is checked after each chunk, so the actual failure count when a run aborts can exceed it by up to one chunk.

The write path tradeoff

A definition without beforeSave() or afterSave() writes rows through a bulk upsert. Declaring either hook switches the whole definition to per-row Eloquent writes, which is far slower but the only path where model events fire.

The package does not guess. If your models rely on observers, or you need to touch a relation per row, declare the hook and accept the cost. If you only need speed, keep the hooks off and do the follow-up work in a SyncCompleted listener.

The resolved path is stored on the run in write_path and printed by sync:status, so which path a run took is a fact you can look up rather than infer.

What each path actually does

The bulk path builds each row into a model instance with forceFill() — so casts and set mutators still apply — reads the resulting attributes, and writes one upsert() per chunk keyed on matchOn(). Match columns and created_at are excluded from the update list, so an existing row keeps its creation timestamp. A CreateOnly definition uses insertOrIgnore() instead.

Because a bulk write cannot report which rows were inserted and which were updated, the path runs one existence check per row to classify it. That is the cost of accurate created and updated counters on the fast path.

The Eloquent path resolves each row with firstOrNew() on the match attributes, applies the write mode, forceFill()s the row, calls beforeSave(), saves, and calls afterSave(). Returning false from beforeSave() skips the row without saving it.

What to read next

• Validation and hooks for the hooks that select the Eloquent path.
• Running a sync for how chunks are dispatched.
• Queues and workers for batching and worker requirements.