byrcsc/laravel-data-sync · 1.x
Sensitive data.
What the package copies where, how to turn failed-row payload capture off per definition or globally, and what you give up when you do.
A sync makes copies. Knowing which ones exist, and for how long, is the whole of this page.
| Copy | Where it lives | Retention |
|---|---|---|
| The staged file | data-sync.staging disk | Not pruned — see below |
| The archived file | data-sync.archive disk | retention.archive_days (30) |
| The preserved failed file | data-sync.failed disk | retention.failed_files_days (14) |
| Failed-row payloads | sync_row_failures.payload | retention.row_failures_days (30) |
| Ledger metadata | sync_files | Never pruned |
Staging is not swept. A dry run discards its staged copy, and a failed run moves its copy to the failed disk, but a successfully processed file leaves its staged copy behind, and
sync:prunedoes not touch the staging disk. Over time the staging area holds a copy of every file the package has imported. Sweep it yourself — a scheduledfind, an object-lifecycle rule on the bucket — sized to how far back a retry needs to reach.
The ledger holds a checksum, a path, a filename, a size, and timestamps — no row contents. Everything else that carries actual data is on a retention window.
Failed-row payloads
When a row fails, its payload is recorded in sync_row_failures alongside the
validation errors. That is what makes sync:status --failures readable and
sync:retry --failed-rows
possible.
For a feed carrying personal or financial data, it is also a copy of that data in a second table, with a different retention window and probably different access controls than the destination model.
Turn it off for one definition:
public function captureFailedPayloads(): bool
{
return false;
}Or, when you build the definition yourself:
$definition = (new PayrollSync)->withoutPayloadCapture();Turn it off everywhere:
// config/data-sync.php
'store_failed_payloads' => false,The global toggle wins. A definition cannot re-enable capture that config has turned off.
What you keep, and what you lose
With capture off, failures still record the row number, the validation errors,
and the exception message — everything except the data itself. You keep the
counts, the reasons, and the histogram in error_summary.
You lose two things:
sync:status --failuresprints[payload capture disabled]instead of the row.sync:retry --failed-rowshas nothing to replay and refuses to run. Retrying the whole archived file still works.
The archived file is the bigger copy
Payload capture covers failed rows. The archive covers every row: it is a
byte-for-byte copy of the supplier's file, kept for retention.archive_days.
For a sensitive feed that is usually the copy worth thinking about first.
Options, in order of how much they cost you:
- Shorten
retention.archive_days. Retries and after-the-fact diagnosis stop working past that window, but the ledger still prevents reprocessing. - Point the archive, staging, and failed disks at encrypted storage — a server-side-encrypted bucket, an encrypted volume. The package writes through Flysystem, so this is a disk configuration concern rather than a package one.
- Restrict access to those disks separately from the application's public storage. Nothing in the package serves archived files to users.
Note that the failed disk holds a copy for retention.failed_files_days after a
failure, and the staging disk keeps one indefinitely unless you sweep it. Point
all three somewhere appropriate, not just the archive.
Sensitive columns you do not want at all
Two mechanisms keep data out of the destination model:
- Map only the fields you need. Unmapped columns are ignored by default, and are
never written — only their names are recorded, on the run's
unmapped_columns. - Do not enable
UnknownColumnPolicy::Captureon a feed with sensitive extras. Capture writes every unmapped column into a JSON attribute, which is the opposite of what you want here. If you need it anyway, cast the target withencrypted:array.
Neither removes the data from the archived file, which is a copy of what the supplier sent.
What to read next
- Retention and pruning for the windows and how to change them.
- Failures and retries for what capture buys you.
- Configuration for the full key reference.