byrcsc/laravel-data-sync · 1.x
File transfers.
Move files from a source to a destination disk with checksum verification, filename metadata, templated destination paths, and an optional Eloquent record per file.
Not every feed becomes model rows. A supplier's monthly statements need to land
on S3 under a predictable path, with a record of what arrived and when. That is
a TransferDefinition: the same discovery, ledger, run history, and commands as
a row sync, with a destination disk instead of a model write.
namespace App\Syncs;
use ByRcsc\LaravelDataSync\Data\TransferredFile;
use ByRcsc\LaravelDataSync\Definitions\Destination;
use ByRcsc\LaravelDataSync\Definitions\Source;
use ByRcsc\LaravelDataSync\Definitions\TransferDefinition;
use ByRcsc\LaravelDataSync\Enums\ExistingFilePolicy;
final class StatementsTransfer extends TransferDefinition
{
public function source(): Source
{
return Source::connection('supplier-sftp')->path('statements');
}
public function matchFilename(): string
{
return 'STMT-{account}-{issued_at:Ymd}.pdf';
}
public function destination(): Destination
{
return Destination::disk('s3')
->path('statements/{account}/{issued_at:Y/m}/{filename}')
->onExisting(ExistingFilePolicy::Fail);
}
public function afterTransfer(TransferredFile $file): void
{
// ...
}
}Scaffold one with:
php artisan make:sync StatementsTransfer --transferRegister it in data-sync.syncs alongside row syncs. It runs through the same
sync:run, sync:status, and sync:retry commands.
Filename patterns
matchFilename() is a template, not a regex:
{name}captures a value.{name:format}captures it and parses it as a date with that PHP date format.- Everything else matches literally.
return 'STMT-{account}-{issued_at:Ymd}.pdf';STMT-000431-20260731.pdf yields account => '000431' and issued_at as a
DateTimeImmutable for 2026-07-31.
Each segment name may appear only once in a pattern. A pattern with no segments,
or a repeated one, throws. A filename whose date segment does not parse — a
{issued_at:Ymd} matching 2026JULY — throws too, so a malformed name is a
failed run rather than a silently wrong path.
Captured values are available to the destination path template, to
afterTransfer(), and to model linking.
Files that do not match
use ByRcsc\LaravelDataSync\Enums\UnmatchedFilenamePolicy;
public function onUnmatched(): UnmatchedFilenamePolicy
{
return UnmatchedFilenamePolicy::Fail;
}| Policy | Effect |
|---|---|
UnmatchedFilenamePolicy::Skip | Default. The run completes with one skipped row and the reason in error_summary; the checksum is marked processed so the file is not seen again |
UnmatchedFilenamePolicy::Fail | The run fails; the file is preserved for retry |
Skip suits a shared directory that carries files for several definitions.
Fail suits a directory that should only ever hold one kind of file — a stray
name there is a supplier mistake worth noticing.
Destination paths
Destination::disk('s3')
->path('statements/{account}/{issued_at:Y/m}/{filename}')
->onExisting(ExistingFilePolicy::Fail);The path is a template rendered from the captured segments plus a set of values the package always provides:
| Value | What it is |
|---|---|
{filename} | The source filename, including extension |
{extension} | The extension, without the dot |
{checksum} | The SHA-256 checksum of the contents |
{size} | Size in bytes |
{mtime} | Source modification time, as a Unix timestamp |
{source_path} | The path the file was discovered at |
Date segments accept a format — {issued_at:Y/m} renders 2026/07 — and
default to Y-m-d when none is given. Referencing a value that does not exist
throws, naming the offending placeholder. The default path is {filename}.
Paths are rendered per file, so a template with a date segment naturally partitions the destination by period.
Existing files at the destination
Before writing, the package checks whether the destination path is already occupied:
- Same checksum — the transfer is skipped. The run completes with
rows_skipped = 1, nothing is rewritten, andafterTransfer()still runs with$file->skipped === true. - Different checksum —
ExistingFilePolicy::Overwrite, the default, writes over it.ExistingFilePolicy::Failfails the run instead.
Fail is the safer choice for regulated documents, where a second file claiming
the same identity is a supplier problem rather than an update.
Verification
The file is streamed from staging to the destination and then re-checksummed at the destination. A mismatch deletes the written file and fails the run, so a truncated upload never survives as a half-written statement.
Only after that verification does the source action — archive, move, or delete — run against the supplier's copy.
After a transfer
use ByRcsc\LaravelDataSync\Data\TransferredFile;
public function afterTransfer(TransferredFile $file): void
{
$file->run; // the SyncRun model
$file->file; // the SyncFile ledger row
$file->destinationDisk; // 's3'
$file->destinationPath; // 'statements/000431/2026/07/STMT-000431-20260731.pdf'
$file->checksum;
$file->segments; // ['account' => '000431', 'issued_at' => DateTimeImmutable]
$file->skipped; // true when the destination already held this file
}afterTransfer() runs on real runs only — never on a dry run — and before the
run is finalized. A throw here fails the run.
Unlike a row sync's afterSave(), it is not wrapped in a database transaction.
Anything it writes stays written even if the run later fails.
Recording a row per file
Set model(), fields(), and matchOn() on a transfer to record one Eloquent
row per transferred file, mapped from the same value set the path template uses:
use App\Models\Statement;
use ByRcsc\LaravelDataSync\Definitions\Field;
public function model(): ?string
{
return Statement::class;
}
public function fields(): array
{
return [
Field::make('account_number', from: 'account'),
Field::make('issued_on', from: 'issued_at'),
Field::make('path'),
Field::make('checksum'),
Field::make('bytes', from: 'size'),
];
}
public function matchOn(): array
{
return ['checksum'];
}from names a key in that value set — a captured segment, or one of the
built-in values, plus path, the rendered destination path. A field referencing
a value that does not exist throws unless it is optional() or has a
default().
The row is written with a single upsert keyed on matchOn(). Model events do
not fire, and sync:doctor warns when the match columns have no unique index —
the same rules as the bulk write path.
What transfers do not support
- Row-level anything. No readers, validation rules,
prepare(), write hooks, chunking, oratomic(). One file is one unit of work. sync:retry --failed-rows. There are no failed rows to replay; retry the whole file instead.- Reading file contents. A transfer moves bytes and reads the filename. Nothing looks inside a PDF.
What to read next
- Sources and connections for where transferred files come from.
- Run history and status for reading transfer runs,
whose write path shows as
transfer. - Failures and retries for retrying one.