›
byrcsc/laravel-data-sync · 1.x
Move files between disks with checksum verification and run history.
Not every feed becomes model rows. A supplier's monthly statements need to land
on S3 under a predictable path, with a record of what arrived and when. That is
a TransferDefinition: the same discovery, ledger, run history, and commands as
a row sync, with a destination disk instead of a model write.
namespace App\Syncs;
use ByRcsc\LaravelDataSync\Data\TransferredFile;
use ByRcsc\LaravelDataSync\Definitions\Destination;
use ByRcsc\LaravelDataSync\Definitions\Source;
use ByRcsc\LaravelDataSync\Definitions\TransferDefinition;
use ByRcsc\LaravelDataSync\Enums\ExistingFilePolicy;
final class StatementsTransfer extends TransferDefinition
{
public function source(): Source
{
return Source::connection('supplier-sftp')->path('statements');
}
public function matchFilename(): string
{
return 'STMT-{account}-{issued_at:Ymd}.pdf';
}
public function destination(): Destination
{
return Destination::disk('s3')
->path('statements/{account}/{issued_at:Y/m}/{filename}')
->onExisting(ExistingFilePolicy::Fail);
}
public function afterTransfer(TransferredFile $file): void
{
// ...
}
}Scaffold one with:
php artisan make:sync StatementsTransfer --transferRegister it in data-sync.syncs alongside row syncs. It runs through the same
sync:run, sync:status, and sync:retry commands.
matchFilename() is a template, not a regex:
{name} captures a value.{name:format} captures it and parses it as a date with that PHP date format.return 'STMT-{account}-{issued_at:Ymd}.pdf';STMT-000431-20260731.pdf yields account => '000431' and issued_at as a
DateTimeImmutable for 2026-07-31.
Each segment name may appear only once in a pattern. A pattern with no segments,
or a repeated one, throws. A filename whose date segment does not parse, a
{issued_at:Ymd} matching 2026JULY, throws too, so a malformed name is a
failed run rather than a silently wrong path.
Captured values are available to the destination path template, to
afterTransfer(), and to model linking.
use ByRcsc\LaravelDataSync\Enums\UnmatchedFilenamePolicy;
public function onUnmatched(): UnmatchedFilenamePolicy
{
return UnmatchedFilenamePolicy::Fail;
}| Policy | Effect |
|---|---|
UnmatchedFilenamePolicy::Skip | Default. The run completes with one skipped row and the reason in error_summary; the checksum is marked processed so the file is not seen again |
UnmatchedFilenamePolicy::Fail | The run fails; the file is preserved for retry |
Skip suits a shared directory that carries files for several definitions.
Fail suits a directory that should only ever hold one kind of file, a stray
name there is a supplier mistake worth noticing.
Destination::disk('s3')
->path('statements/{account}/{issued_at:Y/m}/{filename}')
->onExisting(ExistingFilePolicy::Fail);The path is a template rendered from the captured segments plus a set of values the package always provides:
| Value | What it is |
|---|---|
{filename} | The source filename, including extension |
{extension} | The extension, without the dot |
{checksum} | The SHA-256 checksum of the contents |
{size} | Size in bytes |
{mtime} | Source modification time, as a Unix timestamp |
{source_path} | The path the file was discovered at |
Date segments accept a format, {issued_at:Y/m} renders 2026/07, and
default to Y-m-d when none is given. Referencing a value that does not exist
throws, naming the offending placeholder. The default path is {filename}.
Paths are rendered per file, so a template with a date segment naturally partitions the destination by period.
Before writing, the package checks whether the destination path is already occupied:
rows_skipped = 1, nothing is rewritten, and afterTransfer() still runs with
$file->skipped === true.ExistingFilePolicy::Overwrite, the default, writes
over it. ExistingFilePolicy::Fail fails the run instead.Fail is the safer choice for regulated documents, where a second file claiming
the same identity is a supplier problem rather than an update.
The file is streamed from staging to the destination and then re-checksummed at the destination. A mismatch deletes the written file and fails the run, so a truncated upload never survives as a half-written statement.
Only after that verification does the configured source action run against the supplier's copy. The action can archive, move, or delete the source.
use ByRcsc\LaravelDataSync\Data\TransferredFile;
public function afterTransfer(TransferredFile $file): void
{
$file->run; // the SyncRun model
$file->file; // the SyncFile ledger row
$file->destinationDisk; // 's3'
$file->destinationPath; // 'statements/000431/2026/07/STMT-000431-20260731.pdf'
$file->checksum;
$file->segments; // ['account' => '000431', 'issued_at' => DateTimeImmutable]
$file->skipped; // true when the destination already held this file
}afterTransfer() runs on real runs only, never on a dry run, and before the
run is finalized. A throw here fails the run.
Unlike a row sync's afterSave(), it is not wrapped in a database transaction.
Anything it writes stays written even if the run later fails.
Set model(), fields(), and matchOn() on a transfer to record one Eloquent
row per transferred file, mapped from the same value set the path template uses:
use App\Models\Statement;
use ByRcsc\LaravelDataSync\Definitions\Field;
public function model(): ?string
{
return Statement::class;
}
public function fields(): array
{
return [
Field::make('account_number', from: 'account'),
Field::make('issued_on', from: 'issued_at'),
Field::make('path'),
Field::make('checksum'),
Field::make('bytes', from: 'size'),
];
}
public function matchOn(): array
{
return ['checksum'];
}from names a key in that value set, a captured segment, or one of the
built-in values, plus path, the rendered destination path. A field referencing
a value that does not exist throws unless it is optional() or has a
default().
The row is written with a single upsert keyed on matchOn(). Model events do
not fire. sync:doctor warns when the match columns have no unique index. These
are the same rules as the bulk write
path.
prepare(), write
hooks, chunking, or atomic(). One file is one unit of work.sync:retry --failed-rows. There are no failed rows to replay; retry the
whole file instead.transfer.