Browse documentationOpen

byrcsc/laravel-data-sync · 1.x

File transfers.

Move files from a source to a destination disk with checksum verification, filename metadata, templated destination paths, and an optional Eloquent record per file.

Not every feed becomes model rows. A supplier's monthly statements need to land on S3 under a predictable path, with a record of what arrived and when. That is a TransferDefinition: the same discovery, ledger, run history, and commands as a row sync, with a destination disk instead of a model write.

namespace App\Syncs;

use ByRcsc\LaravelDataSync\Data\TransferredFile;
use ByRcsc\LaravelDataSync\Definitions\Destination;
use ByRcsc\LaravelDataSync\Definitions\Source;
use ByRcsc\LaravelDataSync\Definitions\TransferDefinition;
use ByRcsc\LaravelDataSync\Enums\ExistingFilePolicy;

final class StatementsTransfer extends TransferDefinition
{
    public function source(): Source
    {
        return Source::connection('supplier-sftp')->path('statements');
    }

    public function matchFilename(): string
    {
        return 'STMT-{account}-{issued_at:Ymd}.pdf';
    }

    public function destination(): Destination
    {
        return Destination::disk('s3')
            ->path('statements/{account}/{issued_at:Y/m}/{filename}')
            ->onExisting(ExistingFilePolicy::Fail);
    }

    public function afterTransfer(TransferredFile $file): void
    {
        // ...
    }
}

Scaffold one with:

php artisan make:sync StatementsTransfer --transfer

Register it in data-sync.syncs alongside row syncs. It runs through the same sync:run, sync:status, and sync:retry commands.

Filename patterns

matchFilename() is a template, not a regex:

  • {name} captures a value.
  • {name:format} captures it and parses it as a date with that PHP date format.
  • Everything else matches literally.
return 'STMT-{account}-{issued_at:Ymd}.pdf';

STMT-000431-20260731.pdf yields account => '000431' and issued_at as a DateTimeImmutable for 2026-07-31.

Each segment name may appear only once in a pattern. A pattern with no segments, or a repeated one, throws. A filename whose date segment does not parse — a {issued_at:Ymd} matching 2026JULY — throws too, so a malformed name is a failed run rather than a silently wrong path.

Captured values are available to the destination path template, to afterTransfer(), and to model linking.

Files that do not match

use ByRcsc\LaravelDataSync\Enums\UnmatchedFilenamePolicy;

public function onUnmatched(): UnmatchedFilenamePolicy
{
    return UnmatchedFilenamePolicy::Fail;
}
PolicyEffect
UnmatchedFilenamePolicy::SkipDefault. The run completes with one skipped row and the reason in error_summary; the checksum is marked processed so the file is not seen again
UnmatchedFilenamePolicy::FailThe run fails; the file is preserved for retry

Skip suits a shared directory that carries files for several definitions. Fail suits a directory that should only ever hold one kind of file — a stray name there is a supplier mistake worth noticing.

Destination paths

Destination::disk('s3')
    ->path('statements/{account}/{issued_at:Y/m}/{filename}')
    ->onExisting(ExistingFilePolicy::Fail);

The path is a template rendered from the captured segments plus a set of values the package always provides:

ValueWhat it is
{filename}The source filename, including extension
{extension}The extension, without the dot
{checksum}The SHA-256 checksum of the contents
{size}Size in bytes
{mtime}Source modification time, as a Unix timestamp
{source_path}The path the file was discovered at

Date segments accept a format — {issued_at:Y/m} renders 2026/07 — and default to Y-m-d when none is given. Referencing a value that does not exist throws, naming the offending placeholder. The default path is {filename}.

Paths are rendered per file, so a template with a date segment naturally partitions the destination by period.

Existing files at the destination

Before writing, the package checks whether the destination path is already occupied:

  • Same checksum — the transfer is skipped. The run completes with rows_skipped = 1, nothing is rewritten, and afterTransfer() still runs with $file->skipped === true.
  • Different checksumExistingFilePolicy::Overwrite, the default, writes over it. ExistingFilePolicy::Fail fails the run instead.

Fail is the safer choice for regulated documents, where a second file claiming the same identity is a supplier problem rather than an update.

Verification

The file is streamed from staging to the destination and then re-checksummed at the destination. A mismatch deletes the written file and fails the run, so a truncated upload never survives as a half-written statement.

Only after that verification does the source action — archive, move, or delete — run against the supplier's copy.

After a transfer

use ByRcsc\LaravelDataSync\Data\TransferredFile;

public function afterTransfer(TransferredFile $file): void
{
    $file->run;               // the SyncRun model
    $file->file;              // the SyncFile ledger row
    $file->destinationDisk;   // 's3'
    $file->destinationPath;   // 'statements/000431/2026/07/STMT-000431-20260731.pdf'
    $file->checksum;
    $file->segments;          // ['account' => '000431', 'issued_at' => DateTimeImmutable]
    $file->skipped;           // true when the destination already held this file
}

afterTransfer() runs on real runs only — never on a dry run — and before the run is finalized. A throw here fails the run.

Unlike a row sync's afterSave(), it is not wrapped in a database transaction. Anything it writes stays written even if the run later fails.

Recording a row per file

Set model(), fields(), and matchOn() on a transfer to record one Eloquent row per transferred file, mapped from the same value set the path template uses:

use App\Models\Statement;
use ByRcsc\LaravelDataSync\Definitions\Field;

public function model(): ?string
{
    return Statement::class;
}

public function fields(): array
{
    return [
        Field::make('account_number', from: 'account'),
        Field::make('issued_on', from: 'issued_at'),
        Field::make('path'),
        Field::make('checksum'),
        Field::make('bytes', from: 'size'),
    ];
}

public function matchOn(): array
{
    return ['checksum'];
}

from names a key in that value set — a captured segment, or one of the built-in values, plus path, the rendered destination path. A field referencing a value that does not exist throws unless it is optional() or has a default().

The row is written with a single upsert keyed on matchOn(). Model events do not fire, and sync:doctor warns when the match columns have no unique index — the same rules as the bulk write path.

What transfers do not support

  • Row-level anything. No readers, validation rules, prepare(), write hooks, chunking, or atomic(). One file is one unit of work.
  • sync:retry --failed-rows. There are no failed rows to replay; retry the whole file instead.
  • Reading file contents. A transfer moves bytes and reads the filename. Nothing looks inside a PDF.