›
›
›
  1. docs
  2. ›
  3. byrcsc/laravel-data-sync
1.x
Browse documentationOpenClose

Getting started

  • Introduction
  • Installation and setup
  • Quick start

Core concepts

  • File identity and runs
  • Sync definitions
  • Sources and connections
  • File formats and readers
  • Field mapping
  • Matching and writing
  • Validation and hooks

Running syncs

  • Running a sync
  • Queues and workers
  • Scheduling

Transfers

  • File transfers

Operations

  • Run history and status
  • Events and listeners
  • Failures and retries
  • Sensitive data
  • Health checks
  • Retention and pruning

Reference

  • Configuration
  • Console commands
  • Testing
  • Troubleshooting

Getting started

  • Introduction
  • Installation and setup
  • Quick start

Core concepts

  • File identity and runs
  • Sync definitions
  • Sources and connections
  • File formats and readers
  • Field mapping
  • Matching and writing
  • Validation and hooks

Running syncs

  • Running a sync
  • Queues and workers
  • Scheduling

Transfers

  • File transfers

Operations

  • Run history and status
  • Events and listeners
  • Failures and retries
  • Sensitive data
  • Health checks
  • Retention and pruning

Reference

  • Configuration
  • Console commands
  • Testing
  • Troubleshooting

byrcsc/laravel-data-sync · 1.x

File transfers.

Move files between disks with checksum verification and run history.

Not every feed becomes model rows. A supplier's monthly statements need to land on S3 under a predictable path, with a record of what arrived and when. That is a TransferDefinition: the same discovery, ledger, run history, and commands as a row sync, with a destination disk instead of a model write.

namespace App\Syncs;

use ByRcsc\LaravelDataSync\Data\TransferredFile;
use ByRcsc\LaravelDataSync\Definitions\Destination;
use ByRcsc\LaravelDataSync\Definitions\Source;
use ByRcsc\LaravelDataSync\Definitions\TransferDefinition;
use ByRcsc\LaravelDataSync\Enums\ExistingFilePolicy;

final class StatementsTransfer extends TransferDefinition
{
    public function source(): Source
    {
        return Source::connection('supplier-sftp')->path('statements');
    }

    public function matchFilename(): string
    {
        return 'STMT-{account}-{issued_at:Ymd}.pdf';
    }

    public function destination(): Destination
    {
        return Destination::disk('s3')
            ->path('statements/{account}/{issued_at:Y/m}/{filename}')
            ->onExisting(ExistingFilePolicy::Fail);
    }

    public function afterTransfer(TransferredFile $file): void
    {
        // ...
    }
}

Scaffold one with:

php artisan make:sync StatementsTransfer --transfer

Register it in data-sync.syncs alongside row syncs. It runs through the same sync:run, sync:status, and sync:retry commands.

Filename patterns

matchFilename() is a template, not a regex:

  • {name} captures a value.
  • {name:format} captures it and parses it as a date with that PHP date format.
  • Everything else matches literally.
return 'STMT-{account}-{issued_at:Ymd}.pdf';

STMT-000431-20260731.pdf yields account => '000431' and issued_at as a DateTimeImmutable for 2026-07-31.

Each segment name may appear only once in a pattern. A pattern with no segments, or a repeated one, throws. A filename whose date segment does not parse, a {issued_at:Ymd} matching 2026JULY, throws too, so a malformed name is a failed run rather than a silently wrong path.

Captured values are available to the destination path template, to afterTransfer(), and to model linking.

Files that do not match

use ByRcsc\LaravelDataSync\Enums\UnmatchedFilenamePolicy;

public function onUnmatched(): UnmatchedFilenamePolicy
{
    return UnmatchedFilenamePolicy::Fail;
}
PolicyEffect
UnmatchedFilenamePolicy::SkipDefault. The run completes with one skipped row and the reason in error_summary; the checksum is marked processed so the file is not seen again
UnmatchedFilenamePolicy::FailThe run fails; the file is preserved for retry

Skip suits a shared directory that carries files for several definitions. Fail suits a directory that should only ever hold one kind of file, a stray name there is a supplier mistake worth noticing.

Destination paths

Destination::disk('s3')
    ->path('statements/{account}/{issued_at:Y/m}/{filename}')
    ->onExisting(ExistingFilePolicy::Fail);

The path is a template rendered from the captured segments plus a set of values the package always provides:

ValueWhat it is
{filename}The source filename, including extension
{extension}The extension, without the dot
{checksum}The SHA-256 checksum of the contents
{size}Size in bytes
{mtime}Source modification time, as a Unix timestamp
{source_path}The path the file was discovered at

Date segments accept a format, {issued_at:Y/m} renders 2026/07, and default to Y-m-d when none is given. Referencing a value that does not exist throws, naming the offending placeholder. The default path is {filename}.

Paths are rendered per file, so a template with a date segment naturally partitions the destination by period.

Existing files at the destination

Before writing, the package checks whether the destination path is already occupied:

  • Same checksum, the transfer is skipped. The run completes with rows_skipped = 1, nothing is rewritten, and afterTransfer() still runs with $file->skipped === true.
  • Different checksum, ExistingFilePolicy::Overwrite, the default, writes over it. ExistingFilePolicy::Fail fails the run instead.

Fail is the safer choice for regulated documents, where a second file claiming the same identity is a supplier problem rather than an update.

Verification

The file is streamed from staging to the destination and then re-checksummed at the destination. A mismatch deletes the written file and fails the run, so a truncated upload never survives as a half-written statement.

Only after that verification does the configured source action run against the supplier's copy. The action can archive, move, or delete the source.

After a transfer

use ByRcsc\LaravelDataSync\Data\TransferredFile;

public function afterTransfer(TransferredFile $file): void
{
    $file->run;               // the SyncRun model
    $file->file;              // the SyncFile ledger row
    $file->destinationDisk;   // 's3'
    $file->destinationPath;   // 'statements/000431/2026/07/STMT-000431-20260731.pdf'
    $file->checksum;
    $file->segments;          // ['account' => '000431', 'issued_at' => DateTimeImmutable]
    $file->skipped;           // true when the destination already held this file
}

afterTransfer() runs on real runs only, never on a dry run, and before the run is finalized. A throw here fails the run.

Unlike a row sync's afterSave(), it is not wrapped in a database transaction. Anything it writes stays written even if the run later fails.

Recording a row per file

Set model(), fields(), and matchOn() on a transfer to record one Eloquent row per transferred file, mapped from the same value set the path template uses:

use App\Models\Statement;
use ByRcsc\LaravelDataSync\Definitions\Field;

public function model(): ?string
{
    return Statement::class;
}

public function fields(): array
{
    return [
        Field::make('account_number', from: 'account'),
        Field::make('issued_on', from: 'issued_at'),
        Field::make('path'),
        Field::make('checksum'),
        Field::make('bytes', from: 'size'),
    ];
}

public function matchOn(): array
{
    return ['checksum'];
}

from names a key in that value set, a captured segment, or one of the built-in values, plus path, the rendered destination path. A field referencing a value that does not exist throws unless it is optional() or has a default().

The row is written with a single upsert keyed on matchOn(). Model events do not fire. sync:doctor warns when the match columns have no unique index. These are the same rules as the bulk write path.

What transfers do not support

  • Row-level anything. No readers, validation rules, prepare(), write hooks, chunking, or atomic(). One file is one unit of work.
  • sync:retry --failed-rows. There are no failed rows to replay; retry the whole file instead.
  • Reading file contents. A transfer moves bytes and reads the filename. Nothing looks inside a PDF.

What to read next

  • Sources and connections for where transferred files come from.
  • Run history and status for reading transfer runs, whose write path shows as transfer.
  • Failures and retries for retrying one.
PreviousSchedulingNextRun history and status
View source

On this page

  1. Filename patterns
  2. Files that do not match
  3. Destination paths
  4. Existing files at the destination
  5. Verification
  6. After a transfer
  7. Recording a row per file
  8. What transfers do not support
  9. What to read next