Browse documentationOpen

byrcsc/laravel-data-sync · 1.x

Configuration.

Every key in config/data-sync.php, its default, what it controls, and the validation applied to it at boot.

php artisan vendor:publish --tag="data-sync-config"

config/data-sync.php holds infrastructure only — which classes are registered, which disks, which queue, which retention windows. Behavior belongs on the definition.

return [
    'syncs' => [],

    'connections' => [],

    'staging' => [
        'disk' => 'local',
        'path' => 'data-sync/staging',
    ],

    'archive' => [
        'disk' => 'local',
        'path' => 'data-sync/archive',
    ],

    'failed' => [
        'disk' => null,
        'path' => 'data-sync/failed',
    ],

    'queue' => [
        'connection' => null,
        'queue' => null,
    ],

    'register_schedules' => true,

    'overrides' => [],

    'retention' => [
        'runs_days' => 90,
        'row_failures_days' => 30,
        'archive_days' => 30,
        'failed_files_days' => 14,
    ],

    'store_failed_payloads' => true,

    'tables' => [
        'files' => 'sync_files',
        'runs' => 'sync_runs',
        'row_failures' => 'sync_row_failures',
    ],
];

Reference

KeyDefaultPurpose
syncs[]Registered definition class names
connections[]Inline Flysystem configurations for Source::connection()
staging.disk / staging.pathlocal, data-sync/stagingWhere discovered files are copied for processing
archive.disk / archive.pathlocal, data-sync/archiveWhere processed files are kept
failed.disk / failed.pathnull, data-sync/failedWhere failed files are preserved; falls back to the staging disk
queue.connection / queue.queuenullDefault routing for package jobs
register_schedulestrueWhether the package registers scheduled commands
overrides[]Per-definition enabled, schedule, queue, chunk_size
retention.runs_days90Days finished runs are kept
retention.row_failures_days30Days failed-row records and payloads are kept
retention.archive_days30Days archived files are kept
retention.failed_files_days14Days preserved failed files are kept
store_failed_payloadstrueGlobal failed-row payload capture toggle
tables.files / .runs / .row_failuressync_files, sync_runs, sync_row_failuresTable names, read by the migration and the models

syncs

'syncs' => [
    App\Syncs\ProductsSync::class,
    App\Syncs\StatementsTransfer::class,
],

Each entry must name an existing class extending SyncDefinition or TransferDefinition, and no two may resolve to the same name. Violations throw an InvalidConfigurationException at boot.

connections

Inline Flysystem configurations, built on demand by Source::connection('<key>'):

'connections' => [
    'supplier-sftp' => [
        'driver' => 'sftp',
        'host' => env('SUPPLIER_SFTP_HOST'),
        'username' => env('SUPPLIER_SFTP_USERNAME'),
        'privateKey' => env('SUPPLIER_SFTP_KEY_PATH'),
        'root' => '/outbound',
    ],
],

Each entry must be an array with a non-empty driver. An ftp or sftp driver requires its Flysystem adapter — league/flysystem-ftp or league/flysystem-sftp-v3 — and throws at boot when it is missing. Keys beyond driver are passed straight to the driver, exactly as in config/filesystems.php.

staging, archive, failed

All three name a disk from config/filesystems.php and a path root within it. failed.disk falls back to the staging disk when it is null.

See installation for what each holds, and queues and workers for why a local staging disk is a single-machine configuration.

queue

'queue' => [
    'connection' => 'redis',
    'queue' => 'imports',
],

The default routing for every package job. Resolution order, most specific first:

Connectionoverrides.<name>.queue.connection, then the definition's queueConnection(), then queue.connection.

Queueoverrides.<name>.queue.queue, then the definition's queue(), then overrides.<name>.queue when it is a bare string, then queue.queue.

overrides

Per-definition operational settings, keyed by definition name. Only four keys are accepted; anything else throws at boot, which is deliberate — behavior belongs on the class, not in config.

'overrides' => [
    'products' => [
        'enabled' => true,
        'schedule' => '0 * * * *',
        'queue' => ['connection' => 'redis', 'queue' => 'imports'],
        'chunk_size' => 500,
    ],
],
KeyAcceptsNotes
enabledtrue / falsefalse removes the definition from sync:run with no name, and from scheduling
scheduleA cron expression, or falseWins over the definition's schedule(); false disables scheduling for it
queue['connection' => …, 'queue' => …] or a queue name stringSee above
chunk_sizeAn integer of at least 1Wins over the definition's chunkSize()

The key must name a registered definition. An override for an unknown name is a boot-time error rather than a silent no-op, which catches renamed definitions.

register_schedules

true by default. It registers sync:prune daily, plus sync:run <name> for every enabled definition that declares a schedule() method or has a cron override. Set it to false to register nothing and wire the commands up yourself.

retention

Four independent windows, in days. null disables one. See retention and pruning.

store_failed_payloads

The global switch for recording the raw payload of a failed row. It wins over a definition's captureFailedPayloads(), so turning it off here turns it off everywhere. See sensitive data.

tables

'tables' => [
    'files' => 'data_sync_files',
    'runs' => 'data_sync_runs',
    'row_failures' => 'data_sync_row_failures',
],

Read by both the migration and the models, so custom names work without subclassing anything.

Publish the config before the migration if you intend to change these. The migration reads the values at publish time; changing them afterwards means renaming tables by hand.

Validation

The whole file is validated when the package boots, and again by sync:doctor. An invalid configuration throws InvalidConfigurationException immediately rather than during a 2 a.m. run:

  • syncs entries exist, are the right type, and have unique names.
  • overrides keys name registered definitions and use only the four accepted settings, with a valid cron expression and a positive integer chunk size.
  • retention values are null or non-negative integers.
  • connections entries are arrays with a driver, and their adapters are installed.

Environment variables

The package ships no env() calls of its own — the published config is plain values. Wire your own where they help:

'staging' => [
    'disk' => env('DATA_SYNC_STAGING_DISK', 'local'),
    'path' => env('DATA_SYNC_STAGING_PATH', 'data-sync/staging'),
],

Remember that config:cache freezes the result, so env() outside config files returns null in production.