›
byrcsc/laravel-data-sync · 1.x
Configure sync classes, disks, queues, connections, and retention windows.
php artisan vendor:publish --tag="data-sync-config"config/data-sync.php holds infrastructure only, which classes are
registered, which disks, which queue, which retention windows. Behavior belongs
on the definition.
return [
'syncs' => [],
'connections' => [],
'staging' => [
'disk' => 'local',
'path' => 'data-sync/staging',
],
'archive' => [
'disk' => 'local',
'path' => 'data-sync/archive',
],
'failed' => [
'disk' => null,
'path' => 'data-sync/failed',
],
'queue' => [
'connection' => null,
'queue' => null,
],
'register_schedules' => true,
'overrides' => [],
'retention' => [
'runs_days' => 90,
'row_failures_days' => 30,
'archive_days' => 30,
'failed_files_days' => 14,
],
'store_failed_payloads' => true,
'tables' => [
'files' => 'sync_files',
'runs' => 'sync_runs',
'row_failures' => 'sync_row_failures',
],
];| Key | Default | Purpose |
|---|---|---|
syncs | [] | Registered definition class names |
connections | [] | Inline Flysystem configurations for Source::connection() |
staging.disk / staging.path | local, data-sync/staging | Where discovered files are copied for processing |
archive.disk / archive.path | local, data-sync/archive | Where processed files are kept |
failed.disk / failed.path | null, data-sync/failed | Where failed files are preserved; falls back to the staging disk |
queue.connection / queue.queue | null | Default routing for package jobs |
register_schedules | true | Whether the package registers scheduled commands |
overrides | [] | Per-definition enabled, schedule, queue, chunk_size |
retention.runs_days | 90 | Days finished runs are kept |
retention.row_failures_days | 30 | Days failed-row records and payloads are kept |
retention.archive_days | 30 | Days archived files are kept |
retention.failed_files_days | 14 | Days preserved failed files are kept |
store_failed_payloads | true | Global failed-row payload capture toggle |
tables.files / .runs / .row_failures | sync_files, sync_runs, sync_row_failures | Table names, read by the migration and the models |
'syncs' => [
App\Syncs\ProductsSync::class,
App\Syncs\StatementsTransfer::class,
],Each entry must name an existing class extending SyncDefinition or
TransferDefinition, and no two may resolve to the same
name. Violations throw an
InvalidConfigurationException at boot.
Inline Flysystem configurations, built on demand by
Source::connection('<key>'):
'connections' => [
'supplier-sftp' => [
'driver' => 'sftp',
'host' => env('SUPPLIER_SFTP_HOST'),
'username' => env('SUPPLIER_SFTP_USERNAME'),
'privateKey' => env('SUPPLIER_SFTP_KEY_PATH'),
'root' => '/outbound',
],
],Each entry must be an array with a non-empty driver. An ftp or sftp driver
requires its Flysystem adapter, league/flysystem-ftp or
league/flysystem-sftp-v3, and throws at boot when it is missing. Keys beyond
driver are passed straight to the driver, exactly as in
config/filesystems.php.
All three name a disk from config/filesystems.php and a path root within it.
failed.disk falls back to the staging disk when it is null.
See installation for what each holds, and queues and
workers for why a local
staging disk is a single-machine configuration.
'queue' => [
'connection' => 'redis',
'queue' => 'imports',
],The default routing for every package job. Resolution order, most specific first:
Connection, overrides.<name>.queue.connection, then the definition's
queueConnection(), then queue.connection.
Queue, overrides.<name>.queue.queue, then the definition's queue(),
then overrides.<name>.queue when it is a bare string, then queue.queue.
Per-definition operational settings, keyed by definition name. Only four keys are accepted; anything else throws at boot, which is deliberate, behavior belongs on the class, not in config.
'overrides' => [
'products' => [
'enabled' => true,
'schedule' => '0 * * * *',
'queue' => ['connection' => 'redis', 'queue' => 'imports'],
'chunk_size' => 500,
],
],| Key | Accepts | Notes |
|---|---|---|
enabled | true / false | false removes the definition from sync:run with no name, and from scheduling |
schedule | A cron expression, or false | Wins over the definition's schedule(); false disables scheduling for it |
queue | ['connection' => …, 'queue' => …] or a queue name string | See above |
chunk_size | An integer of at least 1 | Wins over the definition's chunkSize() |
The key must name a registered definition. An override for an unknown name is a boot-time error rather than a silent no-op, which catches renamed definitions.
true by default. It registers sync:prune daily, plus sync:run <name> for
every enabled definition that declares a schedule() method or has a cron
override. Set it to false to register nothing and wire the commands up
yourself.
Four independent windows, in days. null disables one. See retention and
pruning.
The global switch for recording the raw payload of a failed row. It wins over a
definition's captureFailedPayloads(), so turning it off here turns it off
everywhere. See sensitive data.
'tables' => [
'files' => 'data_sync_files',
'runs' => 'data_sync_runs',
'row_failures' => 'data_sync_row_failures',
],Read by both the migration and the models, so custom names work without subclassing anything.
Publish the config before the migration if you intend to change these. The migration reads the values at publish time; changing them afterwards means renaming tables by hand.
The whole file is validated when the package boots, and again by
sync:doctor. An invalid configuration throws
InvalidConfigurationException immediately rather than during a 2 a.m. run:
syncs entries exist, are the right type, and have unique names.overrides keys name registered definitions and use only the four accepted
settings, with a valid cron expression and a positive integer chunk size.retention values are null or non-negative integers.connections entries are arrays with a driver, and their adapters are
installed.The package ships no env() calls of its own, the published config is plain
values. Wire your own where they help:
'staging' => [
'disk' => env('DATA_SYNC_STAGING_DISK', 'local'),
'path' => env('DATA_SYNC_STAGING_PATH', 'data-sync/staging'),
],Remember that config:cache freezes the result, so env() outside config files
returns null in production.