byrcsc/laravel-data-sync · 1.x
Configuration.
Every key in config/data-sync.php, its default, what it controls, and the validation applied to it at boot.
php artisan vendor:publish --tag="data-sync-config"config/data-sync.php holds infrastructure only — which classes are
registered, which disks, which queue, which retention windows. Behavior belongs
on the definition.
return [
'syncs' => [],
'connections' => [],
'staging' => [
'disk' => 'local',
'path' => 'data-sync/staging',
],
'archive' => [
'disk' => 'local',
'path' => 'data-sync/archive',
],
'failed' => [
'disk' => null,
'path' => 'data-sync/failed',
],
'queue' => [
'connection' => null,
'queue' => null,
],
'register_schedules' => true,
'overrides' => [],
'retention' => [
'runs_days' => 90,
'row_failures_days' => 30,
'archive_days' => 30,
'failed_files_days' => 14,
],
'store_failed_payloads' => true,
'tables' => [
'files' => 'sync_files',
'runs' => 'sync_runs',
'row_failures' => 'sync_row_failures',
],
];Reference
| Key | Default | Purpose |
|---|---|---|
syncs | [] | Registered definition class names |
connections | [] | Inline Flysystem configurations for Source::connection() |
staging.disk / staging.path | local, data-sync/staging | Where discovered files are copied for processing |
archive.disk / archive.path | local, data-sync/archive | Where processed files are kept |
failed.disk / failed.path | null, data-sync/failed | Where failed files are preserved; falls back to the staging disk |
queue.connection / queue.queue | null | Default routing for package jobs |
register_schedules | true | Whether the package registers scheduled commands |
overrides | [] | Per-definition enabled, schedule, queue, chunk_size |
retention.runs_days | 90 | Days finished runs are kept |
retention.row_failures_days | 30 | Days failed-row records and payloads are kept |
retention.archive_days | 30 | Days archived files are kept |
retention.failed_files_days | 14 | Days preserved failed files are kept |
store_failed_payloads | true | Global failed-row payload capture toggle |
tables.files / .runs / .row_failures | sync_files, sync_runs, sync_row_failures | Table names, read by the migration and the models |
syncs
'syncs' => [
App\Syncs\ProductsSync::class,
App\Syncs\StatementsTransfer::class,
],Each entry must name an existing class extending SyncDefinition or
TransferDefinition, and no two may resolve to the same
name. Violations throw an
InvalidConfigurationException at boot.
connections
Inline Flysystem configurations, built on demand by
Source::connection('<key>'):
'connections' => [
'supplier-sftp' => [
'driver' => 'sftp',
'host' => env('SUPPLIER_SFTP_HOST'),
'username' => env('SUPPLIER_SFTP_USERNAME'),
'privateKey' => env('SUPPLIER_SFTP_KEY_PATH'),
'root' => '/outbound',
],
],Each entry must be an array with a non-empty driver. An ftp or sftp driver
requires its Flysystem adapter — league/flysystem-ftp or
league/flysystem-sftp-v3 — and throws at boot when it is missing. Keys beyond
driver are passed straight to the driver, exactly as in
config/filesystems.php.
staging, archive, failed
All three name a disk from config/filesystems.php and a path root within it.
failed.disk falls back to the staging disk when it is null.
See installation for what each holds, and queues and
workers for why a local
staging disk is a single-machine configuration.
queue
'queue' => [
'connection' => 'redis',
'queue' => 'imports',
],The default routing for every package job. Resolution order, most specific first:
Connection — overrides.<name>.queue.connection, then the definition's
queueConnection(), then queue.connection.
Queue — overrides.<name>.queue.queue, then the definition's queue(),
then overrides.<name>.queue when it is a bare string, then queue.queue.
overrides
Per-definition operational settings, keyed by definition name. Only four keys are accepted; anything else throws at boot, which is deliberate — behavior belongs on the class, not in config.
'overrides' => [
'products' => [
'enabled' => true,
'schedule' => '0 * * * *',
'queue' => ['connection' => 'redis', 'queue' => 'imports'],
'chunk_size' => 500,
],
],| Key | Accepts | Notes |
|---|---|---|
enabled | true / false | false removes the definition from sync:run with no name, and from scheduling |
schedule | A cron expression, or false | Wins over the definition's schedule(); false disables scheduling for it |
queue | ['connection' => …, 'queue' => …] or a queue name string | See above |
chunk_size | An integer of at least 1 | Wins over the definition's chunkSize() |
The key must name a registered definition. An override for an unknown name is a boot-time error rather than a silent no-op, which catches renamed definitions.
register_schedules
true by default. It registers sync:prune daily, plus sync:run <name> for
every enabled definition that declares a schedule() method or has a cron
override. Set it to false to register nothing and wire the commands up
yourself.
retention
Four independent windows, in days. null disables one. See retention and
pruning.
store_failed_payloads
The global switch for recording the raw payload of a failed row. It wins over a
definition's captureFailedPayloads(), so turning it off here turns it off
everywhere. See sensitive data.
tables
'tables' => [
'files' => 'data_sync_files',
'runs' => 'data_sync_runs',
'row_failures' => 'data_sync_row_failures',
],Read by both the migration and the models, so custom names work without subclassing anything.
Publish the config before the migration if you intend to change these. The migration reads the values at publish time; changing them afterwards means renaming tables by hand.
Validation
The whole file is validated when the package boots, and again by
sync:doctor. An invalid configuration throws
InvalidConfigurationException immediately rather than during a 2 a.m. run:
syncsentries exist, are the right type, and have unique names.overrideskeys name registered definitions and use only the four accepted settings, with a valid cron expression and a positive integer chunk size.retentionvalues arenullor non-negative integers.connectionsentries are arrays with a driver, and their adapters are installed.
Environment variables
The package ships no env() calls of its own — the published config is plain
values. Wire your own where they help:
'staging' => [
'disk' => env('DATA_SYNC_STAGING_DISK', 'local'),
'path' => env('DATA_SYNC_STAGING_PATH', 'data-sync/staging'),
],Remember that config:cache freezes the result, so env() outside config files
returns null in production.
What to read next
- Installation and setup for first-run configuration.
- Console commands for what reads these keys.
- Health checks for validating them on deploy.