byrcsc/laravel-data-sync · 1.x
Running a sync.
Run definitions from the console or from code, choose between the queued and in-process paths, force a reprocess, preview with a dry run, and understand the discovery lock.
php artisan sync:run # every enabled definition
php artisan sync:run products # one definition
php artisan sync:run products --now # process in this process, no queue
php artisan sync:run products --force
php artisan sync:run products --dry-runWith no name, every enabled definition runs in registration order. A
definition disabled through data-sync.overrides.<name>.enabled is skipped; one
named explicitly runs regardless.
What a run does
- Takes a cache lock for the definition and lists the source directory.
- Defers files younger than
minFileAge(), and skips files already recorded as completed. - Streams each remaining file to the staging disk and checksums it.
- Records a run per file.
- Processes it — queued by default, in this process with
--now.
The command prints one line per run plus a summary:
Running [products]...
File run [01JQ8Z3M0K2N4P6R8T0V2X4Y6A] (bulk) ... 2 created, 0 updated, 0 skipped, 0 failed
[products] finished: 1 discovered, 0 skipped, 0 deferred.discovered counts files that produced a run, skipped counts checksums
already completed, and deferred counts files held back by minFileAge().
On the queued path the per-run counters are the state at dispatch — usually all
zeros — because nothing has been written yet. Use
sync:status for the outcome.
Queued by default
sync:run records a run per discovered file and dispatches a job for each.
That job plans the file, then fans its chunks out through Bus::batch() so
several workers can share one file. Nothing is written until a worker picks the
jobs up.
The queued path needs a worker and the job_batches table. See queues and
workers.
--now
--now skips the queue and does everything in the command: discovery, reading,
mapping, writing, archiving. It is the right choice for small feeds, for local
development, for one-off backfills, and for anywhere without a worker.
The two paths write identically. What differs is parallelism, and where an error surfaces: an in-process run rethrows, so the command prints the message and exits non-zero, while a queued run records the failure on the run and fails the job.
--force
--force processes a checksum that has already completed. Without it, discovery
skips such files twice over: once by a path, size, and modification-time match
that avoids reading the file at all, and again after checksumming.
A forced run creates a new run rather than rewriting the old one, so the
history shows both attempts. It does not delete or reconcile anything that the
first run wrote — for an Upsert definition it simply writes the same rows
again.
--dry-run
php artisan sync:run products --dry-runA dry run validates and counts without writing models, running write hooks,
archiving the file, cleaning up the source, or marking the checksum processed.
It always runs in-process; it never queues, even without --now.
The run is still recorded, and sync:status marks it as a dry run. Because
nothing is marked processed, the same file is discovered again on the next run.
Two caveats:
- Dry runs skip
prepare(), so a definition that reshapes rows there can report slightly different numbers than the real run. - Dry runs still dispatch events. A
SyncCompletedlistener that notifies people should check$event->run->dry_run.
The discovery lock
Discovery takes a cache lock per definition for five minutes, so two overlapping
sync:run invocations do not stage or process the same file twice. The second
one reports that discovery is already running and exits successfully:
Skipped [products] because discovery is already running.Exiting zero is deliberate: an overlapping scheduled run is a normal condition, not a failure worth paging anyone about. The lock covers discovery only — once files are dispatched, processing runs as long as it needs to.
The lock lives in the cache, so the cache store must be one that supports atomic
locks. sync:doctor verifies it.
Running from code
use ByRcsc\LaravelDataSync\Facades\DataSync;
$report = DataSync::run('products'); // in-process, like --now
$report = DataSync::run('products', force: true);
$report = DataSync::run('products', dryRun: true);
$report = DataSync::queue('products'); // discover and dispatch, like plain sync:run
$report = DataSync::queue('products', force: true);run() and queue() are the two paths the command exposes as --now and the
default. Both return a RunReport:
$report->definition; // 'products'
$report->discovered; // files that produced a run
$report->skipped; // checksums already completed
$report->deferred; // files held back by minFileAge()
$report->locked; // true when discovery was already running
$report->runs; // list<SyncReport>, one per fileEach SyncReport carries the SyncRun model, its status and write path, and
the total, created, updated, skipped, and failed counters. On the
queued path those counters are the state at dispatch, so a queue() report
tells you what was discovered, not what was written.
Three more methods read the registry:
DataSync::registered(); // ['products' => ProductsSync, ...]
DataSync::enabled(); // minus definitions disabled in config
DataSync::definition('products');registered() validates every definition as it resolves, so calling it is also
a cheap way to assert your configuration is sound. definition() throws an
InvalidDefinitionException for a name that is not registered.
Those five methods are the whole programmatic surface. The facade resolves
ByRcsc\LaravelDataSync\SyncManager, so app(SyncManager::class) works
identically where you would rather inject it.
What to read next
- Queues and workers for batching, routing, and multi-server deployments.
- Scheduling for running definitions unattended.
- Run history and status for reading the outcome.