Browse documentationOpen

byrcsc/laravel-data-sync · 1.x

Troubleshooting.

What the package's error messages mean, why a file was not processed, why rows are missing, and how to recover from each.

Catching package exceptions

Every exception the package throws extends ByRcsc\LaravelDataSync\Exceptions\DataSyncException, which extends RuntimeException. One catch covers the lot:

use ByRcsc\LaravelDataSync\Exceptions\DataSyncException;

try {
    DataSync::run('products');
} catch (DataSyncException $exception) {
    // every failure below, and nothing from outside the package
}
ExceptionThrown when
InvalidConfigurationExceptionconfig/data-sync.php is wrong — at boot, before anything runs
InvalidDefinitionExceptionA definition fails validation, or a name is not registered
SchemaMismatchExceptionA file is missing required columns, or carries unknown ones under Strict
DuplicateRowExceptionTwo rows share an identity and onDuplicate() is Fail
MaxErrorsExceededExceptionThe failed-row threshold was reached and the run aborted

The first two are configuration problems: catching them at run time is usually the wrong move, because sync:doctor would have caught them at deploy time. The last three are per-file conditions worth handling in a listener or a wrapper command.

DataSyncException does not cover I/O. Failures that are neither configuration nor data — an unreadable staged file, a destination template that will not render, a checksum that cannot be computed — throw a plain RuntimeException, and an unreachable disk surfaces as whatever Flysystem or Laravel threw. Since DataSyncException extends RuntimeException, catching RuntimeException covers both; catch DataSyncException when you specifically want the package's own conditions.

Once a run exists, anything thrown while processing it is recorded on the run — status failed, message in error_summary, staged file preserved — before the exception is rethrown. So a failure that reached a file is readable in sync:status whether or not anything caught it. Configuration and definition errors throw before a run is recorded, which is why they only ever show up in the command output and in sync:doctor.

Nothing was processed

"No enabled sync definitions are registered." data-sync.syncs is empty, or every definition is disabled through overrides.<name>.enabled => false. Name one explicitly to run it anyway.

"Skipped [products] because discovery is already running." Another sync:run holds the definition's five-minute discovery lock. This exits zero on purpose. If it persists with nothing running, the lock's cache store is misbehaving — sync:doctor verifies it.

The summary reports 0 discovered, 1 skipped. The checksum has already completed. That is the ledger working. Change the file, or use --force. See file identity.

The summary reports files as deferred. minFileAge() is holding them back until they are old enough. Expected while a supplier is still uploading; unexpected if the source's clock is far ahead of yours.

The summary reports 0 discovered, 0 skipped, 0 deferred. The source directory is empty from the package's point of view. Check the disk root, the path(), and — for a Source::connection() — whether root in the connection config already includes part of the path.

Definition and configuration errors

These all throw at boot or on the first resolve, so they surface in sync:doctor too.

"[…] is missing required source columns […]. Actual header: […]" A field that is neither optional() nor default() names a column the file does not have. The message prints the header actually found — usually a renamed column, a stray BOM, or a wrong delimiter making one giant column.

"[…] contains unknown source columns […]" onUnknownColumns() is Strict and the file grew a column. Map it, mark the policy Ignore, or capture it.

"[…] fields() maps […], which is not a column or writable attribute on the model returned by model()." A typo, or a migration that has not run. The check is skipped when the table does not exist, so this appears once the table is present.

"[…] matchOn() references […], but fields() does not map it." Match columns must be mapped fields. Add a Field for it, even if the value is only used for matching.

"[…] captureUnmappedInto() targets […], which must have an array-compatible Eloquent cast." Cast the attribute to array, json, object, collection, encrypted:array, or encrypted:collection.

"No sync definition named […] is registered in [data-sync.syncs]." The name is derived from the class — ProductsSync becomes products. Check name() and the syncs list.

"The [data-sync.overrides.…] config does not match a registered sync definition." An override keyed on a name that no longer exists, usually after renaming a definition.

"The [data-sync.overrides.….…] config is not allowed." Overrides accept only enabled, schedule, queue, and chunk_size. Everything else belongs on the definition class.

"The [data-sync.connections.…] connection uses [sftp], which requires composer require league/flysystem-sftp-v3." Install the adapter. FTP needs league/flysystem-ftp.

Rows are missing or wrong

Rows are counted as skipped. Three causes, in rough order of likelihood: the write mode declined them (CreateOnly on existing rows, UpdateOnly on new ones); they lost duplicate resolution inside the file; or a beforeSave() hook returned false.

"[…] found duplicate identity […] on rows 12 and 84; onDuplicate() is set to fail." Two rows share the same matchOn() values. Switch to LastWins or FirstWins if the feed legitimately repeats keys.

Values arrive as strings. Readers produce strings and nulls by design. Convert with transform() or with model casts.

Dates from an XLSX file look like 2026-07-31T00:00:00+00:00. Cell values are normalized to ISO-8601 strings before mapping. A datetime cast on the model handles them directly.

Only part of an XLSX workbook was imported. Only the first sheet is read. Nothing else is looked at.

Model observers did not fire. The definition is on the bulk write path. Declare beforeSave() or afterSave() to switch to Eloquent writes, or move the work into a SyncCompleted listener. sync:status prints which path each run took.

A default was not applied. default() applies when the row has no entry for the column at all. A blank CSV cell is an empty string, and a short delimited row yields null — both are values.

"[…] stopped after exceeding maxErrors() […]" The failed-row threshold was reached and the run aborted; on the queued path the batch is cancelled too. Fix the feed and retry the run.

Runs that fail or hang

A run sits at running with no batch progress. Usually a dead worker. Restart it; the batch resumes. If the batch is gone, retry the run. sync:doctor warns about runs running for more than an hour.

"Unable to read staged file […]" The worker cannot see the staged copy. On a multi-server deployment this is almost always a local staging disk — point staging, archive, and failed at shared storage. See queues and workers.

"Archive checksum verification failed for run […]" The copy written to the archive disk does not match the source bytes. The copy is deleted and the run fails, and — importantly — the source file is left alone. Check the archive disk for truncation, quota, or a proxy rewriting content.

The run completed but the supplier's file is still there. Source cleanup is opt-in: archiveTo(), moveSourceTo(), or deleteFromSource(). If one is configured and the file remains, look at sync_runs.warnings — a cleanup failure is recorded there rather than failing an otherwise successful import.

The batch was cancelled. Either maxErrors() tripped it or something cancelled it externally. The run is cancelled, the staged file is preserved, and a retry restores from the failed disk.

Retries

"Run […] has not finished and cannot be retried." Only terminal runs can be retried. Wait for it, or cancel its batch.

"Run […] has no archived or failed-file copy to retry." The run never got far enough to preserve a copy, or its archive_path is empty. Re-deliver the file instead.

"No run or archived file matching […] was found." The run was pruned and no stored file matches the reference. See retention and pruning.

"The archived file […] for this run no longer exists." The run outlived its file. runs_days is longer than archive_days by design — widen archive_days if retries this old need to work.

--failed-rows refuses to run. Either payload capture was off for the original run — see sensitive data — or the run is a transfer, which has no rows. Retry the whole file.

Transfers

"Filename […] does not match […]" Recorded as a skipped run under the default UnmatchedFilenamePolicy::Skip, and the checksum is marked processed so the file is not seen again. Use Fail if a stray filename should be an error.

"destination()->path() references unknown value […]" The path template names a placeholder that is neither a captured segment nor one of the built-in values. See destination paths.

"matchFilename() pattern […] must contain at least one {segment}." A pattern needs at least one placeholder; a fully literal filename is not a pattern.

"Destination file […] already exists with a different checksum." ExistingFilePolicy::Fail is in force and something else claims that path. Either the path template is not unique enough, or the supplier re-issued a document under the same name.

Disks and growth

The staging disk keeps growing. Successful runs leave their staged copy behind and sync:prune does not sweep staging. Add your own cleanup — see retention and pruning.

sync:doctor --strict always fails on one machine. The local-staging-disk warning is expected on a single-server deployment. Use plain sync:doctor there, or move staging to shared storage.

Memory spikes on a JSON feed. Reader::json() loads the whole document. Use NDJSON for anything large. See readers.

Still stuck

Collect the run ULID, the output of sync:status <name> --failures, and the output of sync:doctor. Between them they identify the definition, the file, the write path, the counters, and the environment — which is most of what any diagnosis needs.