›
›
›
  1. docs
  2. ›
  3. byrcsc/laravel-data-sync
1.x
Browse documentationOpenClose

Getting started

  • Introduction
  • Installation and setup
  • Quick start

Core concepts

  • File identity and runs
  • Sync definitions
  • Sources and connections
  • File formats and readers
  • Field mapping
  • Matching and writing
  • Validation and hooks

Running syncs

  • Running a sync
  • Queues and workers
  • Scheduling

Transfers

  • File transfers

Operations

  • Run history and status
  • Events and listeners
  • Failures and retries
  • Sensitive data
  • Health checks
  • Retention and pruning

Reference

  • Configuration
  • Console commands
  • Testing
  • Troubleshooting

Getting started

  • Introduction
  • Installation and setup
  • Quick start

Core concepts

  • File identity and runs
  • Sync definitions
  • Sources and connections
  • File formats and readers
  • Field mapping
  • Matching and writing
  • Validation and hooks

Running syncs

  • Running a sync
  • Queues and workers
  • Scheduling

Transfers

  • File transfers

Operations

  • Run history and status
  • Events and listeners
  • Failures and retries
  • Sensitive data
  • Health checks
  • Retention and pruning

Reference

  • Configuration
  • Console commands
  • Testing
  • Troubleshooting

byrcsc/laravel-data-sync · 1.x

Validation and hooks.

Validate, prepare, skip, or customize model writes for each imported row.

Four extension points sit between a mapped row and the database. In order: prepare(), rules() with withValidator(), then beforeSave() and afterSave() on the Eloquent write path.

Preparing a row

/**
 * @param  array<string, mixed>  $row
 * @return array<string, mixed>
 */
public function prepare(array $row): array
{
    $row['name'] = trim($row['first_name'].' '.$row['last_name']);
    $row['sku'] = strtoupper($row['sku']);

    return $row;
}

prepare() receives the mapped row, keyed by model attribute, and returns the row that will be validated and written. Use it for work that spans more than one field, which a per-field transform() cannot express.

Two things to know:

  • prepare() runs before duplicate resolution, so a prepare() that changes a match column changes which rows count as duplicates.
  • Dry runs skip it. --dry-run maps and validates without calling prepare(), so a definition that reshapes rows there can report slightly different counts than the real run. Treat dry-run numbers as a preview.

Like transforms, prepare() runs on every row on every pass. Keep it pure.

Validating rows

public function rules(): array
{
    return [
        'sku' => ['required', 'string', 'max:32'],
        'price_cents' => ['required', 'integer', 'min:0'],
        'currency' => ['required', 'in:PHP,USD'],
    ];
}

Rules are ordinary Laravel validation rules, applied per row against the mapped attributes, so key them by attribute name, not by source column.

A row that fails validation is recorded in sync_row_failures with its errors and its payload, counted as failed, and skipped. The run continues. Only maxErrors() stops a run part way through.

Validation is per row, so unique rules work but cost a query each, and rules that reference other rows in the file do not apply. Uniqueness within the file is onDuplicate()'s job.

Extending the validator

use Illuminate\Contracts\Validation\Validator;

public function withValidator(Validator $validator): void
{
    $validator->after(function (Validator $validator): void {
        $data = $validator->getData();

        if (($data['price_cents'] ?? 0) > 1_000_000 && ($data['approved'] ?? '') !== 'Y') {
            $validator->errors()->add('price_cents', 'High-value rows need approval.');
        }
    });
}

withValidator() is called with the validator for each row, before it runs. Use it for conditional checks that rules() cannot express. It is invoked per row, so avoid per-call setup work that could be hoisted into the definition's constructor.

Write hooks

use Illuminate\Database\Eloquent\Model;

/**
 * @param  array<string, mixed>  $row
 */
public function beforeSave(Model $model, array $row): mixed
{
    if ($row['status'] === 'archived') {
        return false;   // skip this row without saving
    }

    $model->imported_at = now();

    return null;
}

/**
 * @param  array<string, mixed>  $row
 */
public function afterSave(Model $model, array $row): void
{
    $model->categories()->syncWithoutDetaching(
        Category::whereIn('code', explode('|', $row['category_codes']))->pluck('id'),
    );
}

beforeSave() runs on a model that has already been resolved by matchOn() and filled with the row, immediately before save(). Returning false skips the row, it is counted as skipped, not failed. Any other return value, including null, proceeds.

afterSave() runs on the saved model, inside the chunk's transaction. Work that must not roll back with the chunk, a webhook, an email, belongs in a SyncCompleted listener instead.

Declaring either hook changes how every row in the definition is written. The definition switches from a bulk upsert to per-row Eloquent writes: model events fire, observers apply, and throughput drops. See the write path tradeoff before adding one.

Both hooks run inside the chunk transaction, so a throw rolls the chunk back rather than failing the single row. Guard against exceptions you can foresee and let genuinely broken data fail validation earlier.

Order of operations

For one row on the Eloquent path:

  1. The reader yields raw values.
  2. Field mapping applies from, transform(), defaults, and capture.
  3. prepare() reshapes the row. (Skipped on dry runs.)
  4. Duplicate resolution marks losing rows as skipped.
  5. rules() and withValidator() validate. Failures are recorded; the run continues.
  6. The model is resolved with firstOrNew() and the write mode is applied.
  7. beforeSave() runs; false skips the row.
  8. save() fires model events.
  9. afterSave() runs.

On the bulk path, steps 6 to 9 collapse into one upsert() per chunk with no model events.

What to read next

  • Matching and writing for write modes and the two paths.
  • Failures and retries for what happens to a row that fails validation.
  • Events and listeners for run-level follow-up work.
PreviousMatching and writingNextRunning a sync
View source

On this page

  1. Preparing a row
  2. Validating rows
  3. Extending the validator
  4. Write hooks
  5. Order of operations
  6. What to read next