byrcsc/laravel-data-sync · 2.x

Testing.

Test sync definitions with Laravel storage, queue, event, and database fakes.

A definition is best tested the way it runs: put a real file on a faked disk and process it in-process. Everything below uses Pest syntax and Laravel's own fakes; nothing package-specific is required.

Setting up

use Illuminate\Support\Facades\Storage;

beforeEach(function (): void {
    Storage::fake('incoming');
    Storage::fake('staging');
    Storage::fake('archive');

    config()->set('data-sync.staging.disk', 'staging');
    config()->set('data-sync.archive.disk', 'archive');
    config()->set('data-sync.syncs', [App\Syncs\ProductsSync::class]);
});

Fake the source, staging, and archive disks and point the package's staging and archive config at the fakes. Registering the definition through config in the test, rather than relying on the application's own list, keeps each test independent.

The happy path

use App\Models\Product;
use ByRcsc\LaravelDataSync\Enums\RunStatus;
use ByRcsc\LaravelDataSync\Models\SyncRun;

it('imports products from a supplier csv', function (): void {
    Storage::disk('incoming')->put('products/today.csv', <<<'CSV'
        SKU,Description,Price
        DS-1001,Anglepoise desk lamp,89.00
        DS-1002,Mechanical keyboard,149.50
        CSV);

    $this->artisan('sync:run products --now')->assertSuccessful();

    $run = SyncRun::query()->sole();

    expect($run->status)->toBe(RunStatus::Completed)
        ->and($run->rows_created)->toBe(2)
        ->and(Product::query()->count())->toBe(2)
        ->and(Product::query()->where('sku', 'DS-1001')->value('price_cents'))->toBe(8900);
});

--now keeps everything in the test process, which is what you want for assertions about rows. Calling the manager directly works the same way and gives you the report object:

use ByRcsc\LaravelDataSync\Facades\DataSync;

$report = DataSync::run('products');

expect($report->discovered)->toBe(1)
    ->and($report->runs[0]->created)->toBe(2);

Asserting failures

use ByRcsc\LaravelDataSync\Models\SyncRowFailure;

it('records a row that fails validation and keeps going', function (): void {
    Storage::disk('incoming')->put('products/today.csv', <<<'CSV'
        SKU,Description,Price
        DS-1001,Anglepoise desk lamp,89.00
        ,Mechanical keyboard,149.50
        CSV);

    DataSync::run('products');

    $run = SyncRun::query()->sole();
    $failure = SyncRowFailure::query()->sole();

    expect($run->status)->toBe(RunStatus::CompletedWithErrors)
        ->and($run->rows_created)->toBe(1)
        ->and($run->rows_failed)->toBe(1)
        ->and($failure->row_number)->toBe(3)
        ->and($failure->errors)->toHaveKey('sku')
        ->and($failure->payload)->toBe([
            'SKU' => '',
            'Description' => 'Mechanical keyboard',
            'Price' => '149.50',
        ]);
});

Row numbers count as the reader counts them, with a header, the first data row is row 2. See row numbers.

A definition-level problem, such as a missing required column or an unmapped attribute, throws instead:

use ByRcsc\LaravelDataSync\Exceptions\SchemaMismatchException;

expect(fn () => DataSync::run('products'))
    ->toThrow(SchemaMismatchException::class);

Every exception the package throws extends DataSyncException, so assert on that when you care that the run failed rather than on how. See catching package exceptions.

The checksum ledger in tests

Two traps come from file identity:

Identical content is the same file. Writing the same CSV twice in one test produces one import and one skip, not two imports. Vary a value, or pass force: true:

DataSync::run('products', force: true);

A completed checksum survives within the test's transaction. If a test asserts a re-run, assert on the skip rather than expecting a second import:

$second = DataSync::run('products');

expect($second->discovered)->toBe(0)
    ->and($second->skipped)->toBe(1);

Deferred files

minFileAge() compares against the file's modification time, so a file written by a test is always brand new:

$this->travel(5)->minutes();

DataSync::run('products');

Without travelling, a definition with minFileAge(minutes: 2) reports the file as deferred and imports nothing, which is itself worth one test.

Queue dispatch

Assert that plain sync:run queues rather than writes:

use ByRcsc\LaravelDataSync\Jobs\ProcessFileJob;
use Illuminate\Support\Facades\Queue;

it('dispatches a file job on the configured queue', function (): void {
    Queue::fake();
    config()->set('data-sync.overrides.products.queue', ['connection' => 'redis', 'queue' => 'imports']);
    Storage::disk('incoming')->put('products/today.csv', "SKU,Description,Price\nDS-1001,Anglepoise desk lamp,89.00\n");

    $this->artisan('sync:run products')->assertSuccessful();

    Queue::assertPushed(ProcessFileJob::class, fn (ProcessFileJob $job): bool =>
        $job->connection === 'redis' && $job->queue === 'imports');
});

To exercise the batched path end to end, run the queue synchronously:

config()->set('queue.default', 'sync');

Chunk jobs then execute inline and the batch callbacks finalize the run, so the same assertions as the --now path apply, with the batching code actually covered.

Events

use ByRcsc\LaravelDataSync\Events\SyncCompleted;
use Illuminate\Support\Facades\Event;

Event::fake([SyncCompleted::class]);

DataSync::run('products');

Event::assertDispatched(SyncCompleted::class, fn (SyncCompleted $event): bool =>
    $event->run->rows_created === 2 && ! $event->run->dry_run);

Remember that dry runs dispatch events too, assert on dry_run when it matters.

Dry runs

$report = DataSync::run('products', dryRun: true);

expect($report->runs[0]->created)->toBe(2)
    ->and(Product::query()->count())->toBe(0)
    ->and(Storage::disk('archive')->allFiles())->toBeEmpty();

A dry run reports what it would do, writes nothing, and leaves the checksum unprocessed, so a follow-up real run imports the same file.

Transfers

Storage::fake('outgoing');

Storage::disk('incoming')->put('statements/STMT-000431-20260731.pdf', 'binary');

DataSync::run('statements');

expect(Storage::disk('outgoing')->exists('statements/000431/2026/07/STMT-000431-20260731.pdf'))
    ->toBeTrue();

A transfer run reports 1 created, or 1 skipped when the destination already holds the same checksum or the filename did not match.

API syncs

Fake the endpoint with Http::fake() and run the definition in-process:

use App\Models\Booking;
use ByRcsc\LaravelDataSync\Facades\DataSync;
use ByRcsc\LaravelDataSync\Models\SyncWatermark;
use Illuminate\Support\Facades\Http;

it('imports bookings and advances the watermark', function (): void {
    config()->set('data-sync.syncs', [App\Syncs\BookingsSync::class]);

    Http::preventStrayRequests();
    Http::fake([
        'middleware.example.com/*' => Http::response([
            'data' => [
                ['BookingNo' => 'BK-1001', 'Status' => 'confirmed'],
                ['BookingNo' => 'BK-1002', 'Status' => 'pending'],
            ],
            'meta' => ['next_page_url' => null, 'max_updated_at' => '2026-11-09T04:00:00Z'],
        ]),
    ]);

    $report = DataSync::run('bookings');

    expect($report->runs[0]->created)->toBe(2)
        ->and(Booking::query()->count())->toBe(2)
        ->and(SyncWatermark::query()->where('definition', 'bookings')->sole()->value)
        ->toBe('2026-11-09T04:00:00Z');
});

Only the staging and archive disks need faking, because an API sync has no source disk. The sync_watermarks table must exist in the test database, so run the package migrations there as you do for the other tables.

To test a failed fetch, return an error status. The package sends a 503 retries() more times and waits retryDelay() milliseconds between attempts. Sleep::fake() skips those waits:

use Illuminate\Support\Sleep;

Sleep::fake();
Http::fake(['middleware.example.com/*' => Http::response('Unavailable', 503)]);

expect(fn () => DataSync::run('bookings'))
    ->toThrow(ByRcsc\LaravelDataSync\Exceptions\ApiFetchException::class);

Http::assertSentCount(4);   // the first attempt and 3 retries

Database syncs

Point the definition's connection at an SQLite database in the test, create the source table there, and insert the rows the source would hold:

use App\Models\Order;
use ByRcsc\LaravelDataSync\Facades\DataSync;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;

it('imports orders from the ERP', function (): void {
    config()->set('database.connections.erp', [
        'driver' => 'sqlite',
        'database' => ':memory:',
        'prefix' => '',
    ]);
    config()->set('data-sync.syncs', [App\Syncs\LegacyOrdersSync::class]);

    Schema::connection('erp')->create('Orders', function (Blueprint $table): void {
        $table->string('OrderNo')->primary();
        $table->string('Customer');
        $table->decimal('Total');
        $table->dateTime('UpdatedAt');
    });

    DB::connection('erp')->table('Orders')->insert([
        ['OrderNo' => 'SO-1', 'Customer' => 'Acme', 'Total' => 120, 'UpdatedAt' => now()],
        ['OrderNo' => 'SO-2', 'Customer' => 'Globex', 'Total' => 75, 'UpdatedAt' => now()],
    ]);

    $report = DataSync::run('legacy-orders');

    expect($report->runs[0]->created)->toBe(2)
        ->and(Order::query()->count())->toBe(2);
});

As with an API sync, fake only the staging and archive disks, and run the package migrations so that sync_watermarks exists. SQLite has no dbo schema, so a query that names dbo.Orders fails in this test. Name the table without the schema in query(). SQL Server then reads it from the login's default schema, which is usually dbo.

To test the watermark, insert rows with fixed UpdatedAt values, run the sync, give one row a later value, and run it again. The second run reads the changed row and any row within overlapSeconds of the stored watermark.

In CI

php artisan sync:doctor

Running the doctor against your CI environment catches a definition that no longer resolves, a mapped attribute that a migration removed, a matchOn() that lost its unique index, and a database sync whose source table lost a column. A unit test on one definition notices none of these. See health checks.

Testing.

Test sync definitions with Laravel storage, queue, event, and database fakes.

Setting up

use Illuminate\Support\Facades\Storage;

beforeEach(function (): void {
    Storage::fake('incoming');
    Storage::fake('staging');
    Storage::fake('archive');

    config()->set('data-sync.staging.disk', 'staging');
    config()->set('data-sync.archive.disk', 'archive');
    config()->set('data-sync.syncs', [App\Syncs\ProductsSync::class]);
});

The happy path

use App\Models\Product;
use ByRcsc\LaravelDataSync\Enums\RunStatus;
use ByRcsc\LaravelDataSync\Models\SyncRun;

it('imports products from a supplier csv', function (): void {
    Storage::disk('incoming')->put('products/today.csv', <<<'CSV'
        SKU,Description,Price
        DS-1001,Anglepoise desk lamp,89.00
        DS-1002,Mechanical keyboard,149.50
        CSV);

    $this->artisan('sync:run products --now')->assertSuccessful();

    $run = SyncRun::query()->sole();

    expect($run->status)->toBe(RunStatus::Completed)
        ->and($run->rows_created)->toBe(2)
        ->and(Product::query()->count())->toBe(2)
        ->and(Product::query()->where('sku', 'DS-1001')->value('price_cents'))->toBe(8900);
});

--now keeps everything in the test process, which is what you want for assertions about rows. Calling the manager directly works the same way and gives you the report object:

use ByRcsc\LaravelDataSync\Facades\DataSync;

$report = DataSync::run('products');

expect($report->discovered)->toBe(1)
    ->and($report->runs[0]->created)->toBe(2);

Asserting failures

use ByRcsc\LaravelDataSync\Models\SyncRowFailure;

it('records a row that fails validation and keeps going', function (): void {
    Storage::disk('incoming')->put('products/today.csv', <<<'CSV'
        SKU,Description,Price
        DS-1001,Anglepoise desk lamp,89.00
        ,Mechanical keyboard,149.50
        CSV);

    DataSync::run('products');

    $run = SyncRun::query()->sole();
    $failure = SyncRowFailure::query()->sole();

    expect($run->status)->toBe(RunStatus::CompletedWithErrors)
        ->and($run->rows_created)->toBe(1)
        ->and($run->rows_failed)->toBe(1)
        ->and($failure->row_number)->toBe(3)
        ->and($failure->errors)->toHaveKey('sku')
        ->and($failure->payload)->toBe([
            'SKU' => '',
            'Description' => 'Mechanical keyboard',
            'Price' => '149.50',
        ]);
});

Row numbers count as the reader counts them, with a header, the first data row is row 2. See row numbers.

A definition-level problem, such as a missing required column or an unmapped attribute, throws instead:

use ByRcsc\LaravelDataSync\Exceptions\SchemaMismatchException;

expect(fn () => DataSync::run('products'))
    ->toThrow(SchemaMismatchException::class);

Every exception the package throws extends DataSyncException, so assert on that when you care that the run failed rather than on how. See catching package exceptions.

The checksum ledger in tests

Two traps come from file identity:

Identical content is the same file. Writing the same CSV twice in one test produces one import and one skip, not two imports. Vary a value, or pass force: true:

DataSync::run('products', force: true);

A completed checksum survives within the test's transaction. If a test asserts a re-run, assert on the skip rather than expecting a second import:

$second = DataSync::run('products');

expect($second->discovered)->toBe(0)
    ->and($second->skipped)->toBe(1);

Deferred files

minFileAge() compares against the file's modification time, so a file written by a test is always brand new:

$this->travel(5)->minutes();

DataSync::run('products');

Without travelling, a definition with minFileAge(minutes: 2) reports the file as deferred and imports nothing, which is itself worth one test.

Queue dispatch

Assert that plain sync:run queues rather than writes:

use ByRcsc\LaravelDataSync\Jobs\ProcessFileJob;
use Illuminate\Support\Facades\Queue;

it('dispatches a file job on the configured queue', function (): void {
    Queue::fake();
    config()->set('data-sync.overrides.products.queue', ['connection' => 'redis', 'queue' => 'imports']);
    Storage::disk('incoming')->put('products/today.csv', "SKU,Description,Price\nDS-1001,Anglepoise desk lamp,89.00\n");

    $this->artisan('sync:run products')->assertSuccessful();

    Queue::assertPushed(ProcessFileJob::class, fn (ProcessFileJob $job): bool =>
        $job->connection === 'redis' && $job->queue === 'imports');
});

To exercise the batched path end to end, run the queue synchronously:

config()->set('queue.default', 'sync');

Chunk jobs then execute inline and the batch callbacks finalize the run, so the same assertions as the --now path apply, with the batching code actually covered.

Events

use ByRcsc\LaravelDataSync\Events\SyncCompleted;
use Illuminate\Support\Facades\Event;

Event::fake([SyncCompleted::class]);

DataSync::run('products');

Event::assertDispatched(SyncCompleted::class, fn (SyncCompleted $event): bool =>
    $event->run->rows_created === 2 && ! $event->run->dry_run);

Remember that dry runs dispatch events too, assert on dry_run when it matters.

Dry runs

$report = DataSync::run('products', dryRun: true);

expect($report->runs[0]->created)->toBe(2)
    ->and(Product::query()->count())->toBe(0)
    ->and(Storage::disk('archive')->allFiles())->toBeEmpty();

A dry run reports what it would do, writes nothing, and leaves the checksum unprocessed, so a follow-up real run imports the same file.

Transfers

Storage::fake('outgoing');

Storage::disk('incoming')->put('statements/STMT-000431-20260731.pdf', 'binary');

DataSync::run('statements');

expect(Storage::disk('outgoing')->exists('statements/000431/2026/07/STMT-000431-20260731.pdf'))
    ->toBeTrue();

A transfer run reports 1 created, or 1 skipped when the destination already holds the same checksum or the filename did not match.

API syncs

Fake the endpoint with Http::fake() and run the definition in-process:

use App\Models\Booking;
use ByRcsc\LaravelDataSync\Facades\DataSync;
use ByRcsc\LaravelDataSync\Models\SyncWatermark;
use Illuminate\Support\Facades\Http;

it('imports bookings and advances the watermark', function (): void {
    config()->set('data-sync.syncs', [App\Syncs\BookingsSync::class]);

    Http::preventStrayRequests();
    Http::fake([
        'middleware.example.com/*' => Http::response([
            'data' => [
                ['BookingNo' => 'BK-1001', 'Status' => 'confirmed'],
                ['BookingNo' => 'BK-1002', 'Status' => 'pending'],
            ],
            'meta' => ['next_page_url' => null, 'max_updated_at' => '2026-11-09T04:00:00Z'],
        ]),
    ]);

    $report = DataSync::run('bookings');

    expect($report->runs[0]->created)->toBe(2)
        ->and(Booking::query()->count())->toBe(2)
        ->and(SyncWatermark::query()->where('definition', 'bookings')->sole()->value)
        ->toBe('2026-11-09T04:00:00Z');
});

To test a failed fetch, return an error status. The package sends a 503 retries() more times and waits retryDelay() milliseconds between attempts. Sleep::fake() skips those waits:

use Illuminate\Support\Sleep;

Sleep::fake();
Http::fake(['middleware.example.com/*' => Http::response('Unavailable', 503)]);

expect(fn () => DataSync::run('bookings'))
    ->toThrow(ByRcsc\LaravelDataSync\Exceptions\ApiFetchException::class);

Http::assertSentCount(4);   // the first attempt and 3 retries

Database syncs

Point the definition's connection at an SQLite database in the test, create the source table there, and insert the rows the source would hold:

use App\Models\Order;
use ByRcsc\LaravelDataSync\Facades\DataSync;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;

it('imports orders from the ERP', function (): void {
    config()->set('database.connections.erp', [
        'driver' => 'sqlite',
        'database' => ':memory:',
        'prefix' => '',
    ]);
    config()->set('data-sync.syncs', [App\Syncs\LegacyOrdersSync::class]);

    Schema::connection('erp')->create('Orders', function (Blueprint $table): void {
        $table->string('OrderNo')->primary();
        $table->string('Customer');
        $table->decimal('Total');
        $table->dateTime('UpdatedAt');
    });

    DB::connection('erp')->table('Orders')->insert([
        ['OrderNo' => 'SO-1', 'Customer' => 'Acme', 'Total' => 120, 'UpdatedAt' => now()],
        ['OrderNo' => 'SO-2', 'Customer' => 'Globex', 'Total' => 75, 'UpdatedAt' => now()],
    ]);

    $report = DataSync::run('legacy-orders');

    expect($report->runs[0]->created)->toBe(2)
        ->and(Order::query()->count())->toBe(2);
});

In CI

php artisan sync:doctor

Testing.

Setting up

The happy path

Asserting failures

The checksum ledger in tests

Deferred files

Queue dispatch

Events

Dry runs

Transfers

API syncs

Database syncs

In CI

What to read next

Testing.

Setting up

The happy path

Asserting failures

The checksum ledger in tests

Deferred files

Queue dispatch

Events

Dry runs

Transfers

API syncs

Database syncs

In CI

What to read next