Definition providers.

A definition class in data-sync.syncs registers one definition. On a multi-tenant platform, the sources are data instead. Each account connects its own databases, and a new account must not need a deploy.

A definition provider returns definitions at runtime. Your application decides what an account and a source are. The provider turns each source into a definition, and the package runs each one like a class listed in config:

namespace App\Syncs;

use App\Models\SourceDatabase;
use ByRcsc\LaravelDataSync\Contracts\DefinitionProvider;

final class AccountSources implements DefinitionProvider
{
    public function definitions(): iterable
    {
        foreach (SourceDatabase::query()->where('active', true)->cursor() as $source) {
            config()->set(
                "database.connections.{$source->connectionName()}",
                $source->connectionConfig(),
            );

            yield new AccountOrdersSync($source);
        }
    }
}

List the provider in config/data-sync.php, next to or instead of definition classes:

'syncs' => [
    App\Syncs\ProductsSync::class,
    App\Syncs\AccountSources::class,
],

The package has no notion of tenants. SourceDatabase in this example is your own model, and connectionName() and connectionConfig() are your own methods.

Write the definition

A provider can return any definition: a file sync, an API sync, a database sync, or a transfer. Pass the source to the constructor and build each method from it:

use ByRcsc\LaravelDataSync\Definitions\DatabaseSyncDefinition;
use ByRcsc\LaravelDataSync\Definitions\Field;

final class AccountOrdersSync extends DatabaseSyncDefinition
{
    public function __construct(private readonly SourceDatabase $source) {}

    public function name(): string
    {
        return "orders-{$this->source->account_id}-{$this->source->id}";
    }

    public function connection(): string
    {
        return $this->source->connectionName();
    }

    public function fields(): array
    {
        return [
            Field::make('account_id', from: 'account_id')->default($this->source->account_id),
            Field::make('order_no', from: 'OrderNo'),
            // ...
        ];
    }

    public function matchOn(): array
    {
        return ['account_id', 'order_no'];
    }

    // query(), keyColumns(), model(), and the rest as in any database definition
}

Name each definition from IDs

name() keys the definition's watermark, ledger entries, fetch lock, staging directory, and run history. Two definitions with different names share none of them, so two accounts never read each other's progress.

Return the same name for the same source on every call. A changed name starts over with no watermark, so the next run reads every row again.

A name that a provider returns may use only letters, digits, hyphens, and underscores, and must start with a letter or digit. Build it from IDs, not from a label that an account types. A dot is not allowed, because it would split a data-sync.overrides key. Two definitions with the same name throw an InvalidConfigurationException.

Keep accounts apart

Rows from every account land in the same model table. Include the account in matchOn() and in a unique index on those columns. Otherwise one account's upsert overwrites another account's row with the same order number.

Field::make('account_id', from: 'account_id')->default(...) sets the account on every row, because the source table has no account_id column. See defaults.

Register the connection

Register each source's connection inside definitions(), as in the first example. The package calls definitions() in every process that resolves definitions, so a queue worker that processes a chunk registers the connection too. A connection that you register only in a service provider or a request is missing from those processes, and the definition fails validation.

Your application stores the credentials. The package reads only the connection name. Give every source a read-only login, and check the host that an account enters before you connect to it.

When the package calls the provider

The package calls definitions() whenever it resolves definitions:

• in every sync:* command;
• in a queue worker, when it processes a job for one of the definitions;
• in the scheduler, when schedule:run, schedule:work, or schedule:list resolves Laravel's Schedule.

It never calls definitions() while the application boots. An HTTP request or a first migrate therefore never reads your tables through the provider, and a provider can read tables that a pending migration creates.

At boot, the package checks only that the provider class implements Contracts\DefinitionProvider. It checks each definition the provider returns, and any override keyed by its name, when the definitions are first resolved.

Schedule many sources

Every definition with a schedule() method gets its own scheduled sync:run. The scheduler runs due commands one after another, and a database fetch reads its rows inside the command. With many sources, run each one in the background:

public function schedule(Event $event): void
{
    $event->everyFifteenMinutes()->runInBackground();
}

data-sync.overrides accepts provider names like any other name, and sync:doctor checks every definition a provider returns.

What to read next

• Syncing from a database for the methods a database definition adds.
• Scheduling for how the package registers each definition's sync:run.
• Configuration for the rules on entries in data-sync.syncs.