File formats and readers.

format() returns the reader for the definition's files. Readers are built from the Reader factory:

use ByRcsc\LaravelDataSync\Contracts\Reader as ReaderContract;
use ByRcsc\LaravelDataSync\Definitions\Reader;

public function format(): ReaderContract
{
    return Reader::csv()->delimiter('|')->encoding('ISO-8859-1');
}

Every reader produces the same thing: a stream of rows, each an array keyed by column name — or by zero-based index when there is no header — with every value a string or null. Type conversion is the mapping layer's job, through transform() and model casts.

CSV and delimited text

Reader::csv()
    ->delimiter(';')
    ->enclosure('"')
    ->escape('\\')
    ->encoding('ISO-8859-1')
    ->withoutHeader();

A UTF-8 byte-order mark on the first column is stripped, so a Windows-exported CSV does not silently produce a \xEF\xBB\xBFSKU column.

Non-UTF-8 input is converted with //IGNORE, so undecodable bytes are dropped rather than throwing. If a feed regularly hits that path, fix the encoding at the source.

XLSX

Reader::xlsx()->withHeader();

Only the first sheet is read; other sheets are ignored without warning. There is no cell-range or named-table selection.

Cell values are normalized to strings before mapping: dates and times become ISO-8601 (DATE_ATOM) strings, booleans become "true" and "false", numbers become their string form, and anything else is JSON-encoded. A date cell therefore arrives as 2026-07-31T00:00:00+00:00, which a datetime cast on the model handles directly.

Because the XLSX format is a zip archive, the reader copies the staged file to a local temporary file before opening it. Rows still stream from there, but a workbook needs local scratch space the size of the file.

JSON

Reader::json()->path('data.products');

path() is a dot path to the array of rows inside the document. Omit it when the document is itself an array of objects. The target must be a list of objects; anything else throws, naming the path.

JSON does not stream. Reader::json() reads the entire document into memory and then selects the row array. Multi-gigabyte single-document JSON is out of scope — use NDJSON when rows need to stream independently.

Values are normalized the same way as XLSX cells, so a nested object or array arrives as a JSON string. Map it with a transform() if you need it structured again.

NDJSON

Reader::ndjson();

One JSON object per line, no configuration. Blank lines are skipped. A line that is not valid JSON, or is valid JSON but not an object, throws with its line number.

Rows stream, but note that determining the file's columns reads every line: the header is the union of the keys seen across the whole file. That is what allows a feed whose later rows carry extra keys to validate correctly, at the cost of one extra pass.

Fixed width

Reader::fixedWidth()
    ->column('sku', start: 0, length: 12)
    ->column('name', start: 12, length: 40)
    ->column('price', start: 52, length: 10);

Positions are zero-based byte offsets and lengths. Values are trimmed. There is no header row — the columns you declare are the header, so map fields with Field::make('sku', from: 'sku') using the same names.

A line shorter than the highest start + length throws with its line number. Trailing \r\n is stripped before slicing.

Headers and column addressing

With a header row, address columns by name:

Field::make('sku', from: 'SKU');

Without one — withoutHeader() on the CSV or XLSX reader — address them by zero-based index:

Field::make('sku', from: 0);

Column names are matched exactly, including case and surrounding whitespace. A feed with " SKU" needs from: ' SKU', or a producer who trims their export.

Row numbers

Row numbers appear in failure records and in sync:status --failures, so it helps to know what they count:

Streaming, memory, and chunk size

CSV, XLSX, NDJSON, and fixed-width files stream row by row. Memory is bounded by chunkSize() — the number of rows held for one transaction — not by file size. Reader::json() is the exception noted above.

Queued runs read the staged file more than once: once to plan the file and resolve duplicates, then once per chunk job, which skips forward to its offset. That trades repeated sequential reads of a local staging file for the ability to process one file across several workers. For a very large file where that matters, a bigger chunkSize() means fewer, longer passes.

Custom readers

format() is typed against the ByRcsc\LaravelDataSync\Contracts\Reader interface, which is two methods:

public function header(mixed $stream): array;   // list<string|int>
public function rows(mixed $stream): Generator;  // rowNumber => array<string|int, string|null>

Implementing it against a fixed-format feed the shipped readers do not cover — an EDI segment layout, say — is supported, and the two-method shape is part of the 1.x contract. format() accepts any implementation, so a custom reader needs no registration.

header() receives the open stream and returns the column names, and rows() yields rowNumber => row with every value a string or null — the same normalization the shipped readers apply, and what the mapping layer expects.

What to read next

• Field mapping for turning columns into model attributes.
• Matching and writing for chunking and write policies.