NDJSONLoader

Streaming loader for NDJSON encoded files and related formats (LDJSON and JSONL).

Loader	Characteristic
File Extension	.ndjson, .jsonl, .ldjson
Media Type	application/x-ndjson, application/x-ldjson, application/json-seq
File Type	Text
File Format	NDJSON, [LDJSON][format_], [][format_]
Data Format	Classic Table
Supported APIs	load, parse, parseSync, parseInBatches

Usage

import {NDJSONLoader} from '@loaders.gl/json';
import {load} from '@loaders.gl/core';

const data = await load(url, NDJSONLoader, {ndjson: options});

The NDJSONLoader supports streaming NDJSON parsing, in which case it will yield "batches" of rows, where each row is a parsed line from the NDJSON stream.

import {NDJSONLoader} from '@loaders.gl/json';
import {loadInBatches} from '@loaders.gl/core';

const batches = await loadInBatches('ndjson.ndjson', NDJSONLoader);

for await (const batch of batches) {
  // batch.data will contain a number of rows
  for (const obj of batch.data) {
    // Process obj
    ...
  }
}

Data Format

Parsed batches are of the format.

{
  // standard batch payload
  data: any[] | any;
  bytesUsed: number;
  batchCount: number;
}

Each element in the data array corresponds to a line (Object) in the NDJSON data.

Options

Supports the table category options such as batchSize.