JSONLoader

Streaming loader for JSON encoded files.

Loader	Characteristic
File Extension	.json
Media Type	application/json
File Type	Text
File Format	JSON
Data Format	Classic Table
Supported APIs	load, parse, parseSync, parseInBatches

Usage

For simple usage, you can load and parse a JSON file atomically:

import {JSONLoader} from '@loaders.gl/json';
import {load} from '@loaders.gl/core';

const data = await load(url, JSONLoader, {json: options});

For larger files, JSONLoader supports streaming JSON parsing, in which case it will yield "batches" of rows from one array. To parse a stream of GeoJSON, the user can specify the options.json.jsonpaths to stream the features array.

import {JSONLoader} from '@loaders.gl/json';
import {loadInBatches} from '@loaders.gl/core';

const batches = await loadInBatches('geojson.json', JSONLoader, {json: {jsonpaths: ['$.features']}});

for await (const batch of batches) {
  // batch.data will contain a number of rows
  for (const feature of batch.data) {
    switch (feature.geometry.type) {
      case 'Polygon':
      ...
    }
  }
}

If no JSONPath is specified the loader will stream the first array it encounters in the JSON payload.

When batch parsing an embedded JSON array as a table, it is possible to get access to the containing object supplying the {metadata: true} option.

The loader will yield an initial and a final batch with batch.container providing the container object and batch.batchType set to partial-result and final-result respectively.

import {JSONLoader} from '@loaders.gl/json';
import {loadInBatches} from '@loaders.gl/core';

const batches = await loadInBatches('geojson.json', JSONLoader);

for await (const batch of batches) {
  switch (batch.batchType) {
    case 'partial-result': // contains fields seen so far
    case 'final-result': // contains all fields except the streamed array
      console.log(batch.container);
      break;
    case 'data':
      // batch.data will contain a number of rows
      for (const feature of batch.data) {
        switch (feature.geometry.type) {
          case 'Polygon':
          ...
        }
      }
  }
}

Streaming semantics and avoiding truncated arrays

• JSONLoader streams rows from a single JSON array. Every batch.data entry in a data batch is a complete row that the streaming parser has fully parsed before it is emitted.
• If {metadata: true} is set, the loader also yields partial-result and final-result batches that intentionally exclude the streamed array from batch.container. These batches describe only the surrounding container object; the streamed rows remain in the data batches.

To avoid confusion when inspecting batches:

1. Consume batch.data only when batch.batchType === 'data'; metadata batches will appear “incomplete” by design because they omit the streamed array.
2. If you need the full root object after streaming, enable {metadata: true} and merge the streamed data rows back into the container object instead of relying on the metadata batches alone.

Data Format

Parsed batches are of the format

{
  batchType: 'metadata' | 'partial-result' | 'final-result' | undefined;
  jsonpath: string;

  // standard batch payload
  data: any[] | any;
  bytesUsed: number;
  batchCount: number;
}

Options

Supports table category options such as batchType and batchSize.

Option	From	Type	Default	Description
json.table	[]	boolean	false	Parses non-streaming JSON as table, i.e. return the first embedded array in the JSON. Always true during batched/streaming parsing.
json.jsonpaths	[]	string[]	[]	A list of JSON paths (see below) indicating the array that can be streamed.
metadata (top level)	[]	boolean	If true, yields an initial and final batch containing the partial and final result (i.e. the root object, excluding the array being streamed).

JSONPaths

The loader implements a focused subset of the IETF JSONPath specification (RFC 9535). See the JSONPath support table for the full list of supported and unsupported features.

JSONPaths are used only to identify which array should be streamed, so selectors such as $.features[*] and $.features[:] are normalized to $.features. Descendant operators, element indexes, filters, and unions are not supported. Regardless of the paths provided, only arrays will be streamed.

Attribution

This loader is based on a fork of dscape's clarinet under BSD 2-clause license.