Chevron DownAPI Reference

JSONLoader

Streaming loader for JSON encoded files.

LoaderCharacteristic
File Extension.json,
File TypeText
File FormatJSON
Data FormatClassic Table
Supported APIsload, parse, parseSync, parseInBatches

Usage

import {JSONLoader} from '@loaders.gl/json';
import {load} from '@loaders.gl/core';

const data = await load(url, JSONLoader, {json: options});

The JSONLoader supports streaming JSON parsing, in which case it will yield "batches" of rows from one array. To e.g. parse a stream of GeoJSON, the user can specify the options.json.jsonpaths to stream the features array.

import {JSONLoader} from '@loaders.gl/json';
import {loadInBatches} from '@loaders.gl/core';

const batches = await loadInBatches('geojson.json', JSONLoader, {json: {jsonpaths: ['$.features']}});

for await (const batch of batches) {
  // batch.data will contain a number of rows
  for (const feature of batch.data) {
    switch (feature.geometry.type) {
      case 'Polygon':
      ...
    }
  }
}

If no JSONPath is specified the loader will stream the first array it encounters in the JSON payload.

When batch parsing an embedded JSON array as a table, it is possible to get access to the containing object using the {json: {_rootObjectBatches: true}} option.

The loader will yield an initial and a final batch with batch.container providing the container object and batch.batchType set to root-object-batch-partial and root-object-batch-complete respectively.

import {JSONLoader} from '@loaders.gl/json';
import {loadInBatches} from '@loaders.gl/core';

const batches = await loadInBatches('geojson.json', JSONLoader);

for await (const batch of batches) {
  switch (batch.batchType) {
    case 'root-object-batch-partial': // contains fields seen so far
    case 'root-object-batch-complete': // contains all fields except the streamed array
      console.log(batch.container);
      break;
    default:
    // batch.data will contain a number of rows
    for (const feature of batch.data) {
      switch (feature.geometry.type) {
        case 'Polygon':
        ...
      }
    }
  }
}

Data Format

Parsed batches are of the format

{
  batchType: 'metadata' | 'partial-result' | 'final-result' | undefined;
  jsonpath: string;

  // standard batch payload
  data: any[] | any;
  bytesUsed: number;
  batchCount: number;
}

Options

Supports table category options such as batchType and batchSize.

OptionFromTypeDefaultDescription
json.table[Website shields.io]booleanfalseParses non-streaming JSON as table, i.e. return the first embedded array in the JSON. Always true during batched/streaming parsing.
json.jsonpaths[Website shields.io]string[][]A list of JSON paths (see below) indicating the array that can be streamed.
metadata (top level)[Website shields.io]booleanIf true, yields an initial and final batch containing the partial and final result (i.e. the root object, excluding the array being streamed).

JSONPaths

A minimal subset of the JSONPath syntax is supported, to specify which array in a JSON object should be streamed as batchs.

$.component1.component2.component3

  • No support for wildcards, brackets etc. Only paths starting with $ (JSON root) are supported.
  • Regardless of the paths provided, only arrays will be streamed.

Attribution

This loader is based on a fork of dscape's clarinet under BSD 2-clause license.