Skip to main content

ParquetLoader

From-v3.1 BETA

Streaming loader for Apache Parquet encoded files.

LoaderCharacteristic
File Extension.parquet,
MIME TypeN/A (application/octet-stream)
File TypeBinary
File FormatParquet
Data FormatClassic Table
Supported APIsload, parse, parseInBatches

Please refer to the @loaders.gl/parquet module page for information on which Parquet format features are supported.

Usage

import {ParquetLoader} from '@loaders.gl/parquet';
import {load} from '@loaders.gl/core';

const data = await load(url, ParquetLoader, {parquet: options});

The ParquetLoader supports streaming parsing, in which case it will yield "batches" of rows.

import {ParquetLoader} from '@loaders.gl/parquet';
import {loadInBatches} from '@loaders.gl/core';

const batches = await loadInBatches('geo.parquet', ParquetLoader, {parquet: options}});

for await (const batch of batches) {
// batch.data will contain a number of rows
for (const feature of batch.data) {
switch (feature.geometry.type) {
case 'Polygon':
...
}
}
}

Data Format

For details see parquet documentation.

Options

Supports table category options such as batchType and batchSize.

OptionFromTypeDefaultDescription