Skip to main content

ParquetLoader 🆕 🚧

From-v3.1

 

BETA

Streaming loader for Apache Parquet encoded files.

LoaderCharacteristic
File FormatParquet
Data FormatClassic Table
File Extension.parquet,
MIME TypeN/A (application/octet-stream)
File TypeBinary
Supported APIsload, parse, parseInBatches

Please refer to the parquet format page for information on which Parquet format features are supported.

Usage​

import {ParquetLoader} from '@loaders.gl/parquet';
import {load} from '@loaders.gl/core';

const data = await load(url, ParquetLoader, {parquet: options});

The ParquetLoader supports streaming parsing, in which case it will yield "batches" of rows.

import {ParquetLoader} from '@loaders.gl/parquet';
import {loadInBatches} from '@loaders.gl/core';

const batches = await loadInBatches('geo.parquet', ParquetLoader, {parquet: options}});

for await (const batch of batches) {
// batch.data will contain a number of rows
for (const feature of batch.data) {
switch (feature.geometry.type) {
case 'Polygon':
...
}
}
}

Data Format​

For details see parquet documentation.

Options​

Supports table category options such as batchType and batchSize.

OptionFromTypeDefaultDescription