Introduction

         

This documentation describes loaders.gl v4.0. See our release notes to learn what is new.

Docs for older versions are available on github: v3.3, v2.3, v1.3.

Overview

loaders.gl is a collection of open source loaders and writers for various file formats, primarily focused on supporting visualization and analytics of big data. Tabular, geospatial, and 3D file formats are well covered.

Published as a suite of composable loader modules with consistent APIs and features, offering advanced features such as worker thread and incremental parsing, loaders.gl aims to be a trusted companion when you need to load data into your application.

While loaders.gl can be used with any JavaScript application or framework, vis.gl frameworks such as deck.gl come pre-integrated with loaders.gl.

Loaders

loaders.gl provides a wide selection of loaders organized into categories:

Category	Loaders
Table Loaders	Streaming tabular loaders for CSV, JSON, Arrow etc
Geospatial Loaders	Loaders for geospatial formats such as GeoJSON KML, WKT/WKB, Mapbox Vector Tiles etc.
Image Loaders	Loaders for images, compressed textures, supercompressed textures (Basis). Utilities for mipmapped arrays, cubemaps, binary images and more.
Pointcloud and Mesh Loaders	Loaders for point cloud and simple mesh formats such as Draco, LAS, PCD, PLY, OBJ, and Terrain.
Scenegraph Loaders	glTF loader
Tiled Data Loaders	Loaders for 3D tile formats such as 3D Tiles, I3S and potree

Code Examples

loaders.gl provides a small core API module with common functions to load and save data, and a range of optional modules that provide loaders and writers for specific file formats.

A minimal example using the load function and the CSVLoader to load a CSV formatted table into a JavaScript array:

import {load} from '@loaders.gl/core';
import {CSVLoader} from '@loaders.gl/csv';

const data = await load('data.csv', CSVLoader);

for (const row of data) {
  console.log(JSON.stringify(row)); // => '{header1: value1, header2: value2}'
}

Streaming parsing is available using ES2018 async iterators, e.g. allowing "larger than memory" files to be incrementally processed:

import {loadInBatches} from '@loaders.gl/core';
import {CSVLoader} from '@loaders.gl/csv';

for await (const batch of await loadInBatches('data.csv', CSVLoader)) {
  for (const row of batch) {
    console.log(JSON.stringify(row)); // => '{header1: value1, header2: value2}'
  }
}

To quickly get up to speed on how the loaders.gl API works, please see Get Started.

Supported Platforms

loaders.gl supports both browsers and Node.js:

• Evergreen Browsers recent versions of major evergreen browsers (e.g. Chrome, Firefox, Safari) are supported on both desktop and mobile.
• Node.js All current LTS releases are supported.

Design Goals

Framework Agnostic - Files are parsed into clearly documented plain data structures (objects + typed arrays) that can be used with any JavaScript framework.

Browser Support - supports recent versions of evergreen browsers, and ensures that loaders are easy to bundle.

Node Support - loaders.gl can be used when writing backend and cloud services, and you can confidently run your unit tests under Node.

Worker Support - Many loaders.gl loaders come with pre-built web workers, keeping the main thread free for other tasks while parsing completes.

Loader Categories - loaders.gl groups similar data formats into "categories" that return parsed data in "standardized" form. This makes it easier to build applications that can handle multiple similar file formats.

Format Autodection - Applications can specify multiple loaders when parsing a file, and loaders.gl will automatically pick the right loader for a given file based on a combination of file/url extensions, MIME types and initial data bytes.

Streaming Support - Many loaders can parse in batches from both node and WhatWG streams, allowing "larger than memory" files to be processed, and initial results to be available while the remainder of a file is still loading.

Composability and Bundle-Size Optimization - Loaders for each file format are published in independent npm modules to allow applications to cherry-pick only the loaders it needs. In addition, modules are optimized for tree-shaking, and many larger loader libraries and web workers are loaded from CDN on use and not included in your application bundle.

Binary Data - loaders.gl is optimized to load into compact memory representations and use with WebGL frameworks (e.g. by returning typed arrays whenever possible). Note that in spite of the .gl naming, loaders.gl has no any actual WebGL dependencies and loaders can be used without restrictions in non-WebGL applications.

Multi-Asset Loading - Formats like glTF, Shapefile, or mip mapped / cube textures can require dozens of separate loads to resolve all linked assets (external buffers, images etc). loaders.gl loads all linked assets before resolving the returned Promise.

Modern JavaScript - loaders.gl is written in TypeScript 5.0 and standard ES2018, is packaged as ECMAScript modules, and the API emphasizes modern, portable JavaScript constructs, e.g. async iterators instead of streams, ArrayBuffer instead of Buffer, etc.

Licenses

loaders.gl itself is MIT licensed, however various modules contain forked code under several permissive, compatible open source licenses, such as ISC, BSD and Apache licenses. Each loader module provides some license notes, so if the distinction matters to you, please check the documentation for each module and decide accordingly. We guarantee that loaders.gl will never include code with non-permissive, commercial or copy-left licenses.

Credits and Attributions

loaders.gl is maintained by a group of organizations collaborating through open governance under the OpenJS and Linux Foundations.

While loaders.gl contains substantial amounts of original code, it also repackages lots of superb work done by others in the open source community. We try to be as explicit as we can about the origins and attributions of each piece of code, both in the documentation page for each module and in the preservation of comments relating to authorship and contributions inside forked source code.

Even so, we can make mistakes, and we may not have the full history of the code we are reusing. If you think that we have missed something, or that we could do better in regard to attribution, please let us know.

Primary maintainers

Organizations that currently contribute most significantly to the development and maintenance of loaders.gl:

Open Governance

loaders.gl is a part of the vis.gl framework suite, an open governance Linux Foundation project that is developed collaboratively by multiple organizations and individuals and the Urban Computing Foundation.