Types and Vectors

This documentation reflects Arrow JS v4.0. Needs to be updated for the new Arrow API in v9.0 +.

Overview

Usage

Constructing new Vector instances is done through the static from() methods

Special Vectors

Dictionary Arrays

The Dictionary type is a special array type that enables one or more record batches in a file or stream to transmit integer indices referencing a shared dictionary containing the distinct values in the logical array. Later record batches reuse indices in earlier batches and add new ones as needed.

A Dictionary is similar to a factor in R or a pandas, or "Categorical" in Python. It is is often used with strings to save memory and improve performance.

StructVector

Holds nested fields.

Bool Vectors

Bool Vectors
BoolVector

Binary Vectors

Binary Vectors
BinaryVector

FloatVectors

Float Vectors	Backing	Comments
Float16Vector	Uint16Array	No native JS 16 bit type, additional methods available
Float32Vector	Float32Array	Holds 32 bit floats
Float64Vector	Float64Array	Holds 64 bit floats

Static FloatVector Methods

FloatVector.from(data: Uint16Array): Float16Vector;

FloatVector.from(data: Float32Array): Float32Vector;

FloatVector.from(data: Float64Array): Float64Vector;

FloatVector16.from(data: Uint8Array | Iterable<Number>): Float16Vector;

FloatVector16.from(data: Uint16Array | Iterable<Number>): Float16Vector;

FloatVector32.from(data: Float32['TArray'] | Iterable<Number>): Float32Vector;

FloatVector64.from(data: Float64['TArray'] | Iterable<Number>): Float64Vector;

Float16Vector Methods

Since JS doesn't have half floats, Float16Vector is backed by a Uint16Array integer array. To make it practical to work with these arrays in JS, some extra methods are added.

toArray() : Uint16Array

Returns a zero-copy view of the underlying Uint16Array data.

Note: Avoids incurring extra compute or copies if you're calling toArray() in order to create a buffer for something like WebGL, but makes it hard to use the returned data as floating point values in JS.

toFloat32Array() : Float32Array

This method will convert values to 32 bit floats. Allocates a new Array.

toFloat64Array() : Float64Array

This method will convert values to 64 bit floats. Allocates a new Array.

IntVectors

Int Vectors	Backing	Comments
Int8Vector	Int8Array
Int16Vector	Int16Array
Int32Vector	Int32Array
Int64Vector	Int32Array	64-bit values stored as pairs of lo, hi 32-bit values for engines without BigInt support, extra methods available
Uint8Vector	Uint8Array
Uint16Vector	Uint16Array
Uint32Vector	Uint32Array
Uint64Vector	Uint32Array	64-bit values stored as pairs of lo, hi 32-bit values for engines without BigInt support, extra methods available

Int64Vector Methods

toArray() : Int32Array

Returns a zero-copy view of the underlying pairs of lo, hi 32-bit values as an Int32Array. This Array's length is twice the logical length of the Int64Vector.

toBigInt64Array(): BigInt64Array

Returns a zero-copy view of the underlying 64-bit integers as a BigInt64Array. This Array has the samne length as the length of the original Int64Vector.

Note: as of 03/2019, BigInt64Array is only available in v8/Chrome. In JS runtimes without support for BigInt, this method throws an unsupported error.

Uint64Vector Methods

toArray() : Uint32Array

Returns a zero-copy view of the underlying pairs of lo, hi 32-bit values as a Uint32Array. This Array's length is twice the logical length of the Uint64Vector.

toBigUint64Array(): BigUint64Array

Returns a zero-copy view of the underlying 64-bit integers as a BigUint64Array. This Array has the samne length as the length of the original Uint64Vector.

Note: as of 03/2019, BigUint64Array is only available in v8/Chrome. In JS runtimes without support for BigInt, this method throws an unsupported error.

Static IntVector Methods

IntVector.from(data: Int8Array): Int8Vector;

IntVector.from(data: Int16Array): Int16Vector;

IntVector.from(data: Int32Array, is64?: boolean): Int32Vector | Int64Vector;

IntVector.from(data: Uint8Array): Uint8Vector;

IntVector.from(data: Uint16Array): Uint16Vector;

IntVector.from(data: Uint32Array, is64?: boolean): Uint32Vector | Uint64Vector;

Int8Vector.from(this: typeof Int8Vector, data: Int8Array | Iterable<number>): Int8Vector;

Int16Vector.from(this: typeof Int16Vector, data: Int16Array | Iterable<number>): Int16Vector;

Int32Vector.from(this: typeof Int32Vector, data: Int32Array | Iterable<number>): Int32Vector;

Int64Vector.from(this: typeof Int64Vector, data: Int32Array | Iterable<number>): Int64Vector;

Uint8Vector.from(this: typeof Uint8Vector, data: Uint8Array | Iterable<number>): Uint8Vector;

Uint16Vector.from(this: typeof Uint16Vector, data: Uint16Array | Iterable<number>): Uint16Vector;

Uint32Vector.from(this: typeof Uint32Vector, data: Uint32Array | Iterable<number>): Uint32Vector;

Uint64Vector.from(this: typeof Uint64Vector, data: Uint32Array | Iterable<number>): Uint64Vector;

Date Vectors

Date Vectors	Backing
DateDayVector	Int32Array
DateMillisecondVector	Int32Array	TBD - stride: 2?