Types and Vectors

This documentation reflects Arrow JS v4.0. Needs to be updated for the new Arrow API in v9.0 +.

Overview

Usage

Constructing new Vector instances is done through the static from() methods

Special Vectors

Dictionary Arrays

The Dictionary type is a special array type that enables one or more record batches in a file or stream to transmit integer indices referencing a shared dictionary containing the distinct values in the logical array. Later record batches reuse indices in earlier batches and add new ones as needed.

A Dictionary is similar to a factor in R or a pandas, or "Categorical" in Python. It is is often used with strings to save memory and improve performance.

StructVector

Holds nested fields.

Bool Vectors

Bool Vectors
BoolVector

Binary Vectors

Binary Vectors
BinaryVector

FloatVectors

| Float Vectors | Backing | Comments | | --- | | Float16Vector | Uint16Array | No native JS 16 bit type, additional methods available | | Float32Vector | Float32Array | Holds 32 bit floats | | Float64Vector | Float64Array | Holds 64 bit floats |

Static FloatVector Methods

FloatVector.from(data: Uint16Array): Float16Vector;

FloatVector.from(data: Float32Array): Float32Vector;

FloatVector.from(data: Float64Array): Float64Vector;

FloatVector16.from(data: Uint8Array | Iterable<Number>): Float16Vector;

FloatVector16.from(data: Uint16Array | Iterable<Number>): Float16Vector;

FloatVector32.from(data: Float32['TArray'] | Iterable<Number>): Float32Vector;

FloatVector64.from(data: Float64['TArray'] | Iterable<Number>): Float64Vector;

Float16Vector Methods

Since JS doesn't have half floats, Float16Vector is backed by a Uint16Array integer array. To make it practical to work with these arrays in JS, some extra methods are added.

toArray() : Uint16Array

Returns a zero-copy view of the underlying Uint16Array data.

Note: Avoids incurring extra compute or copies if you're calling toArray() in order to create a buffer for something like WebGL, but makes it hard to use the returned data as floating point values in JS.

toFloat32Array() : Float32Array

This method will convert values to 32 bit floats. Allocates a new Array.

toFloat64Array() : Float64Array

This method will convert values to 64 bit floats. Allocates a new Array.

IntVectors

Int Vectors	Backing	Comments
Int8Vector	Int8Array
Int16Vector	Int16Array
Int32Vector	Int32Array
Int64Vector	Int32Array	64-bit values stored as pairs of lo, hi 32-bit values for engines without BigInt support, extra methods available
Uint8Vector	Uint8Array
Uint16Vector	Uint16Array
Uint32Vector	Uint32Array
Uint64Vector	Uint32Array	64-bit values stored as pairs of lo, hi 32-bit values for engines without BigInt support, extra methods available

Int64Vector Methods

toArray() : Int32Array

Returns a zero-copy view of the underlying pairs of lo, hi 32-bit values as an Int32Array. This Array's length is twice the logical length of the Int64Vector.

toBigInt64Array(): BigInt64Array

Returns a zero-copy view of the underlying 64-bit integers as a BigInt64Array. This Array has the samne length as the length of the original Int64Vector.

Note: as of 03/2019, BigInt64Array is only available in v8/Chrome. In JS runtimes without support for BigInt, this method throws an unsupported error.

Uint64Vector Methods

toArray() : Uint32Array

Returns a zero-copy view of the underlying pairs of lo, hi 32-bit values as a Uint32Array. This Array's length is twice the logical length of the Uint64Vector.

toBigUint64Array(): BigUint64Array

Returns a zero-copy view of the underlying 64-bit integers as a BigUint64Array. This Array has the samne length as the length of the original Uint64Vector.

Note: as of 03/2019, BigUint64Array is only available in v8/Chrome. In JS runtimes without support for BigInt, this method throws an unsupported error.

Static IntVector Methods

IntVector.from(data: Int8Array): Int8Vector;

IntVector.from(data: Int16Array): Int16Vector;

IntVector.from(data: Int32Array, is64?: boolean): Int32Vector | Int64Vector;

IntVector.from(data: Uint8Array): Uint8Vector;

IntVector.from(data: Uint16Array): Uint16Vector;

IntVector.from(data: Uint32Array, is64?: boolean): Uint32Vector | Uint64Vector;

Int8Vector.from(this: typeof Int8Vector, data: Int8Array | Iterable<number>): Int8Vector;

Int16Vector.from(this: typeof Int16Vector, data: Int16Array | Iterable<number>): Int16Vector;

Int32Vector.from(this: typeof Int32Vector, data: Int32Array | Iterable<number>): Int32Vector;

Int64Vector.from(this: typeof Int64Vector, data: Int32Array | Iterable<number>): Int64Vector;

Uint8Vector.from(this: typeof Uint8Vector, data: Uint8Array | Iterable<number>): Uint8Vector;

Uint16Vector.from(this: typeof Uint16Vector, data: Uint16Array | Iterable<number>): Uint16Vector;

Uint32Vector.from(this: typeof Uint32Vector, data: Uint32Array | Iterable<number>): Uint32Vector;

Uint64Vector.from(this: typeof Uint64Vector, data: Uint32Array | Iterable<number>): Uint64Vector;

Date Vectors

Date Vectors	Backing
DateDayVector	Int32Array
DateMillisecondVector	Int32Array	TBD - stride: 2?