Logically the same as Binary, but the internal representation uses a view
struct that contains the string length and either the string’s entire data
inline (for small strings) or an inlined prefix, an index of another buffer,
and an offset pointing to a slice in that buffer (for non-small strings).
Optional compression for the memory buffers constituting IPC message
bodies. Intended for use with RecordBatch but could be used for other
message types
Exact decimal value represented as an integer value in two’s
complement. Currently 32-bit (4-byte), 64-bit (8-byte),
128-bit (16-byte) and 256-bit (32-byte) integers are used.
The representation uses the endianness indicated in the Schema.
For sending dictionary encoding information. Any Field can be
dictionary-encoded, but in this case none of its children may be
dictionary-encoded.
There is one vector / column per dictionary, but that vector / column
may be spread across multiple dictionary batches by using the isDelta
flag
Represents the same logical types that List can, but contains offsets and
sizes allowing for writes in any order and sharing of child values among
list values.
A data header describing the shared memory layout of a “record” or “row”
batch. Some systems call this a “row batch” internally and others a “record
batch”.
Contains two child arrays, run_ends and values.
The run_ends child array must be a 16/32/64-bit integer array
which encodes the indices at which the run with the value in
each corresponding index in the values child array ends.
Like list/struct types, the value array can be of any type.
A Struct_ in the flatbuffer metadata is the same as an Arrow Struct
(according to the physical memory layout). We used Struct_ here as
Struct is a reserved word in Flatbuffers
Time is either a 32-bit or 64-bit signed integer type representing an
elapsed time since midnight, stored in either of four units: seconds,
milliseconds, microseconds or nanoseconds.
Timestamp is a 64-bit signed integer representing an elapsed time since a
fixed epoch, stored in either of four units: seconds, milliseconds,
microseconds or nanoseconds, and is optionally annotated with a timezone.
A union is a complex type with children in Field
By default ids in the type vector refer to the offsets in the children
optionally typeIds provides an indirection between the child offset and the type id
for each child typeIds[offset] is the id used in the type vector
Logically the same as Utf8, but the internal representation uses a view
struct that contains the string length and either the string’s entire data
inline (for small strings) or an inlined prefix, an index of another buffer,
and an offset pointing to a slice in that buffer (for non-small strings).
Provided for forward compatibility in case we need to support different
strategies for compressing the IPC message body (like whole-body
compression rather than buffer-level) in the future