Skip to main content

ArrowTTC — Data types

How Oracle types map to Apache Arrow on the fetch (read) path, and how Arrow types map back to Oracle on the ingest/bind (write) path.

The non-obvious cases — unconstrained NUMBER has no exact Arrow type, DATE carries a time component (so it is a timestamp, not date32), TIMESTAMP WITH TIME ZONE collapses to a UTC instant — are called out below.


Read path: Oracle → Arrow

Numeric

Oracle typeArrow typeNotes
NUMBER(p,0), p ≤ 18int64Max 999,999,999,999,999,999 < 263; better for every consumer than a zero-scale decimal
NUMBER(p,0), p 19–38decimal128(p,0)Can exceed int64
NUMBER(p,s), s > 0decimal128(p,s)Exact; Oracle's max precision is 38, so decimal256 is never needed
NUMBER (unconstrained)utf8 (default) / float64 / decimal128(P,S)Governed by number_mapping
BINARY_INTEGERas NUMBER aboveSame policy path
BINARY_FLOATfloat32
BINARY_DOUBLEfloat64

Character & binary

Oracle typeArrow typeNotes
VARCHAR2 / CHAR / LONGutf8AL32UTF8 passes through unchanged
NVARCHAR2 / NCHARutf8National charset (AL16UTF16) transcoded UTF-16 → UTF-8; distinguished from VARCHAR2 only by the character-set form (csfrm)
RAW / LONG RAWbinary
ROWID / UROWIDutf8rendered as text
CLOButf8fetched via a LOB locator round-trip after the row
NCLOButf8national CLOB, UTF-16 → UTF-8
BLOBbinaryLOB locator round-trip

Temporal

Oracle typeArrow typeNotes
DATEtimestamp[s]Not date32 — Oracle DATE always carries a time component
TIMESTAMP(n)timestamp[s|ms|us|ns]unit follows the column's fractional-seconds precision
TIMESTAMP(n) WITH TIME ZONEtimestamp[…, tz=UTC]offset applied → UTC instant
TIMESTAMP(n) WITH LOCAL TIME ZONEtimestamp[…, tz=UTC]wire value is UTC (DBTIMEZONE assumed UTC)
INTERVAL YEAR TO MONTHinterval (month_day_nano)
INTERVAL DAY TO SECONDinterval (month_day_nano)

Fractional-seconds precision → unit (for TIMESTAMP and its TZ variants):

Column precisionArrow unit
0seconds
1–3milliseconds
4–6microseconds
7–9nanoseconds

Oracle allows up to 9 fractional digits, so the precision is rounded up to the next representable Arrow unit and no precision is lost.

Oracle 23ai native types

Version

Since ArrowTTC v0.2.10, decoded natively on Oracle 23ai / 26ai.

Oracle typeArrow typeNotes
BOOLEANbool23ai native boolean (Oracle type 252), packed one bit per value
JSONutf823ai native binary JSON (OSON, type 119) decoded to JSON text
VECTORlist<float32 | float64 | int8>23ai vectors (type 127); the element type follows the column's VECTOR format

On the wire, JSON and VECTOR are LOB-backed, so their images are prefetched inline in the row and decoded there rather than through a second LOB round trip.

Spatial

Version

Since ArrowTTC v0.3.1, on Oracle with the Spatial option.

Oracle typeArrow typeNotes
MDSYS.SDO_GEOMETRYbinary + geoarrow.wkb extensionDecoded to little-endian ISO WKB, tagged ARROW:extension:name = geoarrow.wkb so consumers such as GeoPandas or DuckDB spatial treat it as geometry. The stored SDO_SRID is surfaced as the geoarrow CRS metadata ({"crs_type":"srid","crs":N}), or forced with the arrowttc.sdo.srid statement option. On read, 2D POINT, LINESTRING, and single-exterior-ring POLYGON are decoded (any size); interior rings, multi-part geometries, 3D/LRS, and collections are out of scope and rejected. The write path below is broader.

Not yet mapped

  • BFILE — a pointer to a server-filesystem file through a DIRECTORY object; a different permission model, not a different encoding, so it is surfaced as unsupported rather than half-decoded.
  • Region-id TIMESTAMP WITH TIME ZONE (a named zone rather than a fixed offset) fails loudly rather than resolving the zone through DST rules; fixed-offset TSTZ and TSLTZ decode normally.
  • JSON / BOOLEAN on Oracle 19c–21c — these are not native column types before 23ai (JSON is stored in a LOB/VARCHAR2 with an IS JSON check), so such a column decodes as its underlying text/LOB type. Oracle 23ai's native JSON and BOOLEAN types are decoded directly — see Oracle 23ai native types above.
  • Unknown Oracle types are surfaced as opaque binary rather than guessed at.

Number Mapping

Oracle's unconstrained NUMBER (no declared precision or scale — describe reports scale −127) is Oracle's default numeric type and has no exact Arrow equivalent. A fixed decimal128(p,s) cannot be chosen safely (it would truncate high-scale values and overflow large ones), and an Arrow schema is fixed before the first batch, so the scale cannot be inferred by looking ahead in the stream. The adbc.arrowttc.number_mapping database option chooses:

ValueOutputNotes
auto (default)utf8Lossless. The deliberate default — a lossy default would silently lose precision on Oracle's most common numeric column
doublefloat64Fast, lossy — 15–17 significant digits against Oracle's 38
decimal:P,Sdecimal128(P,S)For callers who know their data; 1 ≤ P ≤ 38, 0 ≤ S ≤ P

Constrained columns are unaffected by this option — only unconstrained NUMBER is routed through it. The lossless default is not theoretical: the type-matrix fixture's unconstrained column round-trips 21 significant digits, well beyond float64.


Write path: Arrow → Oracle (ingest & bind)

On adbc_ingest and prepared-statement bind, Arrow arrays drive positional binds — one Arrow row is one execution. Ingest issues an array-bound INSERT with a SQL-injection-safe identifier and literal escaper, in create / append / replace / create_append modes, with optional target_db_schema and a GLOBAL TEMPORARY target (adbc.ingest.temporary). See CONNECTION.md for the ingest options.

A geoarrow.wkb Arrow column is ingested as a real MDSYS.SDO_GEOMETRY column — the inverse of the spatial read path (since v0.3.1). The create-table DDL emits the SDO_GEOMETRY type, each row's WKB is converted to Oracle's packed object image and bound as an OBJECT, a geoarrow CRS on the field maps to SDO_SRID, and — when the ingest itself creates the table — USER_SDO_GEOM_METADATA and a spatial index are registered. The write path is broader than read: in addition to points, lines, and single-ring polygons, it supports POLYGON with interior rings and the MULTIPOINT / MULTILINESTRING / MULTIPOLYGON kinds. 3D geometries, geometry collections, and malformed WKB are rejected.


See also