Skip to main content

ArpeOracle — Data types

How Oracle types map to Apache Arrow on the fetch (read) path, and how Arrow types map back to Oracle on the ingest/bind (write) path.

The non-obvious cases — unconstrained NUMBER has no exact Arrow type, DATE carries a time component (so it is a timestamp, not date32), TIMESTAMP WITH TIME ZONE collapses to a UTC instant, and INTERVAL types default to Parquet-writable Arrow types rather than interval (month_day_nano) — are called out below.


Read path: Oracle → Arrow​

Numeric​

Oracle typeArrow typeNotes
NUMBER(p,0), p ≤ 18int64Max 999,999,999,999,999,999 < 263; better for every consumer than a zero-scale decimal
NUMBER(p,0), p 19–38decimal128(p,0)Can exceed int64
NUMBER(p,s), s > 0decimal128(p,s)Exact; Oracle's max precision is 38, so decimal256 is never needed
NUMBER (unconstrained)utf8 (default) / float64 / decimal128(P,S)Governed by number_mapping
BINARY_INTEGERas NUMBER aboveSame policy path
BINARY_FLOATfloat32
BINARY_DOUBLEfloat64

Character & binary​

Oracle typeArrow typeNotes
VARCHAR2 / CHAR / LONGutf8AL32UTF8 passes through unchanged
NVARCHAR2 / NCHARutf8National charset (AL16UTF16) transcoded UTF-16 → UTF-8
RAW / LONG RAWbinary
ROWID / UROWIDutf8Oracle's printable rowid string (the 18-character form ROWIDTOCHAR returns); a logical (index-organized table) UROWID renders as * followed by base-64
CLOButf8fetched via a LOB locator round-trip after the row
NCLOButf8national CLOB, UTF-16 → UTF-8
BLOBbinaryLOB locator round-trip

Temporal​

Oracle typeArrow typeNotes
DATEtimestamp[s]Not date32 — Oracle DATE always carries a time component
TIMESTAMP(n)timestamp[s|ms|us|ns]unit follows the column's fractional-seconds precision
TIMESTAMP(n) WITH TIME ZONEtimestamp[…, tz=UTC]UTC instant; the original offset or named region is not kept
TIMESTAMP(n) WITH LOCAL TIME ZONEtimestamp[…, tz=UTC]wire value is UTC (DBTIMEZONE assumed UTC)
INTERVAL YEAR TO MONTHint64 (default) / interval (month_day_nano)Default is the total number of months; governed by interval_mapping
INTERVAL DAY TO SECONDutf8 (default) / interval (month_day_nano)Default is an ISO-8601 duration string; governed by interval_mapping

Fractional-seconds precision → unit (for TIMESTAMP and its TZ variants):

Column precisionArrow unit
0seconds
1–3milliseconds
4–6microseconds
7–9nanoseconds

Oracle allows up to 9 fractional digits, so the precision is rounded up to the next representable Arrow unit and no precision is lost.

Oracle 23ai native types​

Version

Since ArpeOracle v0.2.10, decoded natively on Oracle 23ai / 26ai.

Oracle typeArrow typeNotes
BOOLEANbool23ai native boolean
JSONutf8Native binary JSON (OSON, Oracle 21c and later) decoded to JSON text
VECTORlist<float32 | float64 | int8>23ai vectors; the element type follows the column's VECTOR format

Spatial​

Version

Since ArpeOracle v0.3.1, on Oracle with the Spatial option.

Oracle typeArrow typeNotes
MDSYS.SDO_GEOMETRYbinary + geoarrow.wkb extensionDecoded to little-endian ISO WKB, tagged ARROW:extension:name = geoarrow.wkb so consumers such as GeoPandas or DuckDB spatial treat it as geometry. The stored SDO_SRID (taken from the first non-null geometry in the result) is surfaced as the geoarrow CRS metadata ({"crs_type":"srid","crs":N}), or forced with the arpeoracle.sdo.srid statement option. On read, 2D POINT, LINESTRING, and single-exterior-ring POLYGON are decoded (any size); interior rings, multi-part geometries, 3D/LRS, and collections are out of scope and rejected. The write path below is broader.

Not yet mapped​

  • BFILE — a pointer to a server-filesystem file through a DIRECTORY object; a different permission model, not a different encoding, so it is surfaced as unsupported rather than half-decoded.
  • JSON before 21c, BOOLEAN before 23ai — not native column types on those versions (JSON is stored in a LOB/VARCHAR2 with an IS JSON check), so such a column decodes as its underlying text/LOB type. The native JSON type (21c and later) and BOOLEAN (23ai) are decoded directly — see Oracle 23ai native types above.
  • Other types — object types other than SDO_GEOMETRY, collections, and any type not listed above — are refused rather than guessed at: the query fails with column <name> (Oracle type <n>) is not supported by this driver yet.

Number Mapping​

Oracle's unconstrained NUMBER (no declared precision or scale) is Oracle's default numeric type and has no exact Arrow equivalent. A fixed decimal128(p,s) cannot be chosen safely (it would truncate high-scale values and overflow large ones), and an Arrow schema is fixed before the first batch, so the scale cannot be inferred by looking ahead in the stream. The adbc.arpeoracle.number_mapping database option chooses:

ValueOutputNotes
auto (default)utf8Lossless. The deliberate default — a lossy default would silently lose precision on Oracle's most common numeric column
doublefloat64Fast, lossy — 15–17 significant digits against Oracle's 38
decimal:P,Sdecimal128(P,S)For callers who know their data; 1 ≤ P ≤ 38, 0 ≤ S ≤ P

Constrained columns are unaffected by this option — only unconstrained NUMBER is routed through it. The lossless default matters in practice: an unconstrained NUMBER can hold values with more significant digits (21 and beyond) than float64 can represent.


Interval Mapping​

Version

Since ArpeOracle v0.3.7. The default changed in v0.3.7 — see the note below.

Arrow's interval(month_day_nano) type has no Parquet representation, so any Parquet-writing consumer (FastBCP, pandas→parquet, …) drops or errors on interval columns rendered that way. The default therefore maps both interval types to Parquet-writable, lossless Arrow types — matching the Arrow schema FastBCP's regular Oracle driver (oraodp) produces, so the Parquet schema is identical across connectors. The adbc.arpeoracle.interval_mapping database option chooses:

ValueINTERVAL YEAR TO MONTHINTERVAL DAY TO SECONDNotes
parquet (default)int64 (total months)utf8 (ISO-8601 duration)Both lossless and Parquet-writable
nativeinterval (month_day_nano)interval (month_day_nano)Arrow-native, but not Parquet-writable

The ISO-8601 duration format is [-]P{D}DT{H}H{Mi}M{S}[.frac]S — e.g. a +4 05:06:07.891011 interval renders as P4DT5H6M7.891011S, and -1 02:03:04.5 as -P1DT2H3M4.5S.

Behavior change in v0.3.7

Before v0.3.7 both interval types defaulted to interval (month_day_nano). A consumer relying on that output must now set interval_mapping=native to restore it.


Write path: Arrow → Oracle (ingest & bind)​

On adbc_ingest and prepared-statement bind, Arrow arrays drive positional binds — one Arrow row is one execution. Ingest issues an array-bound INSERT with a SQL-injection-safe identifier and literal escaper, in create / append / replace / create_append modes, with optional target_db_schema and a GLOBAL TEMPORARY target (adbc.ingest.temporary). See Connection for the ingest options.

Table, schema and column names are quoted exactly as given, so they are case-sensitive: ingesting into my_table creates "my_table", which an unquoted SELECT … FROM my_table (folded to MY_TABLE) does not find. Use upper-case names to get ordinary unquoted Oracle identifiers.

When the ingest creates the table (create, replace, or create_append on a missing table), each Arrow column becomes:

Arrow typeOracle column
boolNUMBER(1)
int8 / uint8NUMBER(3)
int16 / uint16NUMBER(5)
int32 / uint32NUMBER(10)
int64 / uint64NUMBER(19)
float32BINARY_FLOAT
float64BINARY_DOUBLE
decimal128(p,s)NUMBER(p,s)
utf8 / large_utf8VARCHAR2(4000)
binary / large_binaryRAW(2000)
fixed_size_binary(n), n ≤ 2000RAW(n)
binary tagged geoarrow.wkbMDSYS.SDO_GEOMETRY
date32 / date64DATE
timestamp[s|ms|us|ns]TIMESTAMP(0|3|6|9)
timestamp[…, tz=…]TIMESTAMP(0|3|6|9) WITH TIME ZONE
nullVARCHAR2(1)

Any other Arrow type (lists, structs, intervals, …) is rejected with cannot create a column for Arrow type ….

A geoarrow.wkb Arrow column is ingested as a real MDSYS.SDO_GEOMETRY column — the inverse of the spatial read path (since v0.3.1). The create-table DDL emits the SDO_GEOMETRY type, each row's WKB is converted to Oracle's packed object image and bound as an OBJECT, a geoarrow CRS on the field maps to SDO_SRID, and — when the ingest itself creates the table — USER_SDO_GEOM_METADATA and a spatial index are registered. The write path is broader than read: in addition to points, lines, and single-ring polygons, it supports POLYGON with interior rings and the MULTIPOINT / MULTILINESTRING / MULTIPOLYGON kinds. 3D geometries, geometry collections, and malformed WKB are rejected.


See also​