Skip to main content

ArpeMSSQL — Data types

How SQL Server types map to Apache Arrow on the fetch (read) path, and how Arrow types map back to SQL Server on the ingest/bind (write) path.

The non-obvious cases — TINYINT is unsigned, MONEY is a fixed-scale decimal, DATETIMEOFFSET collapses to a UTC instant — are called out below.


SQL type name in the field metadata​

Version

Since ArpeMSSQL v0.6.7.

Every field the driver returns (query results, ExecuteSchema and GetTableSchema) carries the column's SQL Server type in the field metadata key ARPEMSSQL:type, spelled the way SQL Server spells it (the system_type_name column of sp_describe_first_result_set):

ColumnARPEMSSQL:type
INTint
DECIMAL(10,2)decimal(10,2)
NVARCHAR(50)nvarchar(50) (the declared length in characters)
VARCHAR(MAX)varchar(max)
TIMEtime(7) (the default scale is written out)
DATETIME2(3)datetime2(3)
ROWVERSIONtimestamp
GEOGRAPHYgeography (in every geospatial mode; the GeoArrow keys stay alongside)
an alias type over VARCHAR(8)varchar(8) (the base type, as SQL Server reports it)
SYSNAMEnvarchar(128)

The Arrow type says how values are represented; this key says what the column is on the server, so it can be used to recreate the column (varchar(max) and nvarchar(50) are both Arrow utf8).


Read path: SQL Server → Arrow​

Numeric​

SQL Server typeArrow typeNotes
TINYINTuint8SQL Server TINYINT is unsigned (0–255)
SMALLINTint16
INTint32
BIGINTint64
BITboolbit-packed on finish
REALfloat32
FLOATfloat64
DECIMAL / NUMERIC(p,s)decimal128(p,s)precision/scale preserved
MONEYdecimal128(19,4)fixed scale 4
SMALLMONEYdecimal128(10,4)fixed scale 4

Character & binary​

SQL Server typeArrow typeNotes
CHAR / VARCHAR / VARCHAR(MAX)utf8single-byte code page (CP1252/125x/874/OEM) transcoded via the column collation; UTF-8 collations pass through
NCHAR / NVARCHAR / NVARCHAR(MAX)utf8UCS-2 on the wire, decoded to UTF-8
TEXT / NTEXT (legacy LOB)utf8TEXT honours its code page; NTEXT is UCS-2
BINARY / VARBINARY / VARBINARY(MAX)binary
IMAGE (legacy LOB)binaryraw bytes
XMLutf8serialized as text
UNIQUEIDENTIFIERutf836-char canonical form; lowercase by default, casing via uuid_casing
VECTOR(n) (SQL Server 2025)fixed_size_list<float32>[n]Native float32 embeddings, both read and ingest. Requires SQL Server 2025 + the negotiated native-vector transport (auto-on). Without negotiation the server returns the vector as varchar(max) JSON (→ utf8). float16 vectors are JSON-only over TDS.

Temporal​

SQL Server typeArrow typeNotes
DATEdate32days since epoch
TIME(n)time32[s|ms] or time64[us|ns]unit follows the column scale
SMALLDATETIMEtimestamp[s]
DATETIMEtimestamp[ms]1/300 s wire resolution
DATETIME2(n)timestamp[s|ms|us|ns]unit follows the column scale
DATETIMEOFFSET(n)timestamp[s|ms|us|ns, tz=UTC]offset applied → UTC instant

Temporal scale → unit (for TIME / DATETIME2 / DATETIMEOFFSET):

Column scaleArrow unit
0seconds
1–3milliseconds
4–6microseconds
7microseconds by default, nanoseconds with timestamp_precision=nanosecond

SQL Server's scale-7 ceiling is 100 ns, so true Arrow-nanosecond precision is not reachable.

Version

Microsecond default and the timestamp_precision option since ArpeMSSQL v0.6.2. Earlier versions always emitted scale-7 temporals as nanoseconds.

Scale-7 precision (timestamp_precision)

By default a scale-7 DATETIME2 / DATETIMEOFFSET / TIME column is emitted at microseconds (timestamp[us] / time64[us] → Parquet TIMESTAMP_MICROS), because many consumers (for example, older Spark) reject INT64 TIMESTAMP(NANOS). Set the connection option adbc.arpemssql.timestamp_precision=nanosecond to keep the full 100 ns resolution (timestamp[ns] / time64[ns]); microsecond (the default) drops the sub-microsecond digit. Scales 0–6 are unaffected either way.

CLR user-defined types​

SQL Server typeArrow typeNotes
hierarchyidbinaryopaque server bytes
geometry / geographygeoarrow.wkb (default)client-side converted to WKB; see Geospatial types

Not yet mapped​

  • sql_variant — not yet supported on the read path.
  • float16 — no SQL Server type maps to Arrow float16 (cannot be bound).
  • Sub-100 ns temporal precision — a SQL Server limit, not a driver gap: scale-7 columns top out at 100 ns. Nanosecond Arrow output is available via timestamp_precision=nanosecond (see the scale-7 note above); the default is microseconds.

Geospatial types​

Version

Since ArpeMSSQL v0.5.12.

geometry and geography are CLR UDTs whose wire bytes are Microsoft's proprietary serialization (MS-SSCLRT) — not WKB — so passing them through unchanged is useless to Arrow / GeoParquet / DuckDB / GeoPandas consumers. By default the driver converts them to the canonical geoarrow.wkb extension type. The adbc.arpemssql.geospatial database option controls the read path:

ValueOutputNotes
geoarrow (default)Arrow binary + geoarrow.wkb extensionWKB plus ARROW:extension:name=geoarrow.wkb and an ARROW:extension:metadata JSON object; read directly by GeoPandas / GeoParquet / DuckDB
wkbArrow binaryISO WKB (x=lon, y=lat), no extension metadata
varbinary (opt-in)Arrow binaryraw opaque UDT bytes; backward compatible, not usable outside SQL Server

How it works. The conversion is client-side. Your query runs verbatim — nothing is rewritten and there is no extra round-trip. The row decoder recognises geometry/geography columns from the result's wire metadata and converts each value's native serialization to ISO WKB as rows stream in. The output is byte-identical to the server's own STAsBinary() / AsBinaryZM(). NULL geometries are preserved, and ORDER BY — or any other query shape — keeps the conversion.

  • Z/M coordinates are preserved as ISO WKB Z/M/ZM type codes (+1000/+2000/+3000), matching AsBinaryZM().
  • Curved geometries (CircularString / CompoundCurve / CurvePolygon) convert to ISO WKB curve types 8/9/10; geography FullGlobe becomes SQL Server's nonstandard WKB type 126 — both exactly as STAsBinary() renders them. Consumer support for curve WKB varies.

In geoarrow mode the extension metadata carries crs as EPSG:<srid> (read from the first non-NULL value's SRID header; omitted for SRID 4326 / OGC:CRS84 and for undefined SRIDs, per the GeoParquet convention) and, for geography, edges: spherical (geometry is planar, so edges is omitted). The reader decodes rows until every spatial column has an observed SRID — usually just the first row — then fixes the schema.

Known limitations:

  • Mixed SRIDs within one column are not auto-detected; the first observed SRID is used for the whole column's crs.
  • The schema is fixed before rows are delivered, so a spatial column that is NULL for the whole first batch omits its crs. Order the query so a non-NULL value appears early, or raise adbc.arpemssql.batch_size.
  • Schema-only execution (Statement.ExecuteSchema) reports spatial columns as plain binary — it runs under FMTONLY, so no rows and no SRID come back. Bound-parameter SELECTs (WHERE id = ?) are converted normally.
  • A malformed or unknown-version spatial value fails the read with a clear error naming the column; geospatial=varbinary reads the raw UDT bytes as an escape hatch.

The write/ingest path (Arrow WKB → geometry/geography) shipped in ArpeMSSQL v0.5.25 — see Write path below.


Write path: Arrow → SQL Server (ingest & bind)​

adbc_ingest loads through INSERT BULK; prepared-statement parameters are bound through TDS RPC. Arrow arrays are routed to SQL Server types:

  • Integers, floats, bool, decimal, date, time, timestamp (with and without timezone), utf8, binary, and the view types (string_view, binary_view) are accepted on both paths. Dictionary-encoded columns are sent as their values.
  • utf8 columns are sent as NVARCHAR (UTF-16), so every character round-trips; for a VARCHAR target the server converts to the column's code page. mode=create makes string columns NVARCHAR(4000) and binary columns VARBINARY(8000); longer values need a MAX column that you create.
  • timestamp[ns] / time64[ns] values are floored to SQL Server's 100 ns tick.
  • XML targets are routed via NVARCHAR(MAX) + a server-side cast (the raw XML wire type is rejected by INSERT BULK).
  • fixed_size_list<float32>[n] ingests to a SQL Server 2025 VECTOR(n) column (native binary; mode=create emits VECTOR(n) DDL). Only the float32 child is supported; requires the negotiated native-vector transport.
  • Arrow WKB → geometry/geography (since v0.5.25). WKB cells are transcoded client-side to Microsoft's [MS-SSCLRT] UDT format and bulk-loaded with INSERT BULK, so a column extracted as geoarrow.wkb round-trips back in. Point, LineString, Polygon (with interior rings), the Multi* containers, and GeometryCollection are supported recursively, along with Z/M/ZM ordinates and empty geometries; geography gets the (x, y) → (lat, long) axis-order swap. The SRID stamped into each value comes from the column's GeoArrow crs metadata, or, when the column has none, from adbc.arpemssql.ingest.srid, falling back to the kind default (geometry 0, geography 4326). Curves, FullGlobe, malformed WKB, and heterogeneous Multi* children are rejected at encode time with a clear error rather than sent to the server.

Bind inputs with no SQL Server equivalent return NOT_IMPLEMENTED: float16 and null-typed parameters.

See also​