Skip to main content

ArpeTeradata — Data types

How Teradata types map to Apache Arrow on the read path, and how Arrow types map back to Teradata for bound parameters and bulk ingest.

The non-obvious cases are called out below:

  • Teradata has one binary floating type: FLOAT / REAL / DOUBLE PRECISION are all float64.
  • TIMESTAMP WITH TIME ZONE is returned as the UTC instant; TIME WITH TIME ZONE and INTERVAL come back as text.
  • An unbounded NUMBER comes back as lossless decimal text.
  • PERIOD reads as a struct, ARRAY and VECTOR as lists.
  • The GRAPHIC family cannot be read under a UTF8 session; translate it on the server.

Session settings that affect types​

  • The session character set is UTF8 (adbc.arpeteradata.charset, the only value accepted).
  • DECIMAL precision up to 38 is requested at logon. Without it, Teradata would cap DECIMAL at 18.
  • The session time zone is set to UTC right after logon, whatever the site, user or profile default. CURRENT_TIMESTAMP and other session-time functions therefore run in UTC, and bound time-zone-aware Arrow timestamps land unchanged. A SET TIME ZONE sent by your application changes this for the rest of the session.
  • Times and timestamps carry microsecond precision, Teradata's maximum. A column declared with fewer fractional digits is widened.

Read path: Teradata → Arrow​

Numeric​

Teradata typeArrow typeNotes
BYTEINTint8
SMALLINTint16
INTEGERint32
BIGINTint64
FLOAT, REAL, DOUBLE PRECISIONfloat64Teradata has one binary floating type.
DECIMAL(p,s) / NUMERIC(p,s), p 1–38decimal128(p,s)Exact.
NUMBER(p,s), p 1–38decimal128(p,s)Each value is rescaled to the column scale.
NUMBER with no precisionutf8Lossless decimal text without an exponent: 1.5E100 comes back as 15 followed by 99 zeros.

Character and binary​

Teradata typeArrow typeNotes
CHAR(n)utf8Trailing blanks are trimmed.
VARCHAR(n), LONG VARCHARutf8
BYTE(n)binary
VARBYTE(n), LONG VARBYTEbinary
GRAPHIC, VARGRAPHIC, LONG VARGRAPHIC—Refused by the server under a UTF8 session; see Unsupported types.

Temporal​

Teradata typeArrow typeNotes
DATEdate32Dates before 1900 are handled (0001-01-01 round-trips).
TIME(n)time64[us]
TIMESTAMP(n)timestamp[us] (no time zone)
TIMESTAMP(n) WITH TIME ZONEtimestamp[us, UTC]The driver applies the stored offset and returns the UTC instant.
TIME(n) WITH TIME ZONEutf8Arrow has no time-zone-aware time type. The text keeps its offset, e.g. 12:34:56.123456+05:30.
INTERVAL (all 13 forms, YEAR to DAY TO SECOND)utf8Teradata's fixed-width text form, including the leading sign column, e.g. 12 03:04:05.123456.

PERIOD, user-defined types, ARRAY, VECTOR​

Teradata typeArrow typeNotes
PERIOD(DATE)struct<begin: date32, end: date32>A NULL period is a null struct.
PERIOD(TIME(n))struct<begin: time64[us], end: time64[us]>
PERIOD(TIMESTAMP(n))struct<begin: timestamp[us], end: timestamp[us]>
PERIOD(TIMESTAMP(n) WITH TIME ZONE)struct<begin: timestamp[us, UTC], end: timestamp[us, UTC]>UTC instants, as for the scalar type.
PERIOD(TIME(n) WITH TIME ZONE)struct<begin: utf8, end: utf8>Text with offset, as for the scalar type.
Distinct user-defined typethe base type's mappingCREATE TYPE euro AS DECIMAL(10,2) reads as decimal128(10,2).
ARRAY / VARRAY, one dimensionlist<element>Elements of type BYTEINT, SMALLINT, INTEGER, BIGINT, FLOAT, DECIMAL / NUMBER(p,s), CHAR, VARCHAR, DATE, TIME, TIMESTAMP take their scalar mapping; NULL elements are null items. Other element types come back as the server's text, e.g. (1,2,3).
ARRAY, n dimensionsn nested listsINTEGER ARRAY[2][3] reads as [[1,2,3],[4,5,6]].
VECTORlist<float64>One list per row; rows can differ in length.
VECTOR32list<float32>
Structured or internal user-defined type (including MBR)utf8The type's transform text, e.g. (0,0,1,1).

Teradata sends user-defined values through their transform, which for an ARRAY is text: FLOAT elements therefore carry the 15 significant digits the server writes. The first time a connection reads an ARRAY or structured type, it looks the type up once and caches it for the connection's lifetime. A statement is never executed twice for this. On that first read, a user-defined column returned by a CALL or a multi-statement request can come back as the server's text.

LOBs and documents​

Teradata typeArrow typeNotes
CLOB (LATIN or UNICODE)large_stringUTF-8 text.
XMLlarge_stringThe document text, as the server stores it.
DATASET STORAGE FORMAT CSVlarge_string
JSON (text, BSON, UBJSON, LATIN)large_string, extension arrow.jsonJSON text whatever the storage format.
BLOBlarge_binary
DATASET STORAGE FORMAT AVROlarge_binaryVerified on NULL values and the column type only.
ST_Geometrylarge_string, extension geoarrow.wktWKT text such as LINESTRING (0 0,1 1). Teradata's WKT carries no SRID, so no CRS is named.

Extensions are carried in the Arrow field metadata (ARROW:extension:name, ARROW:extension:metadata): a consumer that does not know them sees a plain large_string. An empty LOB and a NULL LOB are distinct. A single LOB value up to Teradata's 2 GB limit reads.

Rows over 64 KB​

Rows up to Teradata's large-row limit (about 1 MB) read. One limit is the server's: with a UTF8 session, Teradata exports at most 64000 bytes of a VARCHAR value, so a 32000-character CJK value comes back as 21333 characters (Teradata's own drivers behave the same).

Type names in field metadata​

Every result field carries an ARPETERADATA:type key in its Arrow field metadata: the column's Teradata type, spelled as Teradata DDL spells it. It is present on query results, on ExecuteSchema and on GetTableSchema, so Teradata types that share an Arrow type (utf8 for VARCHAR, CHAR, TIME WITH TIME ZONE and INTERVAL, for example) stay distinguishable. Since v0.2.9.

ColumnARPETERADATA:type
VARCHAR(100) CHARACTER SET UNICODEVARCHAR(100)
DECIMAL(10,2)DECIMAL(10,2)
NUMBER(*,2)NUMBER(*,2)
TIMESTAMP(6) WITH TIME ZONETIMESTAMP(6) WITH TIME ZONE
INTERVAL DAY(4) TO SECOND(2)INTERVAL DAY(4) TO SECOND(2)
PERIOD(DATE)PERIOD(DATE)
ST_Geometry and other user-defined typesSYSUDTLIB.ST_Geometry

Character lengths are in characters, and the character set is not part of the name. User-defined types, including ST_Geometry, VECTOR and ARRAY types, are named SYSUDTLIB.<name>.


Unsupported types​

Unsupported types fail loudly: the driver never returns wrong or empty values for them.

GRAPHIC, VARGRAPHIC, LONG VARGRAPHIC are refused by the server under the UTF8 session character set (Teradata errors 6705 and 6701, reported as INVALID_ARGUMENT). Teradata exports GRAPHIC only to a KANJI session character set, and Teradata's own drivers fail the same way. Convert the column on the server:

SELECT TRANSLATE(vg USING GRAPHIC_TO_UNICODE) AS vg FROM t;

The result is ordinary Unicode text (utf8). A fixed GRAPHIC(n) value keeps its padding of ideographic spaces (U+3000).

Any other type that reaches the driver without a mapping fails when the schema is built, naming the type code and the column: unsupported column type code <n> for column <name>.


Bound parameters: Arrow → Teradata​

Values bound to ? markers (Bind, Python's executemany) accept these Arrow types:

Arrow typeSent as
int8, int16, int32, int64BYTEINT, SMALLINT, INTEGER, BIGINT
float64FLOAT
decimal32 / decimal64 / decimal128 / decimal256 (p,s), p 1–38DECIMAL(p,s)
utf8VARCHAR, at most 64000 UTF-8 bytes per value
binaryVARBYTE
date32DATE
time64[us]TIME(n), with n the fewest fractional digits the bound values need
timestamp[us] without a time zoneTIMESTAMP(n), n as for TIME
timestamp[us] with a time zoneTIMESTAMP(n) WITH TIME ZONE at +00:00: a WITH TIME ZONE column stores the same instant, a plain TIMESTAMP column stores it in UTC
null typeNULL

Also accepted, for bound parameters and for ingest (since v0.3.2): bool, uint8 / uint16 / uint32 / uint64, float32, large_string, large_binary, fixed_size_binary, date64, time32[s|ms], time64[ns] and timestamp[s|ms|ns]. Nanoseconds are truncated to microseconds.

Every other Arrow type is refused before anything is sent, including decimals with a precision over 38 and dictionary-encoded columns. Dates and timestamps outside years 1 to 9999, and times of 24 hours or more, are refused with value out of range. A string or binary value over 64000 bytes is refused. Because the declared precision follows the data, whole-second values bind into TIME(0) and TIMESTAMP(0) columns; a value with a fraction is refused by Teradata for such a column (error 5404) rather than truncated.


Ingest: Arrow → Teradata column types​

When adbc_ingest creates the table (create, replace, create_append), the Arrow schema gives the column types. Since v0.3.2.

Arrow typeTeradata column
boolBYTEINT (0 or 1)
int8, int16, int32, int64BYTEINT, SMALLINT, INTEGER, BIGINT
uint8, uint16, uint32SMALLINT, INTEGER, BIGINT
uint64DECIMAL(20,0)
float32, float64FLOAT
decimal32 / decimal64 / decimal128 / decimal256 (p,s), p up to 38DECIMAL(p,s)
utf8, large_utf8VARCHAR(n) CHARACTER SET UNICODE, n = adbc.arpeteradata.ingest.varchar_length (default 4000)
binary, large_binaryVARBYTE(2n), capped at 64000
fixed_size_binary(n)BYTE(n)
date32, date64DATE
time32[s], time32[ms], time64[us], time64[ns]TIME(0), TIME(3), TIME(6), TIME(6)
timestamp[s] / [ms] / [us] / [ns] without a time zoneTIMESTAMP(0), TIMESTAMP(3), TIMESTAMP(6), TIMESTAMP(6)
timestamp with a time zonethe same, WITH TIME ZONE
null typeVARCHAR(1) CHARACTER SET UNICODE
  • A non-nullable Arrow field becomes a NOT NULL column.
  • Created tables are MULTISET and NO PRIMARY INDEX. To get a primary index, create the table yourself and ingest with append.
  • Nanosecond times and timestamps are truncated to microseconds: the one lossy case.
  • String width. A string longer than its column is truncated silently in Teradata (BTET) session mode, and refused in ANSI mode (error 3996). This also applies to append into a narrower existing column. Raise adbc.arpeteradata.ingest.varchar_length (up to 32000), or create the table yourself, wide enough, and use append.
  • A table whose declared row is too wide is refused by Teradata at CREATE (error 3933): lower varchar_length.
  • string_view, binary_view, dictionary, list, struct, map, interval and duration columns fail with NOT_IMPLEMENTED, before anything is inserted.
  • With append, the driver creates no columns: Teradata converts the values to the existing column types.

See also​