Formats and rounding#

A format is a way of storing numbers in a fixed number of bits. A floating-point format splits its bits between an exponent, which sets the range, and a significand, which sets the precision: bfloat16 has float32's range with far less precision, and the float8 formats trade both for size. Integer formats store whole numbers in a fixed range. Rounding modes say which nearby value an exact result becomes, and overflow modes what happens past the largest value.

Every dtype argument takes a format or its name. er.FORMATS maps names to format objects, and each format is also available by name, such as er.bfloat16. An unknown name raises UnsupportedFormatError.

Floating-point formats#

Name	Bits	Exponent bits	Precision (bits)	Infinity	-0
`float64`	64	11	53	yes	yes
`float32`	32	8	24	yes	yes
`float16`	16	5	11	yes	yes
`bfloat16`	16	8	8	yes	yes
`float8_e5m2`	8	5	3	yes	yes
`float8_e4m3`	8	4	4	yes	yes
`float8_e3m4`	8	3	5	yes	yes
`float8_e4m3fn`	8	4	4	no	yes
`float8_e5m2fnuz`	8	5	3	no	no
`float8_e4m3fnuz`	8	4	4	no	no
`float8_e4m3b11fnuz`	8	4	4	no	no

Name	Largest finite	Smallest normal	Smallest subnormal
`float64`	1.797693e+308	2.225074e-308	4.940656e-324
`float32`	3.402823e+38	1.175494e-38	1.401298e-45
`float16`	65504	6.103516e-05	5.960464e-08
`bfloat16`	3.389531e+38	1.175494e-38	9.18355e-41
`float8_e5m2`	57344	6.103516e-05	1.525879e-05
`float8_e4m3`	240	0.015625	0.001953125
`float8_e3m4`	15.5	0.25	0.015625
`float8_e4m3fn`	448	0.015625	0.001953125
`float8_e5m2fnuz`	57344	3.051758e-05	7.629395e-06
`float8_e4m3fnuz`	240	0.0078125	0.0009765625
`float8_e4m3b11fnuz`	30	0.0009765625	0.0001220703

The ranges are rounded to 7 significant digits.

Every floating-point format has NaN. Precision counts the implicit leading bit. The fn formats have no infinity, and the fnuz formats have no infinity and no negative zero.

Complex, integer and bool formats#

Name	Parts	Name	Range
`complex32`	two `float16`	`int8`, `int16`, `int32`, `int64`	signed, 8 to 64 bits
`complex64`	two `float32`	`uint8`, `uint16`, `uint32`, `uint64`	unsigned, 8 to 64 bits
`complex128`	two `float64`	`bool`	False, True

Code

import exactref as er

# er.get_format looks a format up by name; component is the format of each part
print("complex64 part:", er.get_format("complex64").component.name)

# The range of int8, and the largest uint16
print("int8 min:", er.int8.min_value)
print("int8 max:", er.int8.max_value)
print("uint16 max:", er.uint16.max_value)

Output

complex64 part: float32
int8 min: -128
int8 max: 127
uint16 max: 65535

Rounding#

class Rounding(*values)#

IEEE 754 rounding-direction attributes.

NEAREST_EVEN = 'nearest_even'#

NEAREST_AWAY = 'nearest_away'#

TOWARD_ZERO = 'toward_zero'#

TOWARD_POSITIVE = 'toward_positive'#

TOWARD_NEGATIVE = 'toward_negative'#

property is_nearest: bool#: Whether this mode rounds to nearest (ties to even or away).

NEAREST_EVEN (the default everywhere) rounds to nearest with ties to even, NEAREST_AWAY to nearest with ties away from zero, and the other three in their direction.

Overflow#

overflow says what happens to a value past the format's range:

Format	`overflow`	Past the largest value
floating-point with infinity	`"ieee"` (default)	infinity, or the largest finite value when the rounding direction points toward zero
floating-point without infinity	`"ieee"` (default)	the largest value when rounding to nearest, toward zero or toward it; NaN when rounding outward
any floating-point	`"nan"`	NaN wherever IEEE 754 would give infinity
any floating-point	`"saturate"`	the largest finite value, also for infinities the format cannot hold
integer	`"wrap"` (default)	wraps around, as PyTorch does
integer	`"error"`	raises `OverflowError`
integer	`"saturate"`	clamps to the format's range

Integer results of operations, and conversions of array data, wrap by default, as in PyTorch; converting Python values that do not fit raises.

Looking formats up#

get_format(dtype: Format | str) -> Format#: Return a format instance, looking up names in FORMATS.

exactref.FORMATS#: A dict from each format's name to its format object.

Format classes#

class Format#

How one element's encoding maps to a value.

Subclasses provide name, itemsize (bytes per element), kind ("float", "int", "bool" or "complex") and components (encodings per element: 1, or 2 for complex).

class FloatFormat(name: str, exponent_bits: int, mantissa_bits: int, special: str = 'ieee', bias: int | None = None, nan_encodings: tuple[int, int] | None = None)#

A signed binary floating-point format.

special selects how non-finite values are encoded:

"ieee": an all-ones exponent encodes infinity (zero mantissa) or NaN.
"fn": finite only; all-ones exponent and mantissa encode NaN (as in float8_e4m3fn).
"fnuz": finite only, no negative zero; the negative-zero encoding is the only NaN (as in float8_e4m3fnuz).

bias defaults to 2**(e-1) - 1, or 2**(e-1) for "fnuz". nan_encodings gives the encodings of positive and negative NaN when they differ from the quiet NaN with the value's sign.

property precision: int#: Significand bits, including the implicit leading bit.

property emin: int#: Exponent of the smallest normal value (2**emin).

property emax: int#: Exponent of the largest finite binade (2**emax <= max_finite).

decode(bits: int) -> FloatValue#: Return the exact value of an encoding.

encode(value: FloatValue | Fraction | int | float, *, rounding: Rounding = Rounding.NEAREST_EVEN, overflow: str = 'ieee') -> int#

Round a value to this format and return its encoding.

overflow="ieee" (the default) follows IEEE 754 in a format with infinity: a result rounding past the largest finite value becomes infinity, or the largest finite value when the rounding direction points toward zero. A format without infinity has no value past its largest, so the result rounds within the values it has: the largest finite value to nearest, toward zero or toward it, and NaN when rounding outward, where the format has no value. overflow="nan" instead gives NaN wherever IEEE 754 would give infinity (to nearest too), as kernels that keep the IEEE rule in such a format do. overflow="saturate" replaces every infinity this format cannot hold, or that overflow would produce, with the largest finite value. An infinite value gives NaN in a format without infinity, except with "saturate".

NaN encodes as this format's canonical NaN: the quiet NaN with the value's sign, or the encoding nan_encodings gives (PyTorch's bfloat16 NaN is always 0x7FC0; its float8_e5m2 NaN has every mantissa bit set).

class IntFormat(name: str, itemsize: int, signed: bool)#: A two's-complement (signed) or unsigned integer format.

class BoolFormat(name: str = 'bool', itemsize: int = 1)#: One byte holding 0 (False) or 1 (True). Other byte values are invalid.

class ComplexFormat(name: str, component: FloatFormat)#: A complex format: real then imaginary part, each in component format.