Results#
Every operation returns an er.Result. This page goes through what it holds
and the options of its methods.
import exactref as er
# Two 2x2 float32 matrices
A = er.asarray([[1.0, 2.0], [3.0, 4.0]], "float32")
B = er.asarray([[0.1, 0.2], [0.3, 0.4]], "float32")
# ExactRef's result for their matrix product
r = er.matmul(A, B)
rounded and guarantee#
r.rounded is the exact answer rounded once to the output format: by
default the input format. add, multiply and gemm promote inputs of
different formats as PyTorch does (bfloat16 with float16 gives float32). The
operation's dtype= and rounding= arguments choose another format and
rounding.
# The exact product, rounded once to float32, the input format
print("rounded:", r.rounded.tolist())
# What rounded satisfies
print("guarantee:", r.guarantee)
# The same for the dot product of two int32 vectors, whose output is int32
print("int32 guarantee:",
er.matmul(er.asarray([1, 2], "int32"), er.asarray([3, 4], "int32")).guarantee)
rounded: [[0.7000000476837158, 1.0], [1.5, 2.200000047683716]]
guarantee: correctly_rounded
int32 guarantee: exact
guarantee says what rounded satisfies: "correctly_rounded" for
floating-point and complex outputs, and "exact" for integer outputs. When
an operation cannot establish its guarantee, it raises
ReferenceUnavailableError instead of returning a result.
exact#
r.exact holds a certified enclosure of every exact value, for comparisons
finer than the output format. r.exact.value(index) is an Enclosure: the
true value lies within radius of midpoint.
# The enclosure of element (0, 0)
e = r.exact.value((0, 0))
# The radius is an exact Fraction
print("radius:", repr(e.radius))
# The midpoint as an exact Fraction, then as the nearest float64
print("midpoint:", float(e.midpoint.to_fraction()))
radius: Fraction(0, 1)
midpoint: 0.700000025331974
A radius of 0 means the midpoint is the exact value, as it is for sums of products. Transcendental values, such as the entries of a matrix exponential, have a tiny nonzero radius:
from fractions import Fraction
# ExactRef's result for the matrix exponential of a 2x2 float64 matrix
m = er.matrix_exp(er.asarray([[0.0, 1.0], [-1.0, 0.0]], "float64"))
# The enclosure of element (0, 0)
e = m.exact.value((0, 0))
# Is its radius above 0 but below 2**-100?
print("tiny nonzero radius:", 0 < e.radius < Fraction(1, 2**100))
tiny nonzero radius: True
Each midpoint has a 128-bit significand and the radius is at most one unit
in its last place. A complex element's enclosure is a pair, real part then
imaginary part. r.exact.values() lists every element's enclosure,
row-major. A magnitude beyond 2**(2**20) (or below its reciprocal) is
stored as an infinity (or a zero) with out_of_range set, which still
decides its rounding to every format.
tolerance()#
r.tolerance(compute=None, *, ulps=None, flush_to_zero=False, standard="normal")
The worst-case |output - rounded| per element over the finite outputs of
correct implementations. The options describe which implementations count
as correct:
Option |
Meaning |
Default |
|---|---|---|
|
the format the implementation rounds and adds in at the required steps |
float32 for inputs narrower than float32, else the input format |
|
how far library functions such as |
2; 1 for matrix functions |
|
the implementation flushes subnormals of the compute format to zero, as |
|
|
|
|
# The tolerance with the default compute format, float32 for these inputs
print("tolerance:")
for row in r.tolerance().tolist():
print(row)
# The tolerance for an implementation that rounds and adds in bfloat16
print("bfloat16 tolerance:")
for row in r.tolerance("bfloat16").tolist():
print(row)
tolerance:
[1.1920928955078125e-07, 1.1920928955078125e-07]
[2.384185791015625e-07, 2.384185791015625e-07]
bfloat16 tolerance:
[0.005490124225616455, 0.007843017578125]
[0.0117645263671875, 0.01725459098815918]
The result is a float64 array, rounded upward. For complex outputs it is
complex128, one bound per part. inf marks an element with no bound at all,
and nan an element whose reference is not finite. Results are kept, so
asking again with the same options does not compute them again.
finite_range(), may_be() and covers()#
They take the same options as tolerance():
r.finite_range()returns(low, high): the lowest and highest finite output of a correct implementation, as arrays of the output's dtype.r.may_be("+inf"),r.may_be("-inf")andr.may_be("nan")say where a correct implementation may produce that value. They can be True where no correct implementation overflows, because the overflow check uses the error bound, but they are never False where one can.r.covers(output)combines them into one verdict per element.outputmust have the result's dtype and shape.
Reading complicated tolerances shows them on overflow, infinities, NaN and complex outputs.
normal()#
r.normal(compute=None, *, flush_to_zero=False)
The output of normal accumulation in index order, emulated exactly. Each
product is rounded to the compute format and added in index order, each
sum rounded to nearest, ties to even. Where adding the next value would
overflow, it starts a new partial sum, and the partial sums are then
combined the same way. The result is rounded to the output format as
rounded is.
# What a loop that adds the products in index order returns
print("normal:", r.normal().tolist())
normal: [[0.7000000476837158, 1.0], [1.5, 2.200000047683716]]
It is computed only when you call it, and kept. Each operation's page says
what else its normal answer fixes. matrix_exp, matrix_exp_frechet and
matrix_exp's backward have no normal answer: they raise
NotImplementedError.
Several outputs#
Some operations return more than one output: attention returns output and
logsumexp, and backward operations return one gradient per input. They are
held by name. r[name] is a result for just that output, which is what
covers() and later operations need:
# The gradients of A @ B for an output gradient equal to the identity matrix
g = er.matmul.backward(A, B, er.asarray([[1.0, 0.0], [0.0, 1.0]], "float32"))
# The names of its outputs
print("outputs:", list(g.rounded))
# One output, read by name from rounded
print("grad_A:")
for row in g.rounded.grad_A.tolist():
print(row)
# g["grad_B"] is a result holding just that output
print("grad_B:", g["grad_B"].rounded.tolist())
outputs: ['grad_A', 'grad_B']
grad_A:
[0.10000000149011612, 0.30000001192092896]
[0.20000000298023224, 0.4000000059604645]
grad_B: [[1.0, 3.0], [2.0, 4.0]]
boundaries#
r.boundaries lists the operation's required rounding steps
(boundaries) by name, each
an er.Boundary with a kind, a description and what it applies_to:
# The names of matmul's required rounding steps: it has one
print("boundaries:", list(r.boundaries))
# That boundary, looked up by name
b = r.boundaries["accumulate"]
# What kind of rounding step it is, and what it applies to
print("kind:", b.kind)
print("applies to:", b.applies_to)
print("description:", b.description)
boundaries: ['accumulate']
kind: accumulate
applies to: dot
description: summation of the k products of one output element
Views: reshape and transpose#
r.reshape(shape) and r.transpose(i, j) give a one-output result in
another layout, for a later operation that needs it, such as a column for an
outer product. Moving elements rounds nothing, so rounded, exact,
tolerance() and normal() move with them:
# r with its two dimensions swapped
print("transposed:", r.transpose(0, 1).rounded.tolist())
# The row sums of A, reshaped into a 2x1 column
print("column:", er.sum(A, axis=1).reshape((2, 1)).rounded.tolist())
transposed: [[0.7000000476837158, 1.5], [1.0, 2.200000047683716]]
column: [[3.0], [7.0]]
A view has no backward of its own: call backward on the original result,
with the gradient reshaped back.
backward()#
r.backward(grad) runs the operation's backward on the same inputs. See
Backward.