Public API

Tracing

Ghost.traceFunction
trace(f, args...; is_primitive, primitives)

Trace function call, produce call value and a Tape.

trace records to the tape primitive methods and recursively dives into non-primitives. There are 2 ways to tell trace that a particular method is a primitive:

  • provide is_primitive(sig) -> Bool function, where sig is is a method signature, e.g. Tuple{map(typeof, (f, args...))...}
  • provide an iterable primitives; in this case trace matches all methods of this function
source
Ghost.is_primitiveFunction
is_primitive(sig)

The default implementation of is_primitive argument in trace(). Returns true if the method with the provided signature is defined in one of the Julia's built-in modules, e.g. Base, Core, Broadcast, etc.

source
Ghost.call_signatureFunction
call_signature(fn, args...)
call_signature(tape::Tape, op::Call)

Get a signature of a function call. The obtain signature is suitable for is_primitive(sig).

source
Ghost.__new__Function
__new__(T, args...)

User-level version of the new() pseudofunction. Can be used to construct most Julia types, including structs without default constructors, closures, etc.

source

Variables

Ghost.VariableType

Variable represents a reference to an operation on a tape. Variables can be used to index tape or keep reference to a specific operation on the tape.

Variables (also aliesed as V) can be:

  • free, created as V(id) - used for indexing into tape
  • bound, created as V(op)` - used to keep a robust reference to an operation on the tape
source
Ghost.rebind!Function
rebind!(tape::Tape, op, st::Dict)
rebind!(tape::Tape, st::Dict; from, to)

Rebind all variables according to substitution table. Example:

tape = Tape()
v1, v2 = inputs!(tape, nothing, 3.0, 5.0)
v3 = push!(tape, mkcall(*, v1, 2))
st = Dict(v1.id => v2.id)
rebind!(tape, st)
@assert tape[v3].args[1].id == v2.id

See also: rebind_context!()

source
Ghost.rebind_context!Function
rebind_context!(tape::Tape, st::Dict)

Rebind variables in the tape's context according to substitution table. By default does nothing, but can be overwitten for specific Tape{C}

source

Tape structure

Ghost.TapeType

Linearized representation of a function execution.

Fields

  • ops - vector of operations on the tape
  • result - variable pointing to the operation to be used as the result
  • parent - parent tape if any
  • meta - internal metadata
  • c - application-specific context
source
Ghost.CallType

Operation represening function call on tape. Typically, calls are constructed using mkcall function.

Important fields of a Call{T}:

  • fn::T - function or object to be called
  • args::Vector - vector of variables or values used as arguments
  • val::Any - the result of the function call
source
Ghost.LoopType

Operation representing a loop in an computational graph. See the online documentation for details.

source
Ghost.mkcallFunction
mkcall(fn, args...; val=missing)

Convenient constructor for Call operation. If val is missing (default) and call value can be calculated from (bound) variables and constants, they are calculated. To prevent this behavior, set val to some neutral value.

source

Tape transformations

Base.push!Function
push!(tape::Tape, op::AbstractOp)

Push a new operation to the end of the tape.

source
Base.insert!Function
insert!(tape::Tape, idx::Integer, ops::AbstractOp...)

Insert new operations into tape starting from position idx.

source
Base.replace!Function
replace!(tape, op  => new_ops; rebind_to=length(new_ops), old_new=Dict())

Replace specified operation with 1 or more other operations, rebind variables in the reminder of the tape to ops[rebind_to].

Operation can be specified directly, by a variable or by ID.

source
Base.deleteat!Function
deleteat!(tape::Tape, idx; rebind_to = nothing)

Remove tape[V(idx)] from the tape. If rebind_to is not nothing, then replace all references to V(idx) with V(rebind_to).

idx may be an index or Variable/AbstractOp directly.

source
Ghost.primitivize!Function
primitivize!(tape::Tape; is_primitive=is_primitive)

Trace non-primitive function calls on a tape and decompose them into a list of corresponding primitive calls.

Example

f(x) = 2x - 1
g(x) = f(x) + 5

tape = Tape()
_, x = inputs!(tape, g, 3.0)
y = push!(tape, mkcall(f, x))
z = push!(tape, mkcall(+, y, 5))
tape.result = z

primitivize!(tape)

# output

Tape{Dict{Any, Any}}
  inp %1::typeof(g)
  inp %2::Float64
  %3 = *(2, %2)::Float64
  %4 = -(%3, 1)::Float64
  %5 = +(%4, 5)::Float64
source

Tape execution

Ghost.play!Function
play!(tape::Tape, args...; debug=false)

Execute operations on the tape one by one. If debug=true, print each operation before execution.

source
Ghost.to_exprFunction
to_expr(tape::Tape)

Generate a Julia expression corresponding to the tape.

source

Loops

Ghost.should_trace_loops!Function
should_trace_loops!(val=false)

Turn on/off loop tracing. Without parameters, resets the flag to the default value

source

Internal API

The following types and functions might be useful for better understanding of Ghost behavior, but are not part of the public API and may not hold backward compatibility guarantees.

Ghost.FunctionResolverType

Dict-like data structure which maps function signature to a value. Unlike real dict, getindex(rsv, sig) returns either exact match, or closest matching function signature. Example:

rsv = FunctionResolver{Symbol}()
rsv[Tuple{typeof(sin), Float64}] = :Float64
rsv[Tuple{typeof(sin), Real}] = :Real
rsv[Tuple{typeof(sin), Number}] = :Number

rsv[Tuple{typeof(sin), Float64}]   # ==> :Float64
rsv[Tuple{typeof(sin), Float32}]   # ==> :Real
source
Ghost.TracerOptionsType

Tracer options. Configured globally via the following methods:

  • shouldtraceloops!()
  • shouldassertbranches!()
source
Ghost.record_or_recurse!Function

Record function call onto a tape or recurse into it.

Params:

  • t::IRTracer - current tracer
  • res_id::Int - IR ID of the operation
  • farg_irvars - IR variables of the operation
  • fargs - values of the operation
source
Ghost.enter_loop!Function

Trigger loop start operations.

Arguments:

  • t :: IRTracer Current tracer
  • loop_id :: Int Unique ID of a loop being entered
  • loopinputir_ids :: Vector{Int} IR IDs of variables which will be used as loop inputs. Includes loop block inputs and any outside IDs

This function is added to the very beginning of the loop block(s). During the first iteration we initialize a subtape which will be used later to create the Loop operation on the parent tape. Since all iterations of the loop produce identical operations, we only need to trace it once. However, it turns out to be easier to record all iterations (seprated by a special _LoopEnd op) and then prune unused iterations.

Another important detail is that Loop's subtape is a completely valid and independent tape with its own call frame and inputs which include all explicit and implicit inputs to the loop's block in the original IR.

source
Ghost.exit_loop!Function

Trigget loop end operations.

This function is added just before the end of the loop block. Since we record all iterations of the loop, we must remember tape IDs of continuation condition and exit variables during the first run.

source

Index