The ORC runtime provides APIs for executor processes in an ORC JIT session (as opposed to the LLVM ORC libraries which provide APIs for controller processes). This includes support for both JIT‘d code itself, and for users of JIT’d code.
LLVM‘s On Request Compilation (ORC) APIs support cross-process loading of JIT’d code. We call the process that defines and links the JIT‘d code the controller and the process that executes JIT’d code the executor. Controller processes will link LLVM‘s ORC library, and construct a JIT’d program using an llvm::orc::ExecutionSession instance (typically through an convenience wrapper like llvm::orc::LLJIT). Executor processes construct an orc_rt::Session object to manage resources for, and access to, JIT'd code within the executor process.
The Session object is the root object for a JIT‘d program. It owns the Service instances that manage services and resources supporting JIT’d code (e.g. JIT'd memory, unwind info registrations, dynamic library handles, etc.).
The Session object must be constructed prior to adding any JIT‘d code, and must outlive execution of any JIT’d code.
An executor may have more than one Session object, in which case each Session object must outlive execution of any JIT'd code added to that specific session.
ControllerAccess objects support bidirectional RPC between JIT'd code in the executor and the ExecutionSession in the controller.
Calls in both directions are to “wrapper functions” with a fixed signature (a function that takes a blob of bytes and returns a blob of bytes as its result). ControllerAccess objects can not generally assume anything about the format of the bytes being sent (their interpretation is up to the called function). The RPC is not fully symmetric: Calls from the controller to the executor specify wrapper function addresses (i.e. the controller can invoke any code in the executor). Calls from the executor to the controller specify tags, which are addresses in the executor processes that are associated with handlers in the controller. This ensures that the executing process can only call deliberately exposed entry points in the controller.
ControllerAccess objects may be detached before the session ends, at which point JIT'd code may continue executing, but will receive no further calls from the controller and can make no further calls to the controller.
Service is an interface for classes that provide services to the Session. E.g. memory managers, or dynamic library loaders.
The Service interface provides two operations: onDetach and onShutdown. onDetach signals that controller access is permanently unavailable. It is always called before onShutdown, regardless of how the Session reaches shutdown -- including when no controller was ever attached. Since no further requests will be made by the controller after onDetach, Services may use it to abandon any fine-grained book-keeping that is only needed to service controller requests. Many Services will implement onDetach as a no-op.
The onShutdown operation will be called at Session destruction time, after all outstanding keepalives have been released. Services should release all held resources during onShutdown.
Session teardown must not proceed while JIT‘d code is still live on a stack: freeing the JIT’d code (and the resources it runs against) out from under those frames would crash the moment control returns to them.
To help enforce this the Session carries a keepalive TaskGroup. Code that is about to run JIT'd code obtains a TaskGroup::Token -- a keepalive -- from the group to bracket that execution, and the group delays Session shutdown until every keepalive has been released. Session::callWithKeepalive acquires and holds one for you around a synchronous call; keepalives can also be acquired manually from the TokenSource returned by Session::keepaliveTokenSource.
A keepalive brackets a single span of execution on a stack -- not a whole chain of asynchronous operations. When an asynchronous operation captures a continuation, whoever later invokes that continuation must obtain a fresh keepalive to bracket the invocation (the continuation can‘t do it itself: its entry point may already be JIT’d code).
Keepalive acquisition fails once Session shutdown has been requested -- and it can fail even for a nested or resumed call whose caller already holds one. So every caller of JIT'd code needs a way to abort and unwind when acquisition is denied; callWithKeepalive reports denial through its return value, which callers must check.
Runs Tasks within the ORC runtime. In particular, calls originating from the controller (via ControllerAccess) will be dispatched as Tasks.
TaskDispatchers are responsible for ensuring that all dispatched Tasks have completed or been destroyed during Session shutdown.
A wrapper function is any function with the following C signature:
void (orc_rt_SessionRef Session, uint64_t CallId, orc_rt_WrapperFunctionReturn Return, orc_rt_WrapperFunctionBuffer ArgBytes);
where orc_rt_WrapperFunctionReturn and orc_rt_WrapperFunctionBuffer are defined as:
typedef struct { orc_rt_WrapperFunctionBufferDataUnion Data; size_t Size; } orc_rt_WrapperFunctionBuffer; /** * Asynchronous return function for an orc-rt wrapper function. */ typedef void (*orc_rt_WrapperFunctionReturn)( orc_rt_SessionRef Session, uint64_t CallId, orc_rt_WrapperFunctionBuffer ResultBytes);
The orc_rt::WrapperFunction class provides APIs for implementing and calling wrapper functions.
An SPS wrapper function is a wrapper function that uses the SimplePackedSerialization scheme (see documentation in orc-rt/include/orc-rt/SimplePackedSerialization.h).
Document...