Jactl Continuations and Virtual Threads in Java 8

Introduction

Jactl is a secure, embeddable scripting language for Java applications. When I first started developing Jactl, I wanted a scripting language that compiled to bytecode for optimum performance, was secure so applications could control exactly what scripts could and couldn't do, and most of all, did not block the execution thread when long-running, blocking operations were performed.

When I began developing Jactl, Java 21 and VirtualThreads did not exist, and event-driven, reactive applications (such as ones based on Vert.x) were the way in which high-throughput Java applications were written. I also had a need for a scripting language that would work on applications still stuck on Java 8 or Java 11.

note

Now, with the later versions of Java, Java applications that want to use VirtualThreads rather than adopting an event-based architecture can choose not to use the built-in Jactl async mechanism described here by configuring the JactlContext.async(false) flag.

A reactive application is one where a pool of event-loop threads process events from a queue. The golden rule is that events should never block as this will pause one of the event-loop threads, preventing it from processing any more events until that blocking operation completes. A blocking operation is one where the thread is no longer actively processing code but is waiting for the result of an operation such as a database request or a remote procedure call. If blocking operations can occur on an event-loop thread, you will eventually see situations where all threads are waiting for long-running operations and no events are being processed.

I wanted a scripting language that could be invoked from an event-loop thread but, when it performed any blocking operation, it would somehow save its state and return, freeing up the thread to process further events. When the result of the long-running operation was available the script would be resumed from the point where it left off and continue with its processing. In Java 21 and later, VirtualThreads provide the same functionality - they preserve the call stack with all the local variables and allow the thread to continue performing other work and when the blocking operation is complete the call stack is restored and the program continues from where it left off.

The goal was only to save the execution state of the Jactl code, not the state of the Java code that was invoking a Jactl script. Since the Java application is event-based, the script will complete as a new event on an event-loop thread and a completion callback will be invoked once the script finishes that calls back into the Java application with the script result. The callback provided by the application can hold onto any state that the application needs.

note

To save having to write method/function everywhere, I will just refer to functions, but everywhere I talk about a function, the same would also apply to a method call.

Continuations

In Java 8, of course, there is no way to preserve the call stack, either in Java or in JVM bytecode, so I had to use a different mechanism to achieve the same end.

Imagine that we have a script that needs to invoke a function that performs a long-running operation. For the sake of the example, let's assume that it needs to perform sleep() for some period of time before it does something else. There will be a Java call stack with a stack frame for each nested method call and then the rest of the stack will be Jactl stack frames, one for each nested Jactl function call, with the topmost stack frame being the stack frame for the sleep() function itself. Each stack frame tracks where in the code the function invocation occurs, along with the values for its local variables:

To capture the execution state, I figured that the easiest thing to do would be to throw an exception at the start of a long-running operation like sleep() and generate code in each Jactl method/function that catches the exception, saves its state, and throws a new exception that is chained to the one it just caught. I called the class for the exception being thrown Continuation since a continuation is a representation of the execution state of a program.

The implementation of the sleep() function will then look something like this:

public static Object sleep(long timeMs) {
  Continuation continuation = new Continuation();
  scheduleEvent(timeMs, () -> continuation.continueExecution());
  throw continuation;
}

As this exception unwinds the call stack, the generated code for each Jactl stack frame catches it, creates its own Continuation to record where it was up to (the location of the call it was waiting on) and the values of its local variables, and then throws a new Continuation chained to the one it just caught. By the time the exception reaches the bottom of the Jactl call stack, the original call stack has been replaced by a chain of Continuation objects, one per frame, that together capture the entire execution state of the script:

For anyone who has done performance tuning of Java applications, the idea of throwing exceptions instantly makes one think of the cost involved, but in reality, the cost of throwing an exception is mostly in the generation of the stack trace that goes along with it. As long as you throw an exception that does not fill in the stack trace, it is actually very efficient.

Async Functions

The Jactl compiler knows which global functions are functions that can perform long-running operations and can throw a Continuation object. These functions are called async functions and the Jactl compiler tracks which methods and functions invoke these async functions and marks these as async as well. This continues up the call chain so the compiler knows at any point in time whether a call could potentially throw a Continuation object, even if the built-in global function that is the one to actually throw the first Continuation is buried many levels within a set of nested calls.

Invoking Async Methods/Functions

When the compiler is generating code that invokes a function that has been flagged as async, it wraps the call in a try/catch that catches any Continuation that is thrown.

The code for the catch block creates a new Continuation object and stores a MethodHandle and a location inside it. The MethodHandle points to the current function and the location is a logical location that records where in the current function the call to the async function that threw the Continuation occurred.

As well as the MethodHandle and location, the compiler generates code to also store the values of the local variables in scope at the time and any values that are currently on the local stack. There are two separate arrays used: a long[] that is used for local variables and stack values that are primitives, and an Object[] that is used for all other types.

Every async function is implicitly passed a Continuation object as its first argument. The first time through, the argument is null, but if the function was suspended due to a long-running operation and is then later resumed, the argument will be non-null and the generated code then uses the location within the Continuation object to work out where in the function to jump to in order to continue execution.

Here is some pseudocode that shows what the generated code from the compiler for a function that invokes another async function might look like:

static MethodHandle processOrderHandle = MethodHandles.lookup().findStatic("processOrder");

Object processOrder(Continuation cont, ...) {
  Order  order;
  Widget widget;
  int    count;
  if (cont != null) {
    // Resume from where we left off after restoring any local variables
    switch (cont.location) {
      case 0:
        // Restore locals
        order  = cont.objArr[0];
        widget = cont.objArr[1];
        count  = cont.longArr[0];
        goto LOCATION_0;
      case 1:
        ...
        goto LOCATION_1;
    }
  }
  // Code for the function
  ...
  try {
    checkInventory(widget, count);
  }
  catch (Continuation c) {
    throw new Continuation(c, processOrderHandle, 
                           0,   // the location
                           new long[]{ count },
                           new Object[]{ order, widget });
  }
 LOCATION_0:
  ...
}

Note that the Continuation constructor chains itself to the just caught Continuation so that the chain starts with the Continuation from the top of the stack.

Resuming Execution

Once a long-running operation completes, the initial Continuation object in the chain is resumed by invoking its continueExecution(Object result) method. This method extracts the MethodHandle and calls it, passing in the Continuation as previously described so that the function can restore any local variable and work out where to continue from.

Since the call stack no longer matches the original call stack, when the function returns, instead of it returning to the original parent function, it will return to the Continuation.continueExecution() method which then extracts the next Continuation object in the chain and calls its MethodHandle. This continues until there are no more Continuation objects in the chain and the registered completion from the application is invoked with the final result.

Invoking a Subsequent Async Function

While walking the chain of Continuation objects and resuming them, another async function may be invoked that throws a new Continuation object for a new long-running operation. When this happens, we take the new chain of continuations and add the remainder of the existing chain to the end of that chain:

When the new long-running operation completes and its Continuation chain is resumed, the chain now consists of all the new continuations as well as the remaining old ones. It will first resume each of the new continuations and then continue on to the remaining ones in the old chain.

Checkpointing Execution State

Once Jactl had the ability to save the current execution state in a chain of continuations, I realised that if these continuations could be serialised into a byte array, I could use this as a way to checkpoint the state of a script. Once a script state has been checkpointed, the state can be persisted to disk, or into a database, or replicated across a network to another application instance. This provides a way to resume a script after the failure of an application host.

For every built-in type and every user defined class, Jactl generates code to store instances of these types into a byte array, along with other types used internally by the Jactl runtime. Jactl then provides hooks that the application can use to persist or replicate these script states as part of a redundancy solution for application state. There is a corresponding Jactl mechanism that the application can use to resume the state when needed. See Checkpointing Proof of Concept for more details and a description of a proof-of-concept implementation of checkpointing for application redundancy.

Conclusion

For reactive applications that need to run on older versions of Java, the Jactl continuation based mechanism for handling blocking operations provides a convenient way for applications to offer customisation via scripting without having to worry about scripts blocking event-loop threads. Scripts can be written with inlined blocking operations in a natural manner without having to pollute the code with async/await or having to deal with Futures or Promises or other mechanisms that programming languages have used to deal with asynchronous code in the past.

From a script point of view, Jactl provides the equivalent programming model as VirtualThreads in Java 21 provides to Java programs.

With modern versions of Java, the Jactl continuation based approach can be disabled and VirtualThreads can be used instead.