Writing simpler reactive REST services with Quarkus Virtual Thread support

When the number of maximum threads has been reached, each subsequent request will need to wait for a thread to be released to fulfill that request. This can cause slowness or even errors in the application when it experiences high traffic. In this scenario thread pool limit and vertx worker pool size were both set to 300 threads.

Virtual threads always have the normal priority and the priority cannot be changed, even with setPriority method. In Java, Virtual threads (JEP-425) are JVM-managed lightweight threads that will help in writing high-throughput concurrent applications . And of course, there would have to be some actual I/O or other thread parking for Loom to bring benefits.

Threads

The process of loading a virtual thread’s data stack into the “carrier” thread stack when it gets assigned to its carrier, is called mounting. How you are migrating from reactive code to virtual threads depends on you and your project. If your project is modularized properly, then it would probably be a good idea to choose the modularized approach and migrate one module after the other. Most JVM implementations today implement Java threads as thin direct wrappers around operating system threads, called sometimes heavyweight, OS-managed threads platform threads. Launching 9000 platform threads didn’t really show much difference, the run time was the same, but one million threads test took eleven seconds which is more than double the time compared to virtual threads.

Java’s way of threading boils down to utilizing operating system threads. It means that each thread created in the VM has its system thread executing the Thread.run() method, respectively. Almost every blog post on the first page of Google surrounding JDK 19 copied the following text, describing virtual threads, verbatim. You can use virtual threads when you run your code using this configuration. On the first line, we create a virtual thread factory that will handle the thread creation for the executor.

Blocking in Reactive code

Project Loom’s virtual threads improved the performance of Java on the Eclipse Jetty web server in tests. Blocking a virtual thread results in yielding its continuation and unparking it results in the continuation being resubmitted to the scheduler. Because the Java runtime can explicitly control when a virtual thread is suspended and resumed, that structure allows a more fine-grained concurrency model for Java. When the Task object is passed to virtual threads for execution, we will get an output similar to the following.

Many server frameworks will choose to do this automatically, starting a new virtual thread for every incoming request and running the application’s business logic in it. Virtual threads, available as a preview release in JDK 19 and 20, are lightweight, inexpensive, and easy to create. They are tied to a platform thread that is connected to the operating system thread. Consequently, we no longer tie up platform threads in our applications and can handle more concurrent requests.

How to Create a Virtual Thread

Existing agents that enable the ThreadStart and ThreadEnd events may encounter performance issues since they lack the ability to limit the events to platform threads. / d
Littles Law doesnt care about what portion of the time is spent “doing work” vs “waiting”, or whether the unit of concurrency is a thread, a CPU, an ATM machine, or a human bank teller. It just states that to scale up the throughput, we either have to proportionally scale down the latency or scale up the number of requests we can handle concurrently. When we hit the limit on concurrent threads, the throughput of the thread-per-task model is limited by Littles Law. Virtual threads address this in a graceful way by giving us more concurrent threads rather than asking us to change our programming model.

• The asynchronous APIs do not wait for the response, rather they work through the callbacks.
• To run code in a virtual thread, the Java runtime arranges for it to run by mounting it on some platform thread, called a carrier thread.
• Therefore platform thread can handle other requests leading to high throughput.
• Structured concurrency can help simplify the multi-threading or parallel processing use cases and make them less fragile and more maintainable.

This is because we are provided with an illusion of having an almost unlimited number of threads available , in a similar way to what virtual memory does. The creation of Java threads is expensive because it involves allocating memory for the thread, initialising the thread stack and also making OS calls to register the OS thread. If your application has clearly separated modules , you can start by rewriting one module after another. If you are using microservices, you should be able to rewrite one microservice at a time without impacting other microservices. Then, you would block for called reactive code while the code calling your code treats your rewritten code as non-reactive code.

A point about performance

Note that after using the virtual threads, our application may be able to handle millions of threads, but other systems or platforms handle only a few requests at a time. For example, we can have only a few database connections or network connections to other servers. Next, we will replace the Executors.newFixedThreadPool with Executors.newVirtualThreadPerTaskExecutor(). This will execute all the tasks in virtual threads instead of platform threads. Similar to traditional threads, a virtual thread is also an instance of java.lang.Thread that runs its code on an underlying OS thread, but it does not block the OS thread for the code’s entire lifetime. Keeping the OS threads free means that many virtual threads can run their Java code on the same OS thread, effectively sharing it.

Creating such platform threads has always been costly , so Java has been using the thread pools to avoid the overhead in thread creation. It’s available since Java 19 in September 2022 as a preview feature. Its goal is to dramatically reduce the effort of writing, maintaining, and observing high-throughput concurrent applications.

API and platform changes

Note that I also set the compiler to use Java 19 and added a line to the .mvn/jvm.config. Mala has authored multiple books with Manning, Packt, and O’Reilly Publications. An international java loom virtual threads speaker at industry conferences, she has more than 2 decades’ experience in the software industry. She has been actively supporting Java certification as a path to career advancement.

Lastly, but not less important, writing explicit asynchronous code is more error-prone due to the complexities that come with it. In Java 19 were introduced Virtual Threads JEP-425 as a preview feature. I will also point out that, as currently proposed, Structured Concurrency must be opted into by the user deciding to take advantage of the API, so it won’t solve issues on its own.