I’m really excited about my latest personal side project. After a couple of false starts, I’ve put together microBean™ Jersey Netty Integration, available on Github.
This project inserts Jersey properly and idiomatically into a Netty pipeline without the flaws usually encountered by other projects that attempt to do this (including the experimental one actually supplied by the Jersey project). Let’s take a look at how it works.
The Pipeline
First, a little background.
The first construct to understand in Netty is the ChannelPipeline. A ChannelPipeline is an ordered collection of ChannelHandlers that each react to a kind of event or message and ideally do a single thing as a result. Events flow in from the “left” in a left-to-right direction, are acted upon, and flow back “out” in a right-to-left direction. (In the ChannelPipeline Javadocs, events flow “in” and “up” from the “bottom”, and are written from the “top” and flow “down”.)
ChannelHandlers in the pipeline can be inbound handlers or outbound handlers or both. If a ChannelHandler is an inbound handler, then it participates in the event flow as the events come in left-to-right from the socket. If a ChannelHandler is an outbound handler, then it participates in the event flow as the events go out right-to-left to the socket.
Used normally, ChannelHandlers are invoked by only a single thread, so you are insulated from threading gymnastics when you’re writing one. However, the thread that invokes them is usually the Netty event loop: a thread whose main job is and that is in the process of ferrying bytes to and from a socket. So it’s critical that any work you do that might block this thread gets offloaded elsewhere.
Events are basically messages that are read, and messages that are written. There are other kinds of events, but that’s a good start.
A ChannelPipeline is hooked up to a Channel, which is an abstraction over sockets. So you can see that a socket read ends up flowing left-to-right through the pipeline, and is transformed at some point along the way by a ChannelHandler into a socket write.
I am painting with a very broad brush and only so I can talk about plugging Jersey in to this machinery properly. For more, you really should buy Netty in Action from your favorite local bookstore.
The Pipeline Philosophy
This elegant architecture is very much in the spirit of Unix’s “do one thing and do it well” philosophy, and I’m sure it is not unintentional that a Netty pipeline resembles a Unix pipeline.
In a well-behaved Netty pipeline, a handler near the head of the pipeline is usually performing translation work. It takes handfuls of bytes, and turns them into meaningful objects that can be consumed as events by handlers further on up the pipeline. This act is called decoding, and Netty ships with lots of decoders.
One such decoder is the HttpRequestDecoder, which converts bytes into HttpRequest and HttpContent objects. When this is at the head of the pipeline, then every other inbound handler upstream from it can now wait to receive HttpRequest and HttpContent objects without worrying about how they were put together.
What’s important to notice here is that the HttpRequestDecoder does just one thing: it takes in bytes and transforms them into another message, fires it up the pipeline, and that’s it.
On the writing front, there is, unsurprisingly, an HttpResponseEncoder that accepts requests to write HttpResponses and HttpContent objects, and turns them into bytes suitable for supplying (eventually) to the socket layer. Like its decoding sister, this handler just does translation, but in the other direction.
Of HttpRequests and ContainerRequests
So now we have a pipeline that deals in HttpObjects (HttpRequests and HttpContent objects on the way in, and HttpResponses and HttpContent objects on the way out). That’s nice.
Let’s say we want to put Jersey “into” this pipeline in some fashion. Clearly we’d put it somehow after the decoder on the way in, so it can read HttpObjects, and before the encoder on the way out, so it can also write HttpObjects.
Sadly, however, Jersey does not natively know about Netty objects such as HttpObject, HttpRequest, HttpContent and HttpResponse. To put Jersey in this pipeline, we will have to adapt these Netty objects into the right kind of Jersey objects using other decoders.
Additionally, of course, Jersey exists to run JAX-RS (or Jakarta RESTful Web Services) applications, and we don’t know what those applications are going to do. Remember the bit above where I said that we shouldn’t block the event loop? That comes into play here.
So what are we to do?
Jersey has a container integration layer. In this parlance, a container is the Thing That Hosts Jersey, whatever “hosts” might mean. Many times this is a Servlet engine, such as Jetty, or a full-blown application server, such as Glassfish.
But it doesn’t have to be. (Some people aren’t aware that JAX-RS (or Jakarta RESTful Web Services) does not require Servlet at all! It’s true!)
As it turns out, all you need to get started with Jersey integration is a ContainerRequest. A ContainerRequest is the bridge from whatever is not Jersey, but is hosting it in some way, to that which is Jersey, which will handle it. So as you can see from its constructor, you pass in information that Jersey needs to do its job from wherever you got it, and Jersey takes it from there. In this case, we’ll harvest that information from HttpRequest and HttpContent objects.
Combining Messages
The proper and idiomatic way to do this sort of thing is to further massage our pipeline. Just as Netty ships relatively small handlers that do one thing and do it well, we’re not going to try to cram Jersey integration into a single class. Instead, we want to turn a collection of HttpRequest and HttpContent objects into a ContainerRequest object first. Do one thing and do it well. We’ll worry about what comes after that in a moment.
This decoding is a form of message aggregation. In some cases, we’ll need to combine an initial HttpRequest with a series of HttpContent objects that may follow it into a single ContainerRequest.
Accordingly, microBean™ Jersey Netty Integration ships with a decoder that consumes HttpRequest and HttpContent messages, and emits ContainerRequest messages in their place. This forms the first part of idiomatic Netty-and-Jersey integration.
Creating a ContainerRequest from an HttpRequest is relatively simple. The harder part is deciding whether to let the ContainerRequest under construction fly up the pipeline or not.
For example, some incoming HttpRequests represent simple GET requests for resources, and have no accompanying payload and therefore a Content-Length header of 0. These are easy to deal with: there’s no further content, so translating such an HttpRequest into a ContainerRequest is a one-for-one operation. Grab some fields out of the HttpRequest, use them to call the ContainerRequest constructor, and you’re done. That case is easy.
On the other hand, a POST usually features an incoming payload. The ContainerRequest we’re building will need an InputStream to represent this payload. This case is a little trickier.
Specifically, in Netty, a complete HTTP request is deliberately represented in several pieces: the initial HttpRequest, and then several “follow-on” HttpContent items representing the incoming payload, terminated with a LastHttpContent message indicating the end of the payload. Netty does things this way to avoid consuming lots of memory, and for speed reasons.
You could wait for all such messages to arrive, and only then combine them together, create an InputStream that somehow represents the whole pile of HttpContents, install it, and fire the resulting ContainerRequest up the pipeline.
But that’s a lot of waiting around, and therefore isn’t very efficient: Jersey is going to have to read from the ContainerRequest before it starts writing, so you might as well give it the ability to do that as soon as possible, even if all the readable content isn’t there yet. Remember too that ideally Jersey will ultimately be running on a thread that is not the event loop!
Really what you need to do is hold the ContainerRequest being built for a moment, specifically only until the first HttpContent describing its payload shows up. At that point, you can create a special InputStream that will represent the yet-to-completely-arrive inbound payload, and install it on the ContainerRequest. Then you can let the ContainerRequest fly upstream, attached to this InputStream pipe, even though strictly speaking it’s still incomplete, and wait for incoming HttpRequests to start the process all over again. The special InputStream you install can read the remaining and yet-to-arrive HttpContent pieces later, on demand. We’ll discuss this special InputStream shortly.
This is, of course, what microBean™ Jersey Netty Integration does. This approach means that more things can happen at the same time that would otherwise be the case, and that keeps the event loop from being blocked. In many ways, the special InputStream is the heart of the microBean™ Jersey Netty Integration project.
Encoding
Now we can happily leave all the translation work behind. Because of the beauty of the Netty pipeline architecture, we can now simply trust that at some point there will be a ContainerRequest delivered via a Netty pipeline.
What are we going to do with it?
The quickest possible answer is: we’re going to hand it to an ApplicationHandler via its handle(ContainerRequest) method. That kicks off the Jersey request/response cycle, and we’re done, right?
No. We haven’t discussed writing yet.
It is true that we’re basically done with the reading side of things. We have relied upon HttpRequestDecoder to turn bytes into HttpRequests and HttpContents. We’ve posited a decoder that reads those messages and turns them into a ContainerRequest and emits it. And we know that the final reader in our pipeline will be some sort of inbound ChannelHandler that will accept ContainerRequest instances. Now we need to handle that ContainerRequest and write something back to the caller.
On the Jersey end of things, ContainerRequest contains everything Jersey needs to know about a JAX-RS (or Jakarta RESTful Web Services) Application. Jersey will use its InputStream to read incoming payloads, and will use its ContainerResponseWriter to write outgoing payloads (by way, of course, of following the setup of the user’s Application). We haven’t talked about ContainerResponseWriter yet, but we will now.
ContainerResponseWriter
Once a ContainerRequest gets a ContainerResponseWriter installed on it it is then possible to actually write messages back to the caller from within Jersey. A Jersey application typically relies on finding (by well-established rules) a MessageBodyWriter to encode some kind of Java object relevant to the user’s Application into byte arrays that can then be written by Jersey to an OutputStream. Once the writeTo method has been called, Jersey considers its writing job done.
Of course our job is not done, as now we have to somehow hook this OutputStream up to the outbound part of the Netty channel pipeline. And, of course, recall that our decoder, following the “do one thing and do it well” philosophy, deliberately never installed a ContainerResponseWriter on the ContainerRequest it sent on up the pipeline.
microBean™ Jersey Netty Integration tackles this problem by having an inbound ChannelHandler implementation that is itself a ContainerResponseWriter. It is set up to consume ContainerRequest objects, and, when it receives one, it immediately installs itself as that ContainerRequest‘s ContainerResponseWriter implementation.
To do this, this handler-that-is-also-a-ContainerResponseWriter will need an OutputStream implementation that it can return from the writeResponseStatusAndHeaders method it is required to implement for those cases where a payload is to be sent back to the caller.
The OutputStream implementation that is used functions as its own kind of mini-decoder. It “accepts” byte arrays, as all OutputStreams do, and it “decodes” them into appropriate HttpObjects, namely HttpResponse and HttpContent objects. Along the way, this requires first “decoding” byte arrays into Netty’s native collection-of-bytes container: ByteBuf.
Without spending much time on it, a ByteBuf is in some ways the lowest-level object in Netty. If you have a ByteBuf in your hand, decoding it into some other Netty-supplied object is usually pretty trivial. In this case, creating a DefaultHttpContent out of a ByteBuf is as simple as invoking a constructor.
Getting a ByteBuf from an array of bytes is also straightforward: just use the Unpooled#wrappedBuffer method! So on every write to this OutputStream implementation you are effectively emitting HttpContent objects of a mostly predictable size.
Next, the OutputStream implementation does not, obviously, “point at” a file or any other kind of traditional destination you might be useful. Instead, it wraps a ChannelOutboundInvoker implementation. A ChannelOutboundInvoker implementation, such as a ChannelHandlerContext, is the “handle” that you use to send a write message up the Netty channel pipeline. So every OutputStream#write operation becomes a ChannelOutboundInvoker#write operation.
Finally, you want the OutputStream implementation that “consumes” byte arrays and writes HttpObjects of the right kind to do so without necessarily waiting for all the content that Jersey might write over the stream before sending it up the pipeline. So the OutputStream implementation in question automatically calls its own flush() method after a configurable number of bytes have been written. The OutputStream, in other words, is auto-flushing. (Unless you don’t want it to be!)
About that flush() method: it’s mapped to (you guessed it) ChannelOutboundInvoker#flush.
So now we have connected the dots: a ContainerRequest goes to Jersey, which processes it. Jersey writes to an OutputStream we provide that itself bridges to the actual Netty channel pipeline. And downstream in the pipeline we have a Netty-supplied HttpResponseEncoder that accepts the HttpResponse and HttpContent objects we emit.
Threading
Now let’s talk about threads. In Netty, as noted, events coming up the pipeline—events being read, events being processed by ChannelInboundHandlers—are executed by the event loop: a thread that is devoted to processing what amount to socket events. It is very important to let these event loop threads run as fast and free as possible. We’ve already talked about how a Jersey application should not, therefore, be run on the event loop, because you don’t know what it is going to do. Will it sleep? Will it run a monstrous blocking database query? You don’t know. More concretely, this means that therefore our ChannelInboundHandler-that-is-also-a-ContainerResponseWriter must not execute ApplicationHandler#handle(ContainerRequest) on the event loop.
Most other projects that integrate Jersey with Netty in some way use a thread pool or Jersey’s own schedulers to do this. But they overlook the fact that Netty lets you do this more natively. This native approach is the one that microBean™ Jersey Netty Integration has taken.
First, let’s just note that the only mention that we’ve made so far of anything in the integration machinery that could block is the special InputStream that is supplied to a ContainerRequest as its entity stream. We mentioned that this InputStream gets installed on a ContainerRequest as a kind of empty pipe before (potentially) the entire incoming payload has been received. Therefore, Jersey might start reading from it before there are bytes to read, and indeed, that InputStream implementation will block in that case, by contract, until the downstream (or “leftstream”) decoder catches up and fills the pipe with other HttpContent messages.
But you’ll note that otherwise we haven’t made mention of anything like a ConcurrentHashMap or a LinkedBlockingQueue or anything from the java.util.concurrent.locks package. That’s on purpose. To understand how microBean™ Jersey Netty Integration gets away with this minimal use of blocking constructs, we have to revisit the ChannelPipeline.
When you add a ChannelHandler to a pipeline—when, as a Netty developer, you build your pipeline in the first place—you typically use various flavors of the ChannelPipeline#addLast method. This appends your ChannelHandler in question to the tail of the pipeline as you might expect. And then all the event handling we’ve talked about flows through the pipeline in the way that we’ve talked about it.
But note that there’s another form of the ChannelPipeline#addLast method that takes an EventExecutorGroup!
In this form, if you supply a new DefaultEventExecutorGroup as you add a ChannelHandler, then its threads will be those that run your ChannelHandler‘s event-handling methods, and not those of the event loop! So all you have to do to get the Jersey ApplicationHandler#handle(ContainerRequest) method to be run on a non-event loop thread is to set up your pipeline using this variant of the ChannelPipeline#addLast method, supplying a DefaultEventExecutorGroup. Then whatever the JAX-RS or Jakarta RESTful Web Services Application does (slow database access, Thread.sleep() calls…) will not block the event loop.
Now, another tenet of the Netty framework is that Thou Shalt Not Write to the Pipeline Except on the Event Loop. So if our ApplicationHandler#handle(ContainerRequest) method is being run on a non-event-loop thread, then don’t we have to do something to “get back on” the event loop thread when our OutputStream implementation calls ChannelOutboundInvoker#write(Object, ChannelPromise)?
As it turns out, no, because since a ChannelOutboundInvoker‘s whole job is to “do” IO, it always ensures that these operations take place on the event loop. In other words, even though our Jersey application is correctly running its workload on one non-event-loop thread, when our special OutputStream invokes ChannelOutboundInvoker#write(Object, ChannelPromise), the implementation of that method will ensure that the write takes place on the event loop by enqueuing a task on the event loop for us.
To put it one final other way, if you have introduced queues of any kind or homegrown thread pools into your Netty integration machinery—other than the minimal amount of blocking necessary for adapting incoming entity payloads into InputStreams as previously discussed—you’re doing it wrong, because Netty already has them.
Conclusion
There is a lot more to this library than I’ve covered here, including HTTP/2 support. I encourage you to take a look at its Github repository and get involved. Thanks for reading!
Blame Laird 