I’m really excited about my latest personal side project. After a couple of false starts, I’ve put together microBean™ Jersey Netty Integration, available on Github.
This project inserts Jersey properly and idiomatically into a Netty pipeline without the flaws usually encountered by other projects that attempt to do this (including the experimental one actually supplied by the Jersey project). Let’s take a look at how it works.
First, a little background.
The first construct to understand in Netty is the
ChannelPipeline is an ordered collection of
ChannelHandlers that each react to a kind of event or message and ideally do a single thing as a result. Events flow in from the “left” in a left-to-right direction, are acted upon, and flow back “out” in a right-to-left direction. (In the
ChannelPipeline Javadocs, events flow “in” and “up” from the “bottom”, and are written from the “top” and flow “down”.)
ChannelHandlers in the pipeline can be inbound handlers or outbound handlers or both. If a
ChannelHandler is an inbound handler, then it participates in the event flow as the events come in left-to-right from the socket. If a
ChannelHandler is an outbound handler, then it participates in the event flow as the events go out right-to-left to the socket.
ChannelHandlers are invoked by only a single thread, so you are insulated from threading gymnastics when you’re writing one. However, the thread that invokes them is usually the Netty event loop: a thread whose main job is and that is in the process of ferrying bytes to and from a socket. So it’s critical that any work you do that might block this thread gets offloaded elsewhere.
Events are basically messages that are read, and messages that are written. There are other kinds of events, but that’s a good start.
ChannelPipeline is hooked up to a
Channel, which is an abstraction over sockets. So you can see that a socket read ends up flowing left-to-right through the pipeline, and is transformed at some point along the way by a
ChannelHandler into a socket write.
I am painting with a very broad brush and only so I can talk about plugging Jersey in to this machinery properly. For more, you really should buy Netty in Action from your favorite local bookstore.
The Pipeline Philosophy
This elegant architecture is very much in the spirit of Unix’s “do one thing and do it well” philosophy, and I’m sure it is not unintentional that a Netty pipeline resembles a Unix pipeline.
In a well-behaved Netty pipeline, a handler near the head of the pipeline is usually performing translation work. It takes handfuls of bytes, and turns them into meaningful objects that can be consumed as events by handlers further on up the pipeline. This act is called decoding, and Netty ships with lots of decoders.
One such decoder is the
HttpRequestDecoder, which converts bytes into
HttpContent objects. When this is at the head of the pipeline, then every other inbound handler upstream from it can now wait to receive
HttpContent objects without worrying about how they were put together.
What’s important to notice here is that the
HttpRequestDecoder does just one thing: it takes in bytes and transforms them into another message, fires it up the pipeline, and that’s it.
On the writing front, there is, unsurprisingly, an
HttpResponseEncoder that accepts requests to write
HttpContent objects, and turns them into bytes suitable for supplying (eventually) to the socket layer. Like its decoding sister, this handler just does translation, but in the other direction.
So now we have a pipeline that deals in
HttpContent objects on the way in, and
HttpContent objects on the way out). That’s nice.
Let’s say we want to put Jersey “into” this pipeline in some fashion. Clearly we’d put it somehow after the decoder on the way in, so it can read
HttpObjects, and before the encoder on the way out, so it can also write
Sadly, however, Jersey does not natively know about Netty objects such as
HttpResponse. To put Jersey in this pipeline, we will have to adapt these Netty objects into the right kind of Jersey objects using other decoders.
Additionally, of course, Jersey exists to run JAX-RS (or Jakarta RESTful Web Services) applications, and we don’t know what those applications are going to do. Remember the bit above where I said that we shouldn’t block the event loop? That comes into play here.
So what are we to do?
Jersey has a container integration layer. In this parlance, a container is the Thing That Hosts Jersey, whatever “hosts” might mean. Many times this is a Servlet engine, such as Jetty, or a full-blown application server, such as Glassfish.
But it doesn’t have to be. (Some people aren’t aware that JAX-RS (or Jakarta RESTful Web Services) does not require Servlet at all! It’s true!)
As it turns out, all you need to get started with Jersey integration is a
ContainerRequest is the bridge from whatever is not Jersey, but is hosting it in some way, to that which is Jersey, which will handle it. So as you can see from its constructor, you pass in information that Jersey needs to do its job from wherever you got it, and Jersey takes it from there. In this case, we’ll harvest that information from
The proper and idiomatic way to do this sort of thing is to further massage our pipeline. Just as Netty ships relatively small handlers that do one thing and do it well, we’re not going to try to cram Jersey integration into a single class. Instead, we want to turn a collection of
HttpContent objects into a
ContainerRequest object first. Do one thing and do it well. We’ll worry about what comes after that in a moment.
This decoding is a form of message aggregation. In some cases, we’ll need to combine an initial
HttpRequest with a series of
HttpContent objects that may follow it into a single
Accordingly, microBean™ Jersey Netty Integration ships with a decoder that consumes
HttpContent messages, and emits
ContainerRequest messages in their place. This forms the first part of idiomatic Netty-and-Jersey integration.
ContainerRequest from an
HttpRequest is relatively simple. The harder part is deciding whether to let the
ContainerRequest under construction fly up the pipeline or not.
For example, some incoming
HttpRequests represent simple
GET requests for resources, and have no accompanying payload and therefore a
Content-Length header of
0. These are easy to deal with: there’s no further content, so translating such an
HttpRequest into a
ContainerRequest is a one-for-one operation. Grab some fields out of the
HttpRequest, use them to call the
ContainerRequest constructor, and you’re done. That case is easy.
On the other hand, a
POST usually features an incoming payload. The
ContainerRequest we’re building will need an
InputStream to represent this payload. This case is a little trickier.
Specifically, in Netty, a complete HTTP request is deliberately represented in several pieces: the initial
HttpRequest, and then several “follow-on”
HttpContent items representing the incoming payload, terminated with a
LastHttpContent message indicating the end of the payload. Netty does things this way to avoid consuming lots of memory, and for speed reasons.
You could wait for all such messages to arrive, and only then combine them together, create an
InputStream that somehow represents the whole pile of
HttpContents, install it, and fire the resulting
ContainerRequest up the pipeline.
But that’s a lot of waiting around, and therefore isn’t very efficient: Jersey is going to have to read from the
ContainerRequest before it starts writing, so you might as well give it the ability to do that as soon as possible, even if all the readable content isn’t there yet. Remember too that ideally Jersey will ultimately be running on a thread that is not the event loop!
Really what you need to do is hold the
ContainerRequest being built for a moment, specifically only until the first
HttpContent describing its payload shows up. At that point, you can create a special
InputStream that will represent the yet-to-completely-arrive inbound payload, and install it on the
ContainerRequest. Then you can let the
ContainerRequest fly upstream, attached to this
InputStream pipe, even though strictly speaking it’s still incomplete, and wait for incoming
HttpRequests to start the process all over again. The special
InputStream you install can read the remaining and yet-to-arrive
HttpContent pieces later, on demand. We’ll discuss this special
This is, of course, what microBean™ Jersey Netty Integration does. This approach means that more things can happen at the same time that would otherwise be the case, and that keeps the event loop from being blocked. In many ways, the special
InputStream is the heart of the microBean™ Jersey Netty Integration project.
Now we can happily leave all the translation work behind. Because of the beauty of the Netty pipeline architecture, we can now simply trust that at some point there will be a
ContainerRequest delivered via a Netty pipeline.
What are we going to do with it?
The quickest possible answer is: we’re going to hand it to an
ApplicationHandler via its
handle(ContainerRequest) method. That kicks off the Jersey request/response cycle, and we’re done, right?
No. We haven’t discussed writing yet.
It is true that we’re basically done with the reading side of things. We have relied upon
HttpRequestDecoder to turn bytes into
HttpContents. We’ve posited a decoder that reads those messages and turns them into a
ContainerRequest and emits it. And we know that the final reader in our pipeline will be some sort of inbound
ChannelHandler that will accept
ContainerRequest instances. Now we need to handle that
ContainerRequest and write something back to the caller.
On the Jersey end of things,
ContainerRequest contains everything Jersey needs to know about a JAX-RS (or Jakarta RESTful Web Services)
Application. Jersey will use its
InputStream to read incoming payloads, and will use its
ContainerResponseWriter to write outgoing payloads (by way, of course, of following the setup of the user’s
Application). We haven’t talked about
ContainerResponseWriter yet, but we will now.
ContainerRequest gets a
ContainerResponseWriter installed on it it is then possible to actually write messages back to the caller from within Jersey. A Jersey application typically relies on finding (by well-established rules) a
MessageBodyWriter to encode some kind of Java object relevant to the user’s
byte arrays that can then be written by Jersey to an
OutputStream. Once the
writeTo method has been called, Jersey considers its writing job done.
Of course our job is not done, as now we have to somehow hook this
OutputStream up to the outbound part of the Netty channel pipeline. And, of course, recall that our decoder, following the “do one thing and do it well” philosophy, deliberately never installed a
ContainerResponseWriter on the
ContainerRequest it sent on up the pipeline.
microBean™ Jersey Netty Integration tackles this problem by having an inbound
ChannelHandler implementation that is itself a
ContainerResponseWriter. It is set up to consume
ContainerRequest objects, and, when it receives one, it immediately installs itself as that
To do this, this handler-that-is-also-a-
ContainerResponseWriter will need an
OutputStream implementation that it can return from the
writeResponseStatusAndHeaders method it is required to implement for those cases where a payload is to be sent back to the caller.
OutputStream implementation that is used functions as its own kind of mini-decoder. It “accepts”
byte arrays, as all
OutputStreams do, and it “decodes” them into appropriate
HttpContent objects. Along the way, this requires first “decoding”
byte arrays into Netty’s native collection-of-
Without spending much time on it, a
ByteBuf is in some ways the lowest-level object in Netty. If you have a
ByteBuf in your hand, decoding it into some other Netty-supplied object is usually pretty trivial. In this case, creating a
DefaultHttpContent out of a
ByteBuf is as simple as invoking a constructor.
ByteBuf from an array of
bytes is also straightforward: just use the
Unpooled#wrappedBuffer method! So on every write to this
OutputStream implementation you are effectively emitting
HttpContent objects of a mostly predictable size.
OutputStream implementation does not, obviously, “point at” a file or any other kind of traditional destination you might be useful. Instead, it wraps a
ChannelOutboundInvoker implementation. A
ChannelOutboundInvoker implementation, such as a
ChannelHandlerContext, is the “handle” that you use to send a write message up the Netty channel pipeline. So every
OutputStream#write operation becomes a
Finally, you want the
OutputStream implementation that “consumes”
byte arrays and writes
HttpObjects of the right kind to do so without necessarily waiting for all the content that Jersey might write over the stream before sending it up the pipeline. So the
OutputStream implementation in question automatically calls its own
flush() method after a configurable number of bytes have been written. The
OutputStream, in other words, is auto-flushing. (Unless you don’t want it to be!)
flush() method: it’s mapped to (you guessed it)
So now we have connected the dots: a
ContainerRequest goes to Jersey, which processes it. Jersey writes to an
OutputStream we provide that itself bridges to the actual Netty channel pipeline. And downstream in the pipeline we have a Netty-supplied
HttpResponseEncoder that accepts the
HttpContent objects we emit.
Now let’s talk about threads. In Netty, as noted, events coming up the pipeline—events being read, events being processed by
ChannelInboundHandlers—are executed by the event loop: a thread that is devoted to processing what amount to socket events. It is very important to let these event loop threads run as fast and free as possible. We’ve already talked about how a Jersey application should not, therefore, be run on the event loop, because you don’t know what it is going to do. Will it sleep? Will it run a monstrous blocking database query? You don’t know. More concretely, this means that therefore our
ContainerResponseWriter must not execute
ApplicationHandler#handle(ContainerRequest) on the event loop.
Most other projects that integrate Jersey with Netty in some way use a thread pool or Jersey’s own schedulers to do this. But they overlook the fact that Netty lets you do this more natively. This native approach is the one that microBean™ Jersey Netty Integration has taken.
First, let’s just note that the only mention that we’ve made so far of anything in the integration machinery that could block is the special
InputStream that is supplied to a
ContainerRequest as its entity stream. We mentioned that this
InputStream gets installed on a
ContainerRequest as a kind of empty pipe before (potentially) the entire incoming payload has been received. Therefore, Jersey might start reading from it before there are bytes to read, and indeed, that
InputStream implementation will block in that case, by contract, until the downstream (or “leftstream”) decoder catches up and fills the pipe with other
But you’ll note that otherwise we haven’t made mention of anything like a
ConcurrentHashMap or a
LinkedBlockingQueue or anything from the
java.util.concurrent.locks package. That’s on purpose. To understand how microBean™ Jersey Netty Integration gets away with this minimal use of blocking constructs, we have to revisit the
When you add a
ChannelHandler to a pipeline—when, as a Netty developer, you build your pipeline in the first place—you typically use various flavors of the
ChannelPipeline#addLast method. This appends your
ChannelHandler in question to the tail of the pipeline as you might expect. And then all the event handling we’ve talked about flows through the pipeline in the way that we’ve talked about it.
But note that there’s another form of the
ChannelPipeline#addLast method that takes an
In this form, if you supply a new
DefaultEventExecutorGroup as you add a
ChannelHandler, then its threads will be those that run your
ChannelHandler‘s event-handling methods, and not those of the event loop! So all you have to do to get the Jersey
ApplicationHandler#handle(ContainerRequest) method to be run on a non-event loop thread is to set up your pipeline using this variant of the
ChannelPipeline#addLast method, supplying a
DefaultEventExecutorGroup. Then whatever the JAX-RS or Jakarta RESTful Web Services
Application does (slow database access,
Thread.sleep() calls…) will not block the event loop.
Now, another tenet of the Netty framework is that Thou Shalt Not Write to the Pipeline Except on the Event Loop. So if our
ApplicationHandler#handle(ContainerRequest) method is being run on a non-event-loop thread, then don’t we have to do something to “get back on” the event loop thread when our
OutputStream implementation calls
As it turns out, no, because since a
ChannelOutboundInvoker‘s whole job is to “do” IO, it always ensures that these operations take place on the event loop. In other words, even though our Jersey application is correctly running its workload on one non-event-loop thread, when our special
ChannelOutboundInvoker#write(Object, ChannelPromise), the implementation of that method will ensure that the write takes place on the event loop by enqueuing a task on the event loop for us.
To put it one final other way, if you have introduced queues of any kind or homegrown thread pools into your Netty integration machinery—other than the minimal amount of blocking necessary for adapting incoming entity payloads into
InputStreams as previously discussed—you’re doing it wrong, because Netty already has them.
There is a lot more to this library than I’ve covered here, including HTTP/2 support. I encourage you to take a look at its Github repository and get involved. Thanks for reading!