long-running operations in Tomcat

Question

Are there any suggested practices for implementing long-running operations in Tomcat?

e.g. is it better to just run the operation and not yield a response until the operation is complete? Or is it better to return immediately with some kind of ticket that can be used to poll for completion? (is there a way to notify a client that completion has occurred?)

Answer 1

For the sake of your server not falling down, and for the sake of not frustrating your user, it is definitely preferable to immediately return with, as you suggest, a "ticket" of some kind, and then to poll periodically for completion of that ticket.

If it's a very long-running operation, you might consider sending an email (such as Vimeo does, for example, when they've re-encoded your video).

Answer 2

Depend on how long you operation will be executed. I'm use both methods depend on situation. I'm have a servlet for usual browser, that wait several seconds and all ok, but another servlet for accessing from PDA fail on long timeouts and i'm return response imm. after putting request into queue. In summary i'm think second way better in general.

Answer 3

If you are using a Servlet 3.0 conformant server (like Tomcat) then you can get an AsyncContext instance with ServletRequest.startAsync() or (in some servers) ServletRequest.getAsyncContext() (use recommended way to avoid automagically created threads).

After getting AsyncContext store it somewhere in your app and return inmediatly from the server request thread call immediately so all non-necesary resources are released. Then you can decide how to manage thousands of parallel connections in a more lightweight way avoiding the bottlenecks of using classic per-connection thread.

• https://docs.oracle.com/javaee/6/api/javax/servlet/ServletRequest.html#startAsync()
• https://docs.oracle.com/javaee/6/api/javax/servlet/ServletRequest.html#getAsyncContext()

You must check also your concrete server's version documentation about parallelism and AsyncContext usage and do lots of trial and error for tuning because some servers recomends specific ways to approach the problem and it can change even between versions. Be careful with testing when server is updated and use LTS always.

There are some caveats while using AsyncContext on a thread-light environment like output stream can block the calling thread... which would normally break all the magic. There are some tricks like the ones described at Servlet-3 Async Context, how to do asynchronous writes? but it can depend on server implementation... the always-work solution is use a thread pool worker for writing outputs if no other works-for-you... with maybe a few dozen of threads you can serve thousands of connections.

I see this question by chance while trying to remember the methods names and classes so this answer is a bit late... and currently there are other more simple and direct Java solutions like Jetty, Netty, ... but if somebody needs to use a corporative "classic" server like Tomcat this may help as long as AsyncContext exists since Servlet 3.0 (which is pretty old and should be supported).