Ramblings on Java: Glassfish Performance

A while back I wrote a test application that pits CORBA (using Java's ORB), EJB 3.0 (using Glassfish), and sockets against one another. For the EJB 3.0 test, this involves accessing stateless session beans on the server.

The client is a fairly simple application that spawns a given number of threads where each one requests data of varying size from the server. The client threads each do this as many times as they can in a given interval and then the results are tallied. The interval is broken down into two parts. A warm up period and a actual period. EJB in particular seems to show better results after a nice warmup period. I've run this test several times since early this year and have noted some steady improvement in the performance of Glassfish (the CORBA and the sockets test remained flat as their server side implementations did not change).

Here are the results for a relatively short test. Only the EJB results are provided as those are the ones we are interested in here. The sizes of data packets are dived into five categories:

TINY = 1000 bytes
SMALL = 5000 bytes
MEDIUM = 10000 bytes
LARGE = 20000 bytes
XLARGE = 50000 bytes

As a side note, Glassfish builds prior to 36 had a timeout problem (Exceeded 1,800,000 milliseconds) so you cannot run extended period tests on older versions.

On Glassfish 36
Thread_1 found 22159 warmup and 35631 actual Tiny datapoints

Thread_4 found 19889 warmup and 31647 actual Tiny datapoints

Thread_0 found 20032 warmup and 31655 actual Tiny datapoints

Thread_3 found 22300 warmup and 35617 actual Tiny datapoints

Thread_2 found 20031 warmup and 31575 actual Tiny datapoints

Thread_0 found 11822 warmup and 18273 actual Small datapoints

Thread_1 found 12950 warmup and 19549 actual Small datapoints

Thread_4 found 11787 warmup and 18100 actual Small datapoints

Thread_3 found 13011 warmup and 19518 actual Small datapoints

Thread_2 found 11815 warmup and 18168 actual Small datapoints

Thread_3 found 6069 warmup and 9687 actual Medium datapoints

Thread_2 found 5692 warmup and 9080 actual Medium datapoints

Thread_4 found 5693 warmup and 9039 actual Medium datapoints

Thread_0 found 5699 warmup and 9084 actual Medium datapoints

Thread_1 found 6081 warmup and 9609 actual Medium datapoints

Thread_4 found 3268 warmup and 5399 actual Large datapoints

Thread_3 found 3300 warmup and 5507 actual Large datapoints

Thread_0 found 3222 warmup and 5392 actual Large datapoints

Thread_1 found 3329 warmup and 5460 actual Large datapoints

Thread_2 found 3263 warmup and 5361 actual Large datapoints

Thread_4 found 1377 warmup and 2233 actual XLarge datapoints

Thread_0 found 1370 warmup and 2227 actual XLarge datapoints

Thread_1 found 1373 warmup and 2225 actual XLarge datapoints

Thread_3 found 1382 warmup and 2219 actual XLarge datapoints

Thread_2 found 1358 warmup and 2226 actual XLarge datapoints

On Glassfish Build 41
Thread_2 found 23514 warmup and 38000 actual Tiny datapoints

Thread_4 found 23581 warmup and 37653 actual Tiny datapoints

Thread_1 found 24727 warmup and 39992 actual Tiny datapoints

Thread_3 found 24543 warmup and 40005 actual Tiny datapoints

Thread_0 found 23553 warmup and 38002 actual Tiny datapoints

Thread_1 found 12854 warmup and 19917 actual Small datapoints

Thread_2 found 12385 warmup and 19169 actual Small datapoints

Thread_3 found 12968 warmup and 19798 actual Small datapoints

Thread_0 found 12497 warmup and 19201 actual Small datapoints

Thread_4 found 12498 warmup and 19215 actual Small datapoints

Thread_2 found 6024 warmup and 9630 actual Medium datapoints

Thread_1 found 6256 warmup and 10028 actual Medium datapoints

Thread_3 found 6229 warmup and 10038 actual Medium datapoints

Thread_0 found 6101 warmup and 9615 actual Medium datapoints

Thread_4 found 6045 warmup and 9636 actual Medium datapoints

Thread_2 found 3238 warmup and 5392 actual Large datapoints

Thread_3 found 3369 warmup and 5678 actual Large datapoints

Thread_0 found 3224 warmup and 5395 actual Large datapoints

Thread_4 found 3237 warmup and 5372 actual Large datapoints

Thread_1 found 3353 warmup and 5622 actual Large datapoints

Thread_2 found 1288 warmup and 2114 actual XLarge datapoints

Thread_3 found 1333 warmup and 2154 actual XLarge datapoints

Thread_0 found 1293 warmup and 2123 actual XLarge datapoints

Thread_4 found 1289 warmup and 2128 actual XLarge datapoints

Thread_1 found 1324 warmup and 2170 actual XLarge datapoints

Although many test runs need to be conducted to weed out anomalies, from this set of data we can see that the newer build clearly had an improvement in performance in Tiny, Small, and Medium packets. By the time we got to the large packets the old version was about as fast and even showed an edge in the XLarge performance criteria. Here it is graphically:

Hmmm...have the Glassfish folks done some tuning as of late? Might it be possible to someday set an option that allows you to tune your installation? For example, some applications deal primarily with returning small data packets of text based data while others deal in larger binary files (big images for example). If you knew the size of the data your app server dealt with, then perhaps there might be a way to tune this on an install by install basis.

When some more app servers suport EJB 3 (note that Oracle's does now as well), I'll run these tests again and pit Glassfish against them. From what I've seen with it so far though, my thought is that Glassfish will do more than hold its own...in fact, Glassfish may just have a little piranha in it and eat their lunch! We'll see.