Objective

Comparing the performance of GT3 security with SSL.

GT3 implements security at the message level: it encrypts and/or signs the XML message sent from the client to the server. The transport is still normal unsecure HTTP. EGEE, instead, implements security at the connection level: it sends unsecure XML messages over a secure connection. The second approach is also suitable when there is not XML messaging (i.e. web interfaces).

Configuration

Tests run on a double PIII 450MHz, JDK 1.4.2, Tomcat 5.0.28 Client and server running on the same machine. The server is just a normal Tomcat 5 server. We just test the transport, so there are no web services involved, just a normal JSP. We compare the same server configured in two ways: one with http, and the other with https using the EGEE trustmanager.

The setups are different, so we can't just compare the numbers. But we can compare the differences between the unsecure and secure setup. But we can compare the overhead put by message level security and transport level security.

No security

First test:

Time from 1 to 100: 1049ms 10.0ms/request
Time from 101 to 200: 626ms 6.0ms/request
Time from 201 to 300: 782ms 7.0ms/request
Time from 301 to 400: 617ms 6.0ms/request
Time from 401 to 500: 644ms 6.0ms/request
Time from 501 to 600: 575ms 5.0ms/request
Time from 601 to 700: 656ms 6.0ms/request
Time from 701 to 800: 569ms 5.0ms/request
Time from 801 to 900: 483ms 4.0ms/request
Time from 901 to 1000: 500ms 5.0ms/request
Time from 1001 to 1100: 478ms 4.0ms/request
Time from 1101 to 1200: 486ms 4.0ms/request
Time from 1201 to 1300: 490ms 4.0ms/request
Time from 1301 to 1400: 527ms 5.0ms/request
Time from 1401 to 1500: 1161ms 11.0ms/request
Time from 1501 to 1600: 490ms 4.0ms/request
Time from 1601 to 1700: 456ms 4.0ms/request
Time from 1701 to 1800: 427ms 4.0ms/request
Time from 1801 to 1900: 494ms 4.0ms/request
Time from 1901 to 2000: 437ms 4.0ms/request
Time from 2001 to 2100: 444ms 4.0ms/request
Time from 2101 to 2200: 477ms 4.0ms/request
Time from 2201 to 2300: 409ms 4.0ms/request
Time from 2301 to 2400: 411ms 4.0ms/request
Time from 2401 to 2500: 415ms 4.0ms/request
Time from 2501 to 2600: 413ms 4.0ms/request
Time from 2601 to 2700: 587ms 5.0ms/request
Time from 2701 to 2800: 439ms 4.0ms/request
Time from 2801 to 2900: 451ms 4.0ms/request
Time from 2901 to 3000: 441ms 4.0ms/request
Time from 3001 to 3100: 408ms 4.0ms/request
Time from 3101 to 3200: 411ms 4.0ms/request
Time from 3201 to 3300: 415ms 4.0ms/request
Time from 3301 to 3400: 413ms 4.0ms/request
Time from 3401 to 3500: 411ms 4.0ms/request
Time from 3501 to 3600: 416ms 4.0ms/request
Time from 3601 to 3700: 429ms 4.0ms/request
Time from 3701 to 3800: 420ms 4.0ms/request
Time from 3801 to 3900: 414ms 4.0ms/request

The timing stabilizes around 4.0ms/request. As in the old test, we run two clients on the same machine to achieve 100% CPU usage, and the timing becomes 4.5ms/request.

The first few requests are:

Time from 1 to 1: 937ms 937.0ms/request
Time from 2 to 2: 16ms 16.0ms/request
Time from 3 to 3: 8ms 8.0ms/request
Time from 4 to 4: 19ms 19.0ms/request
Time from 5 to 5: 8ms 8.0ms/request
Time from 6 to 6: 20ms 20.0ms/request
Time from 7 to 7: 10ms 10.0ms/request
Time from 8 to 8: 9ms 9.0ms/request
Time from 9 to 9: 27ms 27.0ms/request
Time from 10 to 10: 13ms 13.0ms/request
Time from 11 to 11: 12ms 12.0ms/request
Time from 12 to 12: 10ms 10.0ms/request

The key numbers are:

First request time0.9 sec
Avg. request time (after JIT)4.0 ms/req
Overall server frequency444 req/sec

As expected, the performance is a bit higher than with GT3 web services, since there is no XML parsing. The startup time is far less, since a lot less libraries need to be loaded.

With security

After turning on SSL: 

Time from 1 to 100: 2862ms 28.0ms/request
Time from 101 to 200: 1382ms 13.0ms/request
Time from 201 to 300: 1504ms 15.0ms/request
Time from 301 to 400: 1330ms 13.0ms/request
Time from 401 to 500: 1308ms 13.0ms/request
Time from 501 to 600: 1405ms 14.0ms/request
Time from 601 to 700: 1196ms 11.0ms/request
Time from 701 to 800: 1335ms 13.0ms/request
Time from 801 to 900: 1370ms 13.0ms/request
Time from 901 to 1000: 1212ms 12.0ms/request
Time from 1001 to 1100: 1170ms 11.0ms/request
Time from 1101 to 1200: 1182ms 11.0ms/request
Time from 1201 to 1300: 1364ms 13.0ms/request
Time from 1301 to 1400: 1198ms 11.0ms/request
Time from 1401 to 1500: 1337ms 13.0ms/request
Time from 1501 to 1600: 1378ms 13.0ms/request
Time from 1601 to 1700: 1160ms 11.0ms/request
Time from 1701 to 1800: 1202ms 12.0ms/request
Time from 1801 to 1900: 1171ms 11.0ms/request
Time from 1901 to 2000: 1158ms 11.0ms/request
Time from 2001 to 2100: 1359ms 13.0ms/request
Time from 2101 to 2200: 1171ms 11.0ms/request
Time from 2201 to 2300: 1160ms 11.0ms/request
Time from 2301 to 2400: 1148ms 11.0ms/request
Time from 2401 to 2500: 1339ms 13.0ms/request
Time from 2501 to 2600: 1141ms 11.0ms/request
Time from 2601 to 2700: 1134ms 11.0ms/request
Time from 2701 to 2800: 1147ms 11.0ms/request
Time from 2801 to 2900: 1149ms 11.0ms/request
Time from 2901 to 3000: 1357ms 13.0ms/request
Time from 3001 to 3100: 1163ms 11.0ms/request
Time from 3101 to 3200: 1118ms 11.0ms/request
Time from 3201 to 3300: 1339ms 13.0ms/request
Time from 3301 to 3400: 1118ms 11.0ms/request
Time from 3401 to 3500: 1146ms 11.0ms/request
Time from 3501 to 3600: 1135ms 11.0ms/request
Time from 3601 to 3700: 1139ms 11.0ms/request

The time quickly settles around 11ms/req. With two clients, the avg. time becomes 13ms/req.

The first few requests:

Time from 1 to 1: 1255ms 1255.0ms/request
Time from 2 to 2: 35ms 35.0ms/request
Time from 3 to 3: 11ms 11.0ms/request
Time from 4 to 4: 12ms 12.0ms/request
Time from 5 to 5: 11ms 11.0ms/request
Time from 6 to 6: 17ms 17.0ms/request
Time from 7 to 7: 17ms 17.0ms/request
Time from 8 to 8: 18ms 18.0ms/request
Time from 9 to 9: 12ms 12.0ms/request
Time from 10 to 10: 21ms 21.0ms/request
Time from 11 to 11: 11ms 11.0ms/request
Time from 12 to 12: 12ms 12.0ms/request
Time from 13 to 13: 22ms 22.0ms/request
Time from 14 to 14: 17ms 17.0ms/request
Time from 15 to 15: 11ms 11.0ms/request
Time from 16 to 16: 11ms 11.0ms/request
Time from 17 to 17: 11ms 11.0ms/request

The key numbers are:

First request time1.3 sec
Avg. request time (after JIT)11 ms/req
Overall server frequency153 req/sec

Conclusion

The overhead of security in this case is less than in GT3. In GT3 the ratio between req/sec in the two cases is (174/10)=17.4, while with SSL we have (444/153)=2.9.