There was a thread the other day on the Railo Google Group regarding the "Save Class Files" setting on Railo. I piped up with my usual spiel that I don't think it's a worthwhile setting to use on ColdFusion:
Yeah, I can't vouch for Railo, but on ColdFusion saving class files rapidly becomes a performance hit on the system (Windows) if your app is such that you generate more than a thousand or so classes. The reason being - it seems - that Windows really struggles to file-scan a directory to find the pre-compiled class if it's needed... say the one in memory has been garbage collected... once there are more than a few hundred files in that directory. On big apps it's more expedient to let the class recompile from source code than rely on Windows finding and loading a saved one.Mark Drew subsequently suggested it might be a topic for a blog article, so here one is.
I doubt this is a problem on *nix-based file systems.
The issue with CF is that all the classes are stored in one flat directory. If they were stored hierarchically (which shouldn't be too much of an issue?), then there'd be no problem. I did mention this to the Adobe bods, but they looked at me like I was speaking [some language they didn't understand... CF perhaps], and that was that.
Anyway... I dunno if the same consideration exists on Railo, but this would be one reason why one might not want those class files saving. Something to test, perhaps.
Firstly, what does this setting do? Well: before that... what am I on about? In CFAdmin, in the Server Settings / Caching options, there's a setting "Save class files", which is summarised thus:
When you select this option, the class files generated by ColdFusion are saved to disk for reuse after the server restarts. Adobe recommends this for production systems. During development, Adobe recommends that you do not select this option.When one executes a CFML file (CFM or CFC) then it's first compiled to Java, then executed. The CFML itself is never run: it just describes how to write the Java for the page. Having done the compilation, CF is left with a compiled Java class (in memory). The translation from CFML to Java class is quite a weighty one, and needs to be redone if that compiled class is cycled out of memory. This could happen if the JVM runs out of space, and not-oft-used classes are cleared out. So the next time one calls the CFML that would generate a previously "tidied up" class, the CFML needs to be recompiled again. So again with the performance hit of doing this. To mitigate this, ColdFusion can also be set to save the compiled classes to the file system, so if they are ever cycled out of memory but then needed again, the class can simply be reloaded from disk, rather than needing to go through the close CFML-to-Java-class compilation process again. That "Save class files" setting dictates whether to write the classes to disk or not.
As per above, I have always advocated not bothering with this setting - ie: don't ever save the class files to disk - because on all but the smallest systems on Windows it's actually slower to find the file than it is to recompile the CFML. I reached this conclusion at some stage in the past having noticed a tangible performance gain when not saving the files. Equally, anyone who has ever worked on Windows in a directory which has more than a few hundred files in it will know Windows struggles a bit. I have never experienced this on *nix, but then again I don't spend much time using *nix so that's not a good basis for making any sort of sensible observation.
Still: Windows is clearly slow fetching files from directories with heaps of files in them, and ColdFusion generates really a lot of class files in the cfclasses dir ([instance]/WEB-INF/cfclasses), and there seems to be a performance hit when CF is using those files... this seems like a reasonable case of causality.
But just because I've experienced something in the past doesn't mean it's still the case now; or there might have been some other contributing factor when I experienced this before and it's not usually an issue, or any number of other things might invalidate this unmeasured experience I have had. I don't like giving advice (although I still will ;-) on topics I have not investigated, so I figured I should retroactively find out whether or not I'm talking shit.
When ColdFusion compiles CFML files, it will create one class file for every CFM file, one class file for every CFC file, and one class file for every UDF within a CFC (and I think within a CFM either, but am not sure). It saves all of these in a single, unorganised morass in the cfclasses directory, and then expects the OS to be able to fetch them again when requested.
I set up some code which generated 1000 unique CFCs, each with between 5-15 methods in them. The code is unexciting and not really relevant, so I'll not include it inline here, but here's a gist with the code I used.
A test - run.cfm - created an instance of each of the CFCs, and output how long it took to do all 1000. This is a very contrived test, I know, but it's not testing real-world performance, it is testing comparative performance, so is reasonable enough. The test was run as follows:
- Restart the ColdFusion instance.
- Allow the CPU to settle down to an "idle" state.
- Run some code that "warmed" the CF instance. This was just a file that dumped the server scope. I ran this file five times.
- Again allowed the CPU to settle down to an idle state.
- Ran run.cfm, noting how long it took to run.
- Repeat 1-5 six times.
- Class files being saved
- Class files not being saved
- Restarting the PC only between each set of six iterations (still restarting ColdFusion between each iteration, but only restarting the PC before each set of six);
- Restarting the PC between every single execution of run.cfm.
I did the two variations of machine restart because I saw an anomaly in the test results when setting up the experiment. This also explains why I did six iterations and not the more decimal-friendly five. I was going to do it five times, but the first round for a given "life" of the box I was running this on was always noticeably (about 10% more than the next highest round) slower than the other five, so I treated the initial test as a warming exercise too. My suspicion is that once the OS's file system has located a file once, the HDD controller remembers where it is, so the seek time is reduced a lot thereafter. This "indexing" is not persisted between reboots of the computer. So in the end I did six iterations of the test, but only included the latter five in my averages.
The results on ColdFusion were as follows:
| ColdFusion 10 | |||
|---|---|---|---|
| No Restart | Restart | ||
| No Save | Save | No Save | Save |
| 79287 | 65075 | 76695 | 107972 |
| 68639 | 42110 | 77411 | 99051 |
| 65812 | 40439 | 76839 | 96516 |
| 67467 | 42403 | 78412 | 101025 |
| 69575 | 40692 | 76875 | 103695 |
| 69909 | 40515 | 76801 | 95936 |
| 68280.4 | 41231.8 | 77267.6 | 99244.6 |
Time is in milliseconds.
So what we see here is that saving the class files is actually the best performer provided the computer is not restarted, but it is the worst performer a server restart. So I guess my experiences of saving class files being a performance problem stem from experiences of when the whole box was rebooted, not simply when ColdFusion gets restarted. And once Windows actually knows where to find the files (ie: after a first load), it fetches them OK. So I need to go back on my advice about saving class files, I think. Oops.
Railo stores its saved class files in a fair-more-sensibly organised hierarchical structure (matching the file system hierarchy of the original CFML files, which makes sense), so I wanted to see whether there was any performance differentiation when doing it this way, so I performed all the same tests with Railo too (same JVM config as the ColdFusion tests, so as like-for-like as possible).
| Railo Tomcat | |||
|---|---|---|---|
| No Restart | Restart | ||
| No Save | Save | No Save | Save |
| 26415 | 14465 | 36684 | 15052 |
| 24675 | 11824 | 36410 | 14858 |
| 25507 | 11757 | 35846 | 15486 |
| 23007 | 11697 | 39168 | 16740 |
| 24973 | 11722 | 38563 | 14685 |
| 24406 | 11612 | 38460 | 15455 |
| 24513.6 | 11722.4 | 37689.4 | 15444.8 |
Blimey. Sod the performance difference between whether we're reusing saved class files or restarting or whatever, Railo just blows ColdFusion out of the water on all metrics. The slowest performance on Railo is significantly faster than the fastest on ColdFusion. And the fastest on Railo is around four times faster than that! It still seems there's a significant amount of time saved with the file system look-ups with the hierarchical approach to saving the class files, because the time difference between when the file system needs to find the files each test (when there's a reboot between each test), and when it doesn't (no reboot) is nowhere near as pronounced as with ColdFusion.
So I think ColdFusion could get some gains by being less flippant with how it stores its saved class files, but it really oughta also look at why it's so slow compared to Railo.
And I should definitely check my facts before I give out advice!
--
Adam
