<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<font face="Tahoma">Hi<br>
<br>
Ah, I see, I just wasn't following the code structure very well.
It's tricky to read as a diff :-)<br>
<br>
I have no problem with your code, it looks well written. <br>
<br>
We use apache stuff extensively here at work because the license
is business friendly.<br>
<br>
Regards, Noel Grandin<br>
</font><br>
Rolf Lear wrote:
<blockquote cite="mid:4E9806DA.5020605@tuis.net" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Hi Noel<br>
<br>
Thanks for that.<br>
<br>
It comes out in the numbers, but, for the record, I am doing
something very similar to that.... only the structures are
slightly (very) different.<br>
<br>
I do a bunch of GC's, and I do one in a different thread with the
current thread sleeping, then I repeat the GC's until the size
becomes 'stable' at a change of less than 128 bytes.<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb#L1R33">https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb#L1R33</a><br>
<br>
I do a complete once-through of the test suite to warm things up.<br>
Each once-through runs the code through 6 times (hmmm... I thought
it was 12, but that was something else I did yesterday). Each of
the actual tests 'exercises' the code repeatedly because it's all
sort of loop-based code (parsing, scanning, etc.).<br>
<br>
Anyway, the output of the 'warmup' run is always much slower than
the remaining 5 'real' runs, and I do the 'real' runs multiple
times to ensure there is some stability.<br>
<br>
What you see in the web-page is the result of what I believe to be
fully JIT-compiled and 'clean' and 'reliable enough' for the
purposes I want.<br>
<br>
I know that the Java VM testing is 'tricky' when it comes to
performance, and as such I understand that it's easy to get things
wrong, and I'll spend more time looking at it to ensure I'm doing
the reasonable thing, but, are you suggesting that the code I am
running is not actually getting reliable results?<br>
<br>
The code is structured differently to what you have suggested
below, but, the entire 'main' loop is warmed up:
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb#L1R124">https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb#L1R124</a><br>
<br>
Then, the main loop is run 5 times, and I visually inspect the
numbers to ensure that they are consistent:
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb#L1R135">https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb#L1R135</a><br>
<br>
Between each 'test' I do a full GC with 'bells and whistles'<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb#L1R160">https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb#L1R160</a><br>
<br>
It is quite obvious that the runs that come out of the 'real'
loops are optimized, cached, etc.<br>
<br>
What is not clear is whether the optimizer has completely compiled
out some of the code. I have tried to ensure that it does not by
doing some sort of test on each element so that it is not
completely ignored.<br>
now that I think about it though, maybe the 'devnull' Writer is
too 'light' and the optimizer may have completely skipped it
entirely..... and the whole XMLOUtputter code with it.... I will
check.<br>
<br>
So, I appreciate the insight, and I will play around with things
to see if increasing the number of warmup and actual 'real' runs
changes the numbers.<br>
<br>
I'll look in to making sure that some of the code is not being
optimized out completely.<br>
<br>
But, my code already is doing pretty much exactly what you are
suggesting... (it does not calculate the deviation, but it does
ignore the fastest and slowest run.....).<br>
<br>
In fact, it does more because it then repeats the exact same loops
multiple times to ensure the averages remain consistent over runs
(as it happens, it essentially does 20 'runs' of the code to get
the results - 5 loops of 6 runs but the 6 only counts as 4 because
the best and worst are eliminated).<br>
<br>
Have you got specific concerns about the code? Did you run it? Do
you think the results are 'wrong'?<br>
<br>
Thanks for the insight in to the commons-math code. I'm always
'discovering' more and more 'stuff' in commons code. I have some
'stuff' I've done at work I am trying to convince my boss
(actually legal&compliance) to let me use in JDOM, but it's
the sort of thing that belongs in a 'commons' type location, not
JDOM....<br>
<br>
Rolf<br>
<br>
On 14/10/2011 4:08 AM, Noel Grandin wrote:
<blockquote cite="mid:4E97EE04.3070307@peralex.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<font face="Tahoma">Hi<br>
<br>
Performance testing on the Java VM is tricky. <br>
To avoid getting caught out by cache-hot/cache-cold and JIT
vs. not-JIT things, it's preferrable to do something like this
in PerfTest#timeRun(Runnnable)<br>
<br>
// warm up the caches and get the JIT going<br>
for (int i=0; i<10; i++) {<br>
runnable.run();<br>
}<br>
<br>
// give the JIT time to run, and get GC to run - GC can be
stubborn sometimes<br>
</font><font face="Tahoma">for (int i=0; i<3; i++) {<br>
</font><font face="Tahoma">Thread.sleep(100);<br>
</font><font face="Tahoma">System.gc();<br>
}<br>
<br>
// need 20 runs to get a decent average and standard deviation<br>
ArithmeticMean mean = new ArithmeticMean(); // these two
classes are in jakarata-commons-math<br>
</font><font face="Tahoma">Variance deviation = new Variance();<br>
</font><font face="Tahoma">for (int i=0; i<20; i++) {<br>
long time1 = System.currentTimeNanos();<br>
runnable.run();<br>
</font><font face="Tahoma"> long time2 =
System.currentTimeNanos();<br>
mean.increment(time2 - time1);<br>
deviation</font><font face="Tahoma">.increment(time2 -
time1);</font><br>
<font face="Tahoma">}<br>
<br>
System.out.println("result = " + mean.getMean() + " +- " +
deviation.getVariance());<br>
</font><br>
Regards, Noel Grandin<br>
<br>
Rolf wrote:
<blockquote cite="mid:4E9781E5.1080609@tuis.net" type="cite">Hi
all. <br>
<br>
I have put together a 'simple' system for measuring the
relative performance of JDOM2. The idea is that I need to know
whether I am improving or breaking JDOM performance as the
code evolves. <br>
<br>
Currently the metric code is only useful of you compare apples
to apples, and, in this case, it means processing a single
(medium size) XML document on my laptop, yada-yada-yada. But,
it should be useful as a tool to get a feel for what a
code-change does. <br>
<br>
Already I can see that I probably have an issue in the
SAXHandler (possibly an issue in JDOM-1.1.2 actually) because
1.1.2 is 5-times faster in that area than JDOM2. <br>
<br>
I have put together a results page here: <br>
<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://hunterhacker.github.com/jdom/jdom2/performance.html">http://hunterhacker.github.com/jdom/jdom2/performance.html</a>
<br>
<br>
It also describes what each test does. If you are interested
in seeing the code and what it does have a look here (it is
not well documented and it is still perhaps evolving): <br>
<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb">https://github.com/hunterhacker/jdom/commit/8b719c86913398ace8e197b6de145b33d9d300bb</a>
<br>
<br>
<br>
Rolf <br>
_______________________________________________ <br>
To control your jdom-interest membership: <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://www.jdom.org/mailman/options/jdom-interest/youraddr@yourhost.com">http://www.jdom.org/mailman/options/jdom-interest/youraddr@yourhost.com</a>
<br>
<br>
</blockquote>
<br>
<br>
<br>
<hr><font color="808080" size="-2">Disclaimer: <a
moz-do-not-send="true"
href="http://www.peralex.com/disclaimer.html">http://www.peralex.com/disclaimer.html</a><br>
<br>
</font></blockquote>
<br>
</blockquote>
<br><br><br><hr><font size="-2" color=808080>Disclaimer: <a href="http://www.peralex.com/disclaimer.html">http://www.peralex.com/disclaimer.html</a><br><br>
</body>
</html>