SkinBench


SkinBench is a benchmarking program that uses standard Windows API calls with regular windows and window controls (buttons, edit boxes, etc) to evaluate the performance of any skin in use (it will still work without any skin . . . probably much faster, but if you don't notice it, it hardly matters ;-). The idea is that, since it does things a user does (activating windows, moving them around . . .) it should reflect how fast a skin will appear to a user. Obviously, on their own, the numbers mean nothing, so you might want to see how some of the default skins score *on your computer* for comparison.

this program was created by Laurence Parry/GreenReaper Studios ( greenreaper@hotmail.com )






download SkinBench ver. .45 (78.6kb)





SkinBench ReadMe.txt

SkinBench is distributed under the GPL (GNU Public Licence - see copying.txt). As such, you are encouraged to distribute and modify it as you wish, as long as you abide by the terms of the GPL (for example, you *must* ensure that all code derived from SkinBench is freely available to all third parties in source form, and you must make this clear to your users). I've tried to ensure the code is reasonably commented - you should be able to figure it out ;-)

******New in 0.45

Various bugfixes (insane SkinMarks *should* now be a thing of the past - or the future ;-)
New tests, some of which might work . . . menu bar now present, but no tests specifically target menus - yet.

******New in 0.40:

Results are created in a separate directory, and in separate files for each skin (useful for keeping skin results to compare) - so if you benchmark "What is the Matrix?", the results will be in results/What is the Matrix .uis (Note to skin authors who made skins who's names are not valid filenames - I hate you all! :-)
Skin engines detected - WindowBlinds, Chroma, CustomEyes, Illumination, eFX. Not all are fully supported yet (some don't benchmark properly . . .)
Calibration - I'm not sure how well this will work, but click the "calibrate" button and try it out.
Radio and check buttons are now benchmarked.
Normal buttons are now *properly* benchmarked - <g>.
No more embarrassing spelling errors in the results files (I hope . . . I just misspelt "embarrassing", so I'm not that optimistic)
ResizeHoro and ResizeVert tests added to narrow down border performance problems.
Similarly EditHoro and EditVert for the scrollbars.
Vastly expanded settings dialog.
A few bug fixes - mostly to cope with new bugs - <g>

*****What the tests mean:

SkinMarks are my totally bogus method of marking skin speed. They are (theoretically) on a linear scale, and so a skin scoring 200 SkinMarks on a test should be about half as fast (on that *particular* test) as one scoring 400. Note the "about". They will report the number of cycles and the time the benchmarking module was allowed to run - a module may in fact take much longer to run in real time compared to the allocated time, as to ensure the benchmarks are accurate, only the drawing instructions are benchmarked - and if the drawing takes little time compared with the overhead of message handling . . . the net result is that the slowest skins will probably complete the benchmark in the fastest amount of real time. Ignore this. The SkinMarks are what matters, and more *should* mean faster. A SkinMark deviation of 1% or 2% per run is not significant, but 5% or more probably is - if only because you left Winamp on . . .

The scaling value for each test is a means of getting them all to add up correctly. They should be set so that vanilla windows scores 1000 SkinMarks on each test, so it's easier to see which part is letting the skin down. To do this, try clicking the calibration button on the startup dialog (having unloaded any skinning programs) - who knows, it might work!

The Resize and MoveAbout tests are the best indicators of overall skin speed. I find them to be uncannily accurate about how fast skins are, as the majority of the skinning WB and similar programs do is concerned with the title bar and borders. If this is crappy, don't expect your skin to be too popular (although it's very hard to be quantitative, 200 SkinMarks in these two tests is probably a good score for a fully-featured skin - vanilla Windows will probably score 800-1100 [depending on lots of things - of course, 1000 if properly calibrated], but your score may vary). Low marks in this test indicate too many buttons on a skin, or too much stretching or transparency in the borders, OR THE USE OF TEXTBACK IMAGES (officially evil).

The EditSize test measures (mostly) scrollbar speed. The text is just there for your amusement (and to get the scrollbars working). I know it's flickery . . . I *think* that's OK (I have tried to ensure that it is actually measuring how long it takes to repaint the edit box scrollbars when both skinned and unskinned, not just to send the resize message), but it may not be. Expect very high marks in this test if your skin doesn't skin scrollbars, and lower ones if it does. If they are very low, check that you are not stretching where you could be tiling, and try using a larger, repeated background or thumb image to improve performance.

The ButtonMove test measures how fast Windows is able to draw (skinned) buttons onscreen, pressed and unpressed. If this score is low, try tiling a bitmap for the centre of buttons and avoid using transparency. The RadioMove and CheckMove tests do the same for radio and check boxes, respectively.

The Activate test measures the repaint speed of active and inactive windows, and how fast a window that is brought to the front is repainted. This can be a very telling test - if a skin scores low in this test, it will be very obvious to the user, as it effectively measures window "response time", and so you will need to get it higher by applying some of the performance tips below.

The ResizeHoro and ResizeVert tests are provided to give a more specific view of border performance. If ResizeVert is slow, check your left and right borders. If ResizeHoro gives a low mark, you need to look at your top and bottom borders. Similarly, the EditHoro and EditVert tests narrow down differences between the vertical and horizontal scrollbars.

Any test with "Back" at the end has an extra window to cause more annoyance. If this is a *lot* lower than the corresponding test, you need to increase the skin's redraw speed (perhaps make borders smaller?)

The final rating is a simple addition of all the SkinMarks and SHOULD NOT BE QUOTED as a value for the skin, since it is NOT weighted (so a very, very good score in one test - such as EditSize for skins that don't skin the scrollbars - will unbalance all the rest). It should ONLY be used as a quick check to see if something has made a drastic change to the skin's performance.

*****Parameters (to be entered on the command line, or after the program name in the "Target" box for a shortcut - DEPRECIATED - use the settings dialog instead):

/a - Append results to the skin's result file (the normal behaviour is to replace the file). Useful for where you are trying to increase skin performance and want to see the effect of your changes over time.

/d - Enables debug mode. A file named debug.txt will be created in your SkinBench directory containing debug information, and will be displayed on exit. Useful in the event of a crash or bug.

/i - Ignore all non-fatal errors. Some errors cause SkinBench to fail, others do not. This option causes SkinBench to ignore the latter (although they will still be logged in debug.txt if the /d option is present).

/p - Don't attempt to increase SkinBench's priority. Use this if SkinBench is causing problems while it is running (such as suspending network connections). Be aware that this could make the benchmark less accurate (since it is more likely to be interrupted by background processes).

/q - Bypass initial settings dialog window. I've had trouble with always-on-top taskbars when using this option, so you might want to turn them off.

While running the tests, you may left-click to terminate SkinBench, or right-click to skip to the next test.

*****Other settings
Include system information - Gives stuff like windows build, screen size and colour depth, etc.
Pause between each tick - Now, SkinBench already lets other apps have a go with the CPU each time it goes round its loop, but it may not be enough. Use this to make sure it pauses for a bit longer. Things may look smoother with this, but I'm not sure if they are necessarily better . . . it's up to you.
Maximum module run-time - How long a module can take, in real terms. Just in case something goes wrong with the timing, and it thinks you have 5 sec to go . . . forever ;-)

*****More about SkinBench:
SkinBench is not perfect (actually, it's nowhere near perfect :-). Like any benchmark, it is affected by the programs which are running at the time. This means you should *PAUSE* Winamp (or any other CPU-intensive apps that don't run on idle loops) before benchmarking. In theory, if you always ran Winamp, it wouldn't matter (since it would be a constant load), but in practice it does (because it isn't ;-). It is also dependent on screen resolution, colour depth . . . lots of things. Plus, this is my first benchmark, so I don't know how well I've made it reflect the actual speed of a skin in general use. It works well for me, but maybe not for you. Do remember that.

Use of this program for public advertising purposes, either by skinners or by skinning programmers is discouraged, for the simple reason that what may be true for one computer/OS/skinning program may not be true for another, and that these benchmarks measure only certain facets of performance - the whole can be appreciated by the user alone. Its intended use is as a tool for the use of skinners while they are designing skins, in order to increase performance. I can't *stop* you using this program in other ways, but expect to get flamed by me (and by others) if you do . . .

Lastly, remember that performance, although important, is *not* everything. If it was, we'd have no use for skins, since it *is* true that they slow drawing down (by a factor of 10 or more, sometimes). Mercury's skin, "What is the Matrix?" (Mercury: sorry for using it all the time as an example ;-) is one of the slowest skins I have tested. Yet it is hugely popular. The reason for this is simple: it looks stunning, it has lots of value-added features and it is, overall, a great skin. When it comes down to it, *that* is what matters. If you ever feel that your artistic vision is being compromised by performance issues, remember that speed is probably not an overriding concern.

In programming, a good maxim is to make it work first, before you make it work fast. Make your skins look good first, then use SkinBench to help you make it perform better. Of course, checking each time you add a new feature to see if it slows your skin down is prudent, too . . .

SkinBench was programmed to the music of REM, The Beatles, They Might Be Giants, Radiohead and Wierd Al Yankovic. It is dedicated to *you* (the skinning community), the Open Source movement and to those skilled programmers that make skinning possible.

******Performance Tips:
SkinBench is merely one of the tools you can use to ensure your skin reaches its optimum performance. Perhaps the most important one for you to use is common sense, sprinkled with a bit of basic computing knowledge. There are several things that slow all skins down - using transparency, using stretching rather than tiling, stretching *or* tiling really small bitmaps in the borders or backgrounds. Also, the smaller the title bar and borders, the faster the skin will go - this can make a big difference. You might even want to make two versions - one with a big titlebar for higher-resolution screens, one for better performance on lower-resolution screens. Oh yeah, and *don't* use textback images (WindowBlinds). They stretch and slow the skin down a horrendous amount.

Most of tips are fairly obvious, and most can be implemented with a few minor changes to existing skins, but many skinners don't take the time to bother with it. If you don't make that mistake, your users will appreciate it - <g>

******Known issues:
Always-on-top taskbars may show up over SkinBench main window. This could affect speed.
WindowFX "Fade in windows on opening (2k)" causes non-visibility of windows (may only be fixed by the charitable donation of a version of Visual Studio that knows about AnimateWindow ;-)
Some skinning programs don't benchmark properly, either because of the way they do their thing, or something else I haven't found out about yet. I'm working on it.
This readme is too long. Oh well.

******Todo:
CreateWindow test . . . maybe . . .
More and better tests.

*****Contact and Bug reporting:
If you have *any* problems, please email me at greenreaper@hotmail.com, saying when the problem occurred, what skin and skinning program you were using and any other information you feel is relevant (credit card numbers, mother's maiden name, that sort of thing . . . ;-). If they are available, please attach copies of debug.txt and results.txt from the SkinBench directory (to get debug.txt you may need to re-run SkinBench with the /d parameter . . . and duplicate the error, of course - ). If you don't have any problems, feel free to email me anyway - I'd like to hear from you!

Have a lot of fun . . .

TheGreenReaper (Laurence Parry)

******Thanks for input:
Brad Wardell (SD), Neil Barnfield (SD), Adam Najmanowicz, Ian Hanschen (SD), Matthijs Vermeulen, Jason Haag, Bryant Lykes, Alobar Greywalker, Guy Hershberger, Sebastien Caisse, Derek Charboneau.

******Version history:
0.01 - The beginning
0.10 - Modular system completed (internal)
0.20 - Settings system finalized (internal)
0.30 - Initial public alpha release (8 modules complete) - you have been warned . . .
0.40 - Second public alpha release (12 modules complete, auto-calibration added)
0.45 - Beta release (15 modules complete, calibration and timing altered to work ;-)

"SkinBench", "SkinMarks" and "SkinBench - The Name Says It All!" are totally unregistered trademarks of Laurence Parry/GreenReaper Studios ('Delivering bytes at the speed of thought!'). Feel free to use them wherever you deem appropriate, or where they'll get a laugh.

*WARNING* - Benchmarking stresses components, and in the course of testing your skin, SkinBench may bend, fold or mutilate windows, for which GreenReaper Studios cannot be held accountable. Please avoid the use of naked flames in conjunction with this product. No small furry anime characters were tortured in the making of SkinBench - they're being saved up for later . . .