Kassen wrote:

Some notes on your method; you are using a file (and hence a shred) per Ugen. I believe there to be some small overhead for the usage of a shred so I believe you may be ending up with slightly high numbers.

I'll check the difference. But it seems to me that for this purpose ("which of A and B is more expensive") it doesn't matter since we're interested in the relative speed of code.

As for further tests that might be useful/ interesting; you could compare the Blit osc's to the regular ones (as well as try to deterimine what -if any- difference the amount of harmonics makes). Another interesting thing to compare might be LiSa with multiple voices v.s. as many copies of SndBuf.

Agreed, and I have lot's of other ideas + plan on testing things as they occur in my work.

If at all possible it would also be interesting to fully automate a series of tests like this so we could compare versions of ChucK later.

You're absolutely right. I actually started something automatic, but got stuck, and thought a "by-hand" start was better than nothing. I'll for sure see if I can make a 100% automatic test. Thanks for your feedback! -- Atte http://atte.dk http://modlys.dk

Atte; I'll check the difference. But it seems to me that for this purpose ("which

of A and B is more expensive") it doesn't matter since we're interested in the relative speed of code.

I think getting as close as possible to the actual cost of a single UGen would be useful. We may have several ways of constructing a certain sound out of a few UGens, in that case the relative cost of single UGens would be quite relevant. I recognise that we may not be able to find the exact cost in all cases as it may depend on the context, especially if/when we would start optimising ChucK but it could still be interesting to get as close as possible. Even the question of whether or not such a difference really exists in a appreciable form would be interesting; such info would help us make more informed choices about the the structure of our programs. Agreed, and I have lot's of other ideas + plan on testing things as they

occur in my work.

Great! I'm looking forward to reading about this. You're absolutely right. I actually started something automatic, but got

stuck, and thought a "by-hand" start was better than nothing. I'll for sure see if I can make a 100% automatic test.

Oh, yes, clearly, this is already quite interesting. Eric had a interesting note about different kernels; I wonder whether that would make a difference beyond the relative cost of UGens compared to eachother (as percentages) and the overhead cost any given OS has for running a program at all. I could imagine different CPU's making a difference, provided we have a good compiler and compile each on it's own system. We could create a standard for testing this; for example saying the cost of a single SinOsc is "1", always using the same buffer size and so on. Yours, Kas.

Atte André Jensen wrote:

Automation is on it's way, need some polish, but it basically works now!

Done. Slightly different output... -- Atte http://atte.dk http://modlys.dk

Kassen wrote:

I meant this as a reason to be curious about the exact (as opposed to relative) cost of UGens, not as a speciffic scenario to test.

I hope that clarifies.

Kind of. I still don't get it. You're saying that it's a problem that I run several shreds per test, right? Obviously I'm doing that to get numbers (cpu usage) in a range where they make sense, admittedly entirely based on my gut feeling. Comparing cpu loads of 2.1 and 2.2 is not as good as 84.0 and 88.0 + I'd expect small numbers to be relatively more polluted with "stuff from the system", including the vm. Also close to 100% things are useless, the system would start to blow up, stutter etc. But supposed we compare these lines (a result of the current version of the test): file x10 x50 x100 -------------------------------------------------------- 01_PulseOsc.ck 2.7 8.5 18.5 01_SawOsc.ck 3.0 10.2 22.5 01_SinOsc.ck 5.0 28.0 44.5 01_SqrOsc.ck 2.7 8.7 19.2 01_TriOsc.ck 3.0 10.2 22.5 Another run: file x10 x50 x100 -------------------------------------------------------- 01_PulseOsc.ck 3.0 9.5 19.5 01_SawOsc.ck 3.0 11.0 22.0 01_SinOsc.ck 5.5 21.5 45.0 01_SqrOsc.ck 3.0 9.5 19.5 01_TriOsc.ck 3.5 10.5 23.5 Wouldn't you say that it's safe to say that SinOsc is *about* twice as expensive as the others? I mean the numbers in the same column should be directly comparable, or? Naturally that is provided that the measurements are sane. Thinking about the statics mentioned by Tom (http://www.zedshaw.com/essays/programmer_stats.html) this is actually a real challenge. A simple way would be to have chuck run for a number of seconds, and take measurements of the cpu load at certain intervals. Then (with out knowing much about statistics) something like throwing away measurements that are way of and averaging between the remaining could make sense. Or one might be interested in the maximum load generated by the code, although many things outside of chuck (the system) could account for the "jitter". For instance the x50 of SinOsc are 28.0 compared to 21.5 in the two different runs. I have no idea where this jitter comes from (the specific UGen or my systems or something else), but clearly that's something that should be improved upon. -- Atte http://atte.dk http://modlys.dk

Kassen wrote:

Let's say a bag of candy costs 2$, a piza is 4$ and driving to the store costs me 1$. This means buying a bag of candy costs 3$ (in practice), a piza would cost me 5$ but driving o the store to buy both would only by 2+4+1=7$ as I'd only have to drive once. Here driving equates to "having a shred".

But if the prize of driving were magnitudes lower (say $.01) than candy and pizza and you could only pickup one thing at a time, you could compare the prize of candy and pizza by getting 100 of each: pizza: 100 * (4 + 0.01) = 401 candy: 100 * (2 + 0.01) = 201 Still the picture is a little screwed by the cost of driving, but if that's low (and my tests suggest the shreds are) you're pretty close. Still I'm gonna work with your idea! Stupid me doesn't know of a way to pass arguments to a chuck file from the commandline. I'd rather not have the 100 as in "create 100 SinOsc's" hardcoded in the files. -- Atte http://atte.dk http://modlys.dk

Kassen wrote:

Well, even if the difference is slight I do think my idea has some merit.

Agreed.

Comand line argument syntax is documented in VERSIONS.txt in your ChucK dir as well as in a example or two in the examples dir. Regardless of the exact numerical outcome using those should simplify the testing process which has advantages in itself. For one thing the outcome of a more simple test should be more simple to analyse.

Never used that, but now I have :-) If you could The test now uses Your Way (TM), differences goes like this: Old style file x10 x50 x100 -------------------------------------------------------- 00_wait.ck 1.5 1.7 1.7 01_PulseOsc.ck 2.7 8.5 18.5 01_SawOsc.ck 3.0 10.2 22.5 01_SinOsc.ck 5.0 28.0 44.5 01_SqrOsc.ck 2.7 8.7 19.2 01_TriOsc.ck 3.0 10.2 22.5 02_SndBuf.ck 2.5 7.7 15.5 02_SndBuf_loaded_not_playing.ck 1.5 2.0 3.0 02_SndBuf_loop.ck 5.2 24.2 51.0 03_Gain.ck 2.5 7.5 14.7 03_Pan2.ck 5.0 24.0 49.2 05_Delay.ck 3.2 13.5 30.7 05_Echo.ck 3.5 14.7 39.7 06_BPF.ck 4.0 10.2 20.5 06_BRF.ck 3.0 10.2 20.5 06_HPF.ck 3.0 10.0 21.5 06_LPF.ck 2.7 10.0 20.5 07_JCRev.ck 12.2 75.2 97.2 07_NRev.ck 18.7 97.0 97.2 07_PRCRev.ck 7.2 45.5 93.2 08_Chorus.ck 7.2 37.7 79.5 90_SinOscLPF.ck 6.5 32.5 67.5 New style: file x10 x50 x100 -------------------------------------------------------- 00_wait.ck 1.5 1.5 1.5 01_PulseOsc.ck 2.8 8.3 18.2 01_SawOsc.ck 2.5 10.2 22.0 01_SinOsc.ck 4.1 21.5 43.2 01_SqrOsc.ck 2.6 8.5 24.3 01_TriOsc.ck 3.0 10.2 22.0 02_SndBuf.ck 2.5 7.3 14.7 02_SndBuf_loaded_not_playing.ck 1.5 2.0 2.5 02_SndBuf_loop.ck 5.4 24.7 52.5 03_Gain.ck 2.5 7.2 14.0 03_Pan2.ck 5.0 24.5 49.2 05_Delay.ck 3.2 13.2 28.7 05_Echo.ck 3.6 12.8 40.5 06_BPF.ck 2.8 9.7 20.4 06_BRF.ck 3.0 10.0 20.4 06_HPF.ck 2.8 9.7 20.5 06_LPF.ck 2.4 10.0 20.2 07_JCRev.ck 12.7 75.7 97.2 07_NRev.ck 18.2 97.5 97.7 07_PRCRev.ck 6.3 45.2 92.0 08_Chorus.ck 7.2 33.3 81.7 90_SinOscLPF.ck 6.7 32.5 67.0 One thing that clutters up the picture is that I also changed the script to take three measurements and throw away the largest and smallest value, which should (and seems to have) get rid of the worst jitter. Besides that they don't look that different, or... Anyways, this is the content of for instance 01_SinOsc.ck, do you think it looks sane? 0 => int instances; if(me.args()) me.arg(0) => Std.atoi => instances; repeat(instances){ SinOsc s => dac; 0 => s.gain; } 1::week => now; -- Atte http://atte.dk http://modlys.dk

Kassen wrote:

Anyways, this is the content of for instance 01_SinOsc.ck, do you think it looks sane?

Yeah, that looks "by the book" to me :¬)

Ok, thanks.

One thing that has me wondering now is the cost of a non-playing SndBuf; that's very low indeed, lower than a gain. Some optimisation must be going on there; nice.

OR: it wasn't connected to dac :-) I'm not sure it was on purpose, but agree it's confusing at least by looking at the filename (I suspect you didn't look at the chuck code). With it connected it looks different: file x10 x50 x100 x1000 -------------------------------------------------------- 02_SndBuf.ck 2.6 7.5 15.2 96.7 02_SndBuf_loaded_not_playing.ck 4.6 24.2 53.0 97.2 02_SndBuf_loop.ck 5.3 24.7 53.2 97.0 03_Gain.ck 2.6 7.3 14.4 96.2 A different story: Do you have any ideas about how to test stuff like multiplication vs division? These results lead me to believe that the usual rave about division being expensive it totally irrelevant as soon as you connect even a single UGen in your chuck code :-) file x10 x50 x100 x1000 -------------------------------------------------------- 11_cast_float_to_int.ck 2.2 2.2 2.2 2.2 11_divide_integers.ck 2.5 2.3 2.5 2.4 11_modulo_integer.ck 2.2 2.2 2.2 2.4 11_multiply_integers.ck 2.4 2.2 2.2 2.2 atte@vestbjerg:~/music/chuck/performance_tests$ cat tests/11_divide_integers.ck 0 => int instances; if(me.args()) me.arg(0) => Std.atoi => instances; int i; repeat(instances){ while(true){ 1::samp => now; 10 / 2 => i; } } 1::week => now;

PS; As a side note; GMail users can enable a google labs extension that will allow them to view selected emails in fixed-width font. This makes looking at tables like this a lot more convenient.

I'm using thunderbird... -- Atte http://atte.dk http://modlys.dk

Atte; OR: it wasn't connected to dac :-)

Right, yes, that would explain it :¬)

I'm not sure it was on purpose, but agree it's confusing at least by looking at the filename (I suspect you didn't look at the chuck code). With it connected it looks different:

Ah, yes. (pasting your notes from your other mail here)

This means that the non-playing trick "s.samples() => s.pos" won't do anything for the cpu (which I thought). It must be disconnected from dac to be easy on the cpu...

That's right; that trick saves the ears, not the cpu. It's still a good trick as it saves a lot of noise in setups where we use a lot of samples. A different story: Do you have any ideas about how to test stuff like

multiplication vs division?

What I came up seems very close to your solution These results lead me to believe that the usual rave about division being

expensive it totally irrelevant as soon as you connect even a single UGen in your chuck code :-)

The difference seems quite small indeed; intersting. I wonder how conditions compare against those; I always thought those were relatively expensive. PS; As a side note; GMail users can enable a google labs extension that will

...

allow them to view selected emails in fixed-width font. This makes looking at tables like this a lot more convenient.

I'm using thunderbird...

My lifestyle forces me to use a web-based solution. I'd like to say that's because of traveling so much but in reality it has to do with tinkering with computers and OS's and occasionally having lost a install and all the mail archives that were in it :¬) Kas.

Kassen wrote:

As for further tests that might be useful/ interesting; you could compare the Blit osc's to the regular ones (as well as try to deterimine what -if any- difference the amount of harmonics makes).

They are indeed more expensive than their counterparts. Number of harmonics doesn't seem to matter, though (see link below).

Another interesting thing to compare might be LiSa with multiple voices v.s. as many copies of SndBuf.

Hmm. Could you supply a test file for this?

The list goes on; we might want to know the cost of member functions like the .freq() parameter of filters, we may even want to know the cost of certain operations.

But this (and other operations needs to be done at a rate. Would every samp make sense? Or should we pump more operations per samp, for instance with a inline repeat? Another thing: Some UGens could prove more expensive in use (Envelope and ADSR seems to fall into this category), which ones should I test, and how? For Envelope/ADSR I simply connected them to a SinOsc, and found that the cost of ADSR => SinOsc => dac is more than ADSR => dac + SinOsc => dac. I've wrapped it up a bit, with direct links to the test files for easier browsing: http://atte.dk/chuck/ -- Atte http://atte.dk http://modlys.dk

On Mar 12, 2009, at 3:26 AM, Atte André Jensen wrote:

Another thing: Some UGens could prove more expensive in use (Envelope and ADSR seems to fall into this category), which ones should I test, and how? For Envelope/ADSR I simply connected them to a SinOsc, and found that the cost of ADSR => SinOsc => dac is more than ADSR => dac + SinOsc => dac.

There is an additional cost to chucking anything into SinOsc -- that might be what you are seeing here. SinOsc (and some other Osc's) map its input to frequency (depending on the .sync parameter); this entire code path is skipped if the tick function detects that there are no input ugens. Perhaps a more fair comparison would be to replace SinOsc with a ugen that does not make this optimization, e.g. gain, or to do something like: ADSR env => SinOsc s => dac; Gain g => s; vs. Gain g => SinOsc s => dac; ADSR env => dac; spencer

Tom; Also, recommended reading:

http://www.zedshaw.com/essays/programmer_stats.html If you skip the first half, there's actual information down below.

Yes. The first half implies being tall should lead to less problems with women. At 2 meters I must count as "tall" and I didn't find this at all. It does get interesting and relevant right after that though (and as opinionated as it is; it's quite pleasant to read). Fortunately we are quite safe from one of the major complications; optimisation. If ChucK would have a optimising compiler, for example, we'd be in serious trouble. :¬) As for Gain & Pan2; you are quite right. Here we could say that Gain is more or less like the "base cost" of all UGens. Still; a modest little Gain is capable of adding, multiplying and so on. Clearly Pan2 needs it's own code but it's not clear to me yet why that code has to be so much more expensive than just two Gains. One explanation would be that the base cost of a Gain doesn't account for everything; a Gain set to multiply might be more expensive than a plain one or maybe stereo adds cost. I suspect there are hidden issues that we haven't evaluated yet. At any rate this shows that the base-costs of adding a UGen at all are there and are quite signifficant compared to the cost a full UGen. IMHO this pleads for the "blob" project. For example a crossfader is a device that we could quite easily make ourselves out of other UGens, so is a clipper and a state-variable filter (the list goes on) but this table already shows us that desipite these being possible to create already there would be signifficant benefits to increasing our range of available low-level building blocks. This should in turn cut down on the costs of implementing more advanced/ interesting synthesis techniques in ChucK. Oh, and SndBuf is very cheap indeed, far cheaper than I thought it was, that's nice to know. Cheers, Kas.

Tom;

I'm pretty sure there being no Gain-specific functionality is all there is to it. Can you find a UGen that doesn't support "3 => u.op"? I think "Gain" is just the most generic UGen possible.

I agree.

(I should really go verify it in code, but I'm a little busy tonight.) (Same reason I haven't tried my hand at the ChucK benchmarks.) (I'll probably forget about both of these by the time I'm free, though. ;p)

Well, there are UGens with no input (SndBuf, the STKInstruments...) I'm not sure what good .op() is for them but I do believe the docs claim they have it. That's not actually useful, but then again; this compiles as well; 2 => blackhole.op; //it has a .gain() as well, there 's something Zen about that. While we're at it; dac has a .op as well, I found that earlier. This can indeed be used to create terrible noises after sporking a few shreds as the dac is the same across all shreds, in case you were wondering. In my experience it's best left for those situation where you were planning to end your set anyway :¬).

...

At any rate this shows that the base-costs of adding a UGen at all are there and are quite signifficant compared to the cost a full UGen. IMHO this pleads for the "blob" project.

Or an optimizing ChucK compiler! (heh, heh, any takers?)

That would mean re-compiling the whole UGen graph at any additions as we can hot-swap running UGens, I posted about that when we were talking about more advanced uses of casting. This would be a spectacularly bad idea unless we could also somehow preserve state of individual UGens to prevent glitches. We may be able to have a optimising compiler but we can't (preserving current functionality) condence the UGen graph like some digital modulars do (simplifying groups of UGens internally). Well, maybe we could but even thinking about it makes my head hurt. IMHO we need more facilities for hot-swapping UGens, not less. Cheers, Kas.

Steve;

We may be able to have a optimising compiler but we can't (preserving

...

current functionality) condence the UGen graph like some digital modulars do (simplifying groups of UGens internally). Well, maybe we could but even thinking about it makes my head hurt. IMHO we need more facilities for hot-swapping UGens, not less.

That's basically true. Even if the compiler produced machine code as fast as C, it would still issue calls to connect up UGens the same way it is done now.

Yes, I was talking about optimising UGen performance though. There are some digital modular synths that optimise by taking a group of "UGens" and converting those to instructions in a optimised way. I meant we couldn't do that as we want to be able to re-route them.

On the other hand, if the compiler really could generate code that fast, it would be worthwhile to implement UGens directly in ChucK.

I still think that would be worthwhile to have as a option. Even if it would be slow(er) it would be great for prototyping and would lead to cleaner code. For example; I sometimes use a overdrive that consists of a a set of UGens. When I use that a few times, on a few sounds, I end up with code that may work and be relatively fast but that's not so pretty.

I've been playing more and more with LLVM lately, but still too early to say anything. I was messing around with the ChucK compiler previously, and found a few places to play with its opcode generator. Don't hold your breath though, as I'm way too busy on other things. :) I'd love to spend some more time with it in the summer though.

Could you explain what you mean by "ChucK's opcode generator"? "Opcode" is a CSound term that translates roughly to "UGen", right? Kas.

Steve; That's basically true. It's not _impossible_ to optimize them

(inlining "tick" functions, etc), but it would be a completely different problem to tackle. A pretty interesting one however!

That would be a interesting approach indeed. We would need to be careful and make sure we could still consider the internal state of individual UGens in such a graph, as we may want to change the UGen graph with as few glitches as possible.

Sorry for the confusion. I meant opcode as it is used in talking about compilers. The word just refers to "operation codes", i.e., codes that represent machine instructions. In ChucK, its VM opcodes are actually instances of the Chuck_Instr class. Not the same as CSound's opcodes. (Though, shamefully, I don't actually know CSound very well.)

Ah, I see. IMHO CSound is nice as it's so old and established that many very worthwhile articles have been written with it in mind. It can pay off to be able to read it to be able to understand those articles. To me personally it does look a lot like a language that's decades old if we compare it to ChucK; I think you'll be fine without it. Anyway, that's something completely different indeed. Fortunately we have words like "spork" and "shred", at least those won't easily be confused with other concepts elsewhere! Yours, Kas.

Tom;

Really?

adc => StifKarp k => dac;

Look out, there could be a Gain behind you, too.

He he he. StifKarp doesn't do much more with it's input than blackhole does with it's gain though. I'm not sure it would even tick things connected to it, but it might. Kas.