Re: fwd: [chuck-dev] Big Nasty Mutex (tm) + Jack + ChucK evilness
On Sun, 2005-13-03 at 03:35 -0500, Ge Wang wrote:
Greetings,
This is continued from the JACK correctness thread.
The problem has been plaguing Jack/ChucK users (all 3 of them by now). Let's figure it out!
Dave R. or Gary S. please help lead the way!
Best, Ge!
Well, I've done about as good a job as possible at screwing up this cross-list conversation. Anyway! Summary: Basically, the problem is that ChucK is a horrible realtime jack client (ie it zombies all the time) because the audio callback sleeps, waiting for ChucK's synthesis thread to finish. The solution is to make the audio callback thread the synthesis thread (ie make the audio callback drive all synthesis), eliminating this race. This is how things "should" be. Making the callback drive everything in a deterministic way would make ChucK a rock-solid jack client in realtime. I can see why this might be a nuisance though, because ChucK is probably driven by wall clock time (judging from the timing system of the language). It could be not that difficult though - I'm not sure how the ChucK vm core works yet, just the part that interacts with the audio callback. Essentially all we actually need is a function that can compute n samples of chuck audio on request. So I guess it comes down to either: a) It's possible - how to do it? b) It's not possible - why, and how can we make it at least a bit better? Cheers, -DR-
Hi Everyone, I hope to have some time later this week to look at the RtAudio Jack code, though it appears that the main problems lie elsewhere in the ChucK code. With respect to the "big fat mutex", it is there to protect _all_ the RtAudio functions from the user making multiple calls simultaneously (perhaps on a multi-processor system) that might interfere with each other. In the long run, it might be unnecessary and perhaps we can make do without it, but I chose to be "safer" than "sorrier". I'll add "my two cents" with regard to the scheduling: The Jack community espouses the callback paradigm as the only solution to realtime audio i/o ... I've been hearing this loudly for the last 4-5 years. Then there's a bunch of us that used to do things with blocking calls and know how easy it was to make them work with the same consistency as the callback paradigm. This last sentence will raise arguments like "there is no way that a blocking paradigm can be as robust as a callback paradigm". I'm not here to convince ... I simply know from experience. In fact, I find it much easier to work with blocking functions and embed them in threads to "simulate" a callback scheme ... the reverse is nearly impossible (yes, I agree with Dave that my attempt to "shoehorn" Jack into a blocking scheme is not robust). We're dealing with computers here and no scheme, callback or blocking, will ever provide perfect "glitch-free" audio in all circumstances. I can make Jack glitch on my Linux system by doing lots of processing and moving windows around and such. We must admit that there are limits to what the computer can do and when you start to push the processor close to those limits, no scheme will be perfect. I just wish the "callback crowd" would get off their high-horse and admit as much. I'm not saying this against Dave. Dave knows how Jack works and if ChucK is to interface cleanly with Jack, the changes he suggests are likely necessary to make that happen. --gary On Mar 13, 2005, at 8:35 PM, Dave Robillard wrote:
On Sun, 2005-13-03 at 03:35 -0500, Ge Wang wrote:
Greetings,
This is continued from the JACK correctness thread.
The problem has been plaguing Jack/ChucK users (all 3 of them by now). Let's figure it out!
Dave R. or Gary S. please help lead the way!
Best, Ge!
Well, I've done about as good a job as possible at screwing up this cross-list conversation. Anyway! Summary:
Basically, the problem is that ChucK is a horrible realtime jack client (ie it zombies all the time) because the audio callback sleeps, waiting for ChucK's synthesis thread to finish.
The solution is to make the audio callback thread the synthesis thread (ie make the audio callback drive all synthesis), eliminating this race. This is how things "should" be. Making the callback drive everything in a deterministic way would make ChucK a rock-solid jack client in realtime.
I can see why this might be a nuisance though, because ChucK is probably driven by wall clock time (judging from the timing system of the language). It could be not that difficult though - I'm not sure how the ChucK vm core works yet, just the part that interacts with the audio callback.
Essentially all we actually need is a function that can compute n samples of chuck audio on request.
So I guess it comes down to either:
a) It's possible - how to do it?
b) It's not possible - why, and how can we make it at least a bit better?
Cheers,
-DR-
_______________________________________________ chuck-dev mailing list chuck-dev@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-dev
On Sun, 2005-13-03 at 09:43 -0500, Gary P.Scavone wrote:
Hi Everyone,
I hope to have some time later this week to look at the RtAudio Jack code, though it appears that the main problems lie elsewhere in the ChucK code. With respect to the "big fat mutex", it is there to protect _all_ the RtAudio functions from the user making multiple calls simultaneously (perhaps on a multi-processor system) that might interfere with each other. In the long run, it might be unnecessary and perhaps we can make do without it, but I chose to be "safer" than "sorrier".
I'd say it's pretty reasonable to require clients to not do thing like that. The mutex really should go, anyway. Maybe some functions need to be changed to be okay to execute in parallel without locking?
I'll add "my two cents" with regard to the scheduling: The Jack community espouses the callback paradigm as the only solution to realtime audio i/o ... I've been hearing this loudly for the last 4-5 years. Then there's a bunch of us that used to do things with blocking calls and know how easy it was to make them work with the same consistency as the callback paradigm. This last sentence will raise arguments like "there is no way that a blocking paradigm can be as robust as a callback paradigm". I'm not here to convince ... I simply know from experience. In fact, I find it much easier to work with blocking functions and embed them in threads to "simulate" a callback scheme ... the reverse is nearly impossible.
Hmm. I would say the exact opposite actually. Wrapping a callback function in a blocking IO system is incredibly trivial - just call the callback in your main loop to get the audio, and push it on it's way. Doesn't get much simpler than that. Wrapping BIO in a callback system requires hacky race systems with multiple threads waiting on each other and hoping they finish on time. Hardly as simple or elegant as just calling a function.
We're dealing with computers here and no scheme, callback or blocking, will ever provide perfect "glitch-free" audio in all circumstances. I can make Jack glitch on my Linux system by doing lots of processing and moving windows around and such. We must admit that there are limits to what the computer can do and when you start to push the processor close to those limits, no scheme will be perfect. I just wish the "callback crowd" would get off their high-horse and admit as much.
(Moving windows around most definitely will not glitch jack on a properly configured system) You're right blocking I/O is perfectly fine in some cases - when there is exactly one realtime audio program pumping it's audio in to something. (Heck, jack itself pushes audio to the sound card this way!). The problem is when you have multiple apps running, depending on each other's audio, like Jack allows. This situation is entirely different - blocking I/O is crap here, and callback is the only way to go. Blocking may have worked well in your experience, but not on platforms like jack that allow these sorts of things. With callbacks you have one realtime thread generating everything and there's no problems. Multiple realtime thread for every little app causes tons of problems (and puts you at the mercy of the OS's scheduler) But jack or no jack, a realtime app should be - well, realtime. Deterministic. The only difference with jack is people actually notice then things fail because it kicks apps out, whereas on OSX or windows it will just glitch and happily continue on.
I'm not saying this against Dave. Dave knows how Jack works and if ChucK is to interface cleanly with Jack, the changes he suggests are likely necessary to make that happen.
Well, the proof is in the pudding. I can tell you that every single audio app I have that does blocking IO and then shoehorns it into the jack callback is completely unreliable. Zynaddsubfx, ChucK, even pd (which is a shame) - I would never, ever trust these apps in a live scenario. On the other hand, I'm working on a certain project that is 100% realtime safe in the callback - it has never zombied, once, ever, regardless of load. I can trust it, I know it's not going to die. SuperCollider is also designed like this, and it's rock solid in my experience. I want ChucK to be like that! The advantages would definitely trickle over to other platforms as well, just not in as obvious a way. -DR-
I must confess. I don't know Jack. So... Before we, heh, re-implement ChucK and RtAudio, I would like to know the following, at least: 1. Why is the mutex actually bad? More specifically, how exactly does it cause Jack to fail or zombie? ChucK does not call RtAudio from multiple threads, so the mutex shouldn't be blocking at all. 2. If the concern is things taking too long in the callback, how does putting audio synthesis in the callback help? The current scheme has the audio pre-computed before the callback. Putting the most time consuming part of ChucK inside the callback will more likely delay the callback. Dave, you said that replacing blocking I/O with computing audio in callback makes things robust, regardless of load. How does Jack know the difference between the callback blocking versus just taking a long time computing audio? I am clearly missing something here. Related question: 3. Under what conditions does Jack decide to boot a process? 4. If our blocking scheme is the problem, then why doesn't sndpeek work? The callback just copies the input buffer and returns. Confused, Ge!
On Mon, 2005-14-03 at 03:05 -0500, Ge Wang wrote:
I must confess. I don't know Jack.
So...
Before we, heh, re-implement ChucK and RtAudio, I would like to know the following, at least:
1. Why is the mutex actually bad? More specifically, how exactly does it cause Jack to fail or zombie? ChucK does not call RtAudio from multiple threads, so the mutex shouldn't be blocking at all.
2. If the concern is things taking too long in the callback, how does putting audio synthesis in the callback help? The current scheme has the audio pre-computed before the callback. Putting the most time consuming part of ChucK inside the callback will more likely delay the callback.
Dave, you said that replacing blocking I/O with computing audio in callback makes things robust, regardless of load. How does Jack know the difference between the callback blocking versus just taking a long time computing audio? I am clearly missing something here.
Related question:
3. Under what conditions does Jack decide to boot a process?
4. If our blocking scheme is the problem, then why doesn't sndpeek work? The callback just copies the input buffer and returns.
I think sndpeek might have seperate issues. I've never even managed to get a window to display for any amount of time, realtime or no.
The current scheme has the audio pre-computed before the callback. Putting the most time consuming part of ChucK inside the callback will more likely delay the callback.
I would rephrase it: "The current scheme MIGHT have the audio pre-computed before the callback". This is one of those things best explained with questions: - How do you know your thread is going fast enough? Or even too fast? - How do you know how much audio is actually demanded of you? - What about offline processing? (in theory) - Who gets more scheduling priority? Your thread? Jack thread? - Will usleep() return on time? Will pthread_mutex_lock()? Will the synthesis thread get back in time? - What does "in time" mean anyway? How much time do we really have? The answer to every one of these question is "I dunno". It's all about determinism. Not really a specific Jack thing, it's just hard realtime requirements. pthread_mutex_lock is not deterministic. usleep() isn't deterministic (it is perfectly acceptable for usleep(1) to sleep for 10 /seconds/ if it wants to). Whether or not the other thread makes it back in time is not deterministic. The problem with attempting to wait on the other thread is - how long can you wait? You have no way of knowing what an acceptable time to wait is. Jack does, but you don't. So, if you don't wait long enough, you won't get your audio. If you wait too long, jack will kick you out. Having the other realtime thread running in parallel causes a whole whack of scheduling problems too, which might be a cause of a lot of the problems (I'm no kernel hacker) - the idea behind Jack is that there is /one/ realtime priority audio thread that does everything and knows what is going on, that's where the robustness comes from. I find the best way to think about it is in terms of offline processing, ie everything moving reeally slowly - imagine the audio callback being called every 5 seconds. Everything should still function perfectly well (gigantic queues of timestamped events aside). Currently you'd have a realtime thread going crazy, generating way, way too much audio with nowhere to store it (of course, the opposite is what's happening right now). Everything is so, so much nicer when the audio callback just says "compute and give me 64 samples now". To be honest, I don't know exactly when jack will decide it's had enough of misbehaving clients, I'm not a jack developer. I understand what you mean, time is time. Unfortunately it doesn't work that way, but I freely admit to not really knowing how jack itself knows the difference. I assume the lack of determinism just rears it's ugly head the your callback takes too much time, but there could be more clever things going on. No idea - I just know that making the audio callback deterministic is what you have to do (ie it's a known requirement documented alongside Jack's API), and that my personal experience (and reading code) has backed that up. It's easy to get bitter at Jack for being more picky about these things, but realtime systems should be coded properly regardless - determinism is king. I'm sure that on OSX if you run ChucK alongside a few other audio programs and really thrash the CPU, you'll get lots of dropouts as well. Ever tried it? (I don't have a Mac) Sorry if I don't understand this stuff very well at a super-low level, I just know what I have to know as an app developer.. I learned all this the hard way, that's for sure. Cheers, -DR- P.S. I unintentionally come off sounding really negative here.. I really think ChucK is the greatest thing since sliced bread. Bravo. :)
Well, the proof is in the pudding. I can tell you that every single audio app I have that does blocking IO and then shoehorns it into the jack callback is completely unreliable. Zynaddsubfx, ChucK, even pd (which is a shame) - I would never, ever trust these apps in a live scenario.
On the other hand, I'm working on a certain project that is 100% realtime safe in the callback - it has never zombied, once, ever, regardless of load. I can trust it, I know it's not going to die. SuperCollider is also designed like this, and it's rock solid in my experience.
I, for one, will take my chances with ChucK, no matter how much it dies or sucks!!! For great justice!!! (And yes, we do need to make ChucK better)
On Mon, 2005-14-03 at 03:52 -0500, Ge Wang wrote:
Well, the proof is in the pudding. I can tell you that every single audio app I have that does blocking IO and then shoehorns it into the jack callback is completely unreliable. Zynaddsubfx, ChucK, even pd (which is a shame) - I would never, ever trust these apps in a live scenario.
On the other hand, I'm working on a certain project that is 100% realtime safe in the callback - it has never zombied, once, ever, regardless of load. I can trust it, I know it's not going to die. SuperCollider is also designed like this, and it's rock solid in my experience.
I, for one, will take my chances with ChucK, no matter how much it dies or sucks!!!
For great justice!!!
(And yes, we do need to make ChucK better)
Hehe. I knew mentioning SuperCollider would provoke some kind of response. :) FWIW, I intentionally avoided learning SC because I like ChucK's philosophy much better. -DR-
participants (3)
-
Dave Robillard -
Gary P.Scavone -
Ge Wang