OSC reliability issues
Hello list, I have been running experiments with Chuck's OSC implementation and have found that when I have an instance of Chuck producing regular OSC events under a relatively heavy load (heavier than the example files), the receiver only gets a few of these events - perhaps less than 20% of them. This situation makes it very difficult for parallel instances of Chuck (on different machines) to communicate reliably. I have reproduced this on both Mac and Linux (Ubuntu) installations. I am curious if anyone else on the list has come across this issue. Does anyone have suggestions for dealing with OSC communication reliably? Best, David
Just to narrow this issue down- the Chuck app is sending regular messages ( or should be ) and a second, ( chuck/ non-chuck ) application on the ( same / separate ) machine is not receiving? David Michael wrote:
Hello list,
I have been running experiments with Chuck's OSC implementation and have found that when I have an instance of Chuck producing regular OSC events under a relatively heavy load (heavier than the example files), the receiver only gets a few of these events - perhaps less than 20% of them. This situation makes it very difficult for parallel instances of Chuck (on different machines) to communicate reliably.
I have reproduced this on both Mac and Linux (Ubuntu) installations. I am curious if anyone else on the list has come across this issue. Does anyone have suggestions for dealing with OSC communication reliably?
Best, David _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
Chuck is sending regular "heartbeat" messages to a remote OSC app
(written in Ruby: http://www.funaba.org/en/ruby.html#osc). When Chuck
is only in the business of sending these messages, they all reach
their target. As Chuck takes on more tasks, the events stop arriving
at the remote machine.
I have not tested this between 2 chuck applications on different hosts
under heavy loads, but I will do this tonight or tomorrow and post the
results back to this space.
The reason I would like to send these messages reliably is because I
am trying to figure out a way to get data out of Chuck. There is a
posting on this list with a method for dumping the debugging output to
a file instead of the console, but this does not work when running
Chuck with separate class files (it only dumps the last one in the arg
list). The alternative seems to be cut and paste all my classes into
one huge file. I just thought that there would be a more elegant way
to get data out of Chuck, perhaps by OSC so that another language
could handle formating, file i/o, etc...
Thanks
David
On 1/23/07, Philip Davidson
Just to narrow this issue down-
the Chuck app is sending regular messages ( or should be ) and a second, ( chuck/ non-chuck ) application on the ( same / separate ) machine is not receiving?
David Michael wrote:
Hello list,
I have been running experiments with Chuck's OSC implementation and have found that when I have an instance of Chuck producing regular OSC events under a relatively heavy load (heavier than the example files), the receiver only gets a few of these events - perhaps less than 20% of them. This situation makes it very difficult for parallel instances of Chuck (on different machines) to communicate reliably.
I have reproduced this on both Mac and Linux (Ubuntu) installations. I am curious if anyone else on the list has come across this issue. Does anyone have suggestions for dealing with OSC communication reliably?
Best, David _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
Hi David, Yikes! It sounds like you are running into two big limitations of ChucK--lack of file i/o, and lack of a reliable networking API. OSC in ChucK, as in most other OSC implementations, uses UDP as its transport protocol, which basically means that one's OSC messages sent from ChucK are not guaranteed to ever arrive at their destination, nor are they guaranteed to arrive in order, nor are they guaranteed to arrive only once. Under heavy load it is very likely that the network will drop packets. Believe it or not this is actually desirable for many computer audio applications, as the latency overhead of a reliable transport protocol like TCP is often too great. But it sounds like its definitely not desirable for your situation. The best solution would be to use TCP, which guarantees packet arrival. Unfortunately ChucK doesn't support TCP in any form (yet), so thats not that helpful of a suggestion. Another route would be to implement TCP-like reliability mechanisms over ChucK's OSC API. To be honest this sounds like a huge pain too--implementing a light TCP stack over OSC over UDP in ChucK, no thanks! Unfortunately until file i/o comes along this could be your only option.
There is a posting on this list with a method for dumping the debugging output to a file instead of the console, but this does not work when running Chuck with separate class files
Could you be a bit more specific about this particular problem, possibly providing an example? It is possible to pipe ChucK's <<<
style output to a file or another application--is this what you are saying does not work for you? I think trying to get this working would be the best way to go right now.
spencer On Jan 24, 2007, at 9:20 AM, David Michael wrote:
Chuck is sending regular "heartbeat" messages to a remote OSC app (written in Ruby: http://www.funaba.org/en/ruby.html#osc). When Chuck is only in the business of sending these messages, they all reach their target. As Chuck takes on more tasks, the events stop arriving at the remote machine.
I have not tested this between 2 chuck applications on different hosts under heavy loads, but I will do this tonight or tomorrow and post the results back to this space.
The reason I would like to send these messages reliably is because I am trying to figure out a way to get data out of Chuck. There is a posting on this list with a method for dumping the debugging output to a file instead of the console, but this does not work when running Chuck with separate class files (it only dumps the last one in the arg list). The alternative seems to be cut and paste all my classes into one huge file. I just thought that there would be a more elegant way to get data out of Chuck, perhaps by OSC so that another language could handle formating, file i/o, etc...
Thanks David
On 1/23/07, Philip Davidson
wrote: Just to narrow this issue down-
the Chuck app is sending regular messages ( or should be ) and a second, ( chuck/ non-chuck ) application on the ( same / separate ) machine is not receiving?
David Michael wrote:
Hello list,
I have been running experiments with Chuck's OSC implementation and have found that when I have an instance of Chuck producing regular OSC events under a relatively heavy load (heavier than the example files), the receiver only gets a few of these events - perhaps less than 20% of them. This situation makes it very difficult for parallel instances of Chuck (on different machines) to communicate reliably.
I have reproduced this on both Mac and Linux (Ubuntu) installations. I am curious if anyone else on the list has come across this issue. Does anyone have suggestions for dealing with OSC communication reliably?
Best, David _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
On Thu, 25 Jan 2007, Spencer Salazar wrote:
OSC in ChucK, as in most other OSC implementations, uses UDP as its transport protocol, which basically means that one's OSC messages sent from ChucK are not guaranteed to ever arrive at their destination, nor are they guaranteed to arrive in order, nor are they guaranteed to arrive only once. Under heavy load it is very likely that the network will drop packets. Believe it or not this is actually desirable for many computer audio applications, as the latency overhead of a reliable transport protocol like TCP is often too great.
If timing is critical, I wouldn't recommend TCP. I foold around a bit with using TCP to determine ensemble (yay plOrK!) timing, and the variance was too much. UDP worked and I didn't notice as great a packet loss as I had expected, although 1) it was a 'closed' network, and 2) my packet traffic wasn't high. If you need to have cake and eat it too, then you can probably use TCP but you would need to imbed some kind of time-stamping to allow you to compensate for late arrivals, or use UDP with sequence numbers to compensate for dropped packets. This would assume that you put some timing 'intelligence' on the receiver end so that it can continue when things get messed up. brad http://music.columbia.edu/~brad
Hi Brad
If timing is critical, I wouldn't recommend TCP. I foold around a bit with using TCP to determine ensemble (yay plOrK!) timing, and the variance was too much. UDP worked and I didn't notice as great a packet loss as I had expected, although 1) it was a 'closed' network, and 2) my packet traffic wasn't high.
In my particular application, timing isn't critical - but receiving the message is (TCP). I am also on a closed network, so my sneaking suspicion is that UDP in Chuck can be used reliably, I just do not have the right pieces in place or a bad implementation on my end.
On Thu, 25 Jan 2007, David Michael wrote:
In my particular application, timing isn't critical - but receiving the message is (TCP).
I wonder about something -- this is based on really ancient knowledge (like 10+ years ago...) and contemporary implementations of socket-reading code may just Do the RIght Thang, but I recall sometimes having to do repeated reads on a socket to get the full packet, especially if some 'non-standard' size was being transmitted. Something like this: amt = read(ns, (void *)sptr, sizeof(struct sockdata)); while (amt < sizeof(struct sockdata)) amt += read(ns, (void *)(sptr+amt), sizeof(struct sockdata)-amt); I also haven't looked at cHUcK's implementation of of the socket stuff. brad http://music.columbia.edu/~brad
Thanks to all who replied I have done some experimenting with OSC over the past few days and found that Chuck's OSC works well, even under load. The issues I was having with messages being sent ~from~ Chuck reliably had to do with where in the program I was triggering them. I am still in the process of tracking this bug down. This is most likely the source of my reported unstable firing patterns from the OSC object. As for the Ruby OSC server implementation... hmm... that one's a bugger - I could possibly work with it some more to get it running smoothly, but for the moment, I am going to give the Python implementation a go.
Hi Spencer
OSC in ChucK, as in most other OSC implementations, uses UDP as its transport protocol, which basically means that one's OSC messages sent from ChucK are not guaranteed to ever arrive at their destination, nor are they guaranteed to arrive in order, nor are they guaranteed to arrive only once. Under heavy load it is very likely that the network will drop packets. Believe it or not this is actually desirable for many computer audio applications, as the latency overhead of a reliable transport protocol like TCP is often too great.
I understood this about UDP in general, but I thought that packets would be dropped by the ~receiver~ rather than the sender (because of network congestion, collisions, etc.) What appears to be happening is that when a Chuck OSC sender is under load, then it fails to fire off the message. So while UDP delivery is not guaranteed, and this is generally not a big deal, it seems that Chuck is getting bogged down somewhere and not actually firing off the OSC event. I need to write some simple test cases for this to actually prove that it is the sender and not the receiver or the network that has the problem - Ill also take the Ruby OSC out of the equation.
Could you be a bit more specific about this particular problem, possibly providing an example? It is possible to pipe ChucK's <<<
style output to a file or another application--is this what you are saying does not work for you? I think trying to get this working would be the best way to go right now.
Yes, this posting talks about that: https://lists.cs.princeton.edu/pipermail/chuck-users/2006-January/000293.htm... I am calling classes from several different files in order, all which produce console output, and some of which I am interested in capturing as text (until file i/o comes along), like so: $ chuck file1 file2 file3 file4 >& output.txt Since "file 4" is usually my "main()" and does not contain any console output via the <<<>>> syntax, the txt file is empty. It seems that a workaround to this would be to concatenate all the files into one, and run a much simpler command: $ chuck all_files_in_one.ck >& output.txt For sure, I need to do more testing. It is entirely possible that I have a logical lapse somewhere, especially since my knowledge of piping is newish. Ultimately I think that you are right, redirecting the <<<>>> output to a file or application will be the best solution. Just for informations sake, I would like to get some control data generated by Chuck into Matlab (I know Chuck can deal with .mat files, but this is only for audio matrices right??). If anyone on list has experiences with this, do tell. Thanks David
participants (4)
-
Bradford Garton
-
David Michael
-
Philip Davidson
-
Spencer Salazar