[chuck-users] TCP protocol

Tue Dec 4 14:22:44 EST 2018

reading through the Aura paper ...

TCP and UDP packets both go over IP. At the IP level, they're handled the
same. The packet transit time of each is the same. They can both be fast!
They just have different properties.

the paper isn't particularly forthcoming with regards to Linux networking
arcanery. There are many places where UDP packets may be lost, and the
paper doesn't indicate the source of packet loss. On the sending side, each
outgoing UDP socket has a send buffer that buffers packets to be sent. If
that outgoing buffer gets full because your application is writing packets
into the buffer faster than your networking card can publish them on the
network, the packet will be dropped by the client, and it won't ever get
broadcast on the network. On the receiving side, each open UDP socket has a
receive buffer for packets that have been received by the network card but
have not been read out of the buffer by the application. If that buffer
gets full (because you're receiving packets faster than your application
can process them), the network card will drop the packets and you'll have
packet loss on the receiver.

The default UDP socket buffer sizes on Linux are pretty small. For running
a system with a high volume of UDP traffic, adjusting the kernel options
with respect to UDP buffer sizes goes a long way towards mitigating packet
loss. On my machine at least, the default UDP socket buffer read size is
~200kb; for a production system, I make that more like 25mb.

The third major type of packet loss is packets lost in transmission.
Although the paper doesn't indicate what fraction of dropped packets were
dropped in transit, I suspect that most of the dropped packets were
*not *dropped
in transit. My suspicion is based on the fact that I'm fairly certain that
if there was significant in-transit packet loss, there would have also been
significant in-transit packet loss in the TCP trials, and they would have
observed poor TCP performance due to packet re-transmission delays.

Given the nod to disabling Nagle's algorithm by setting TCP_NODELAY, my
guess is they didn't change the UDP buffer sizes, that much of the UDP
packet loss was due to UDP buffers being filled, and that there was
probably a lot of room for performance improvements to be had in the UDP
stack. Of course, they may have tried all this and it didn't make the
paper! That is to say, I would be cautious in assuming their conclusion
with regards to network protocols has been as fully explored as it could.

(fwiw, and this is rich, I forgot to set TCP_NODELAY on my own thing so
it's subject to Nagle's algorithm! But I can't work on it right now;
setting TCP_NODELAY on the client's TCP socket will yield latency
improvements)

On Tue, Dec 4, 2018 at 1:13 PM mario buoninfante <
mario.buoninfante at gmail.com> wrote:

> Hi Jordan,
>
>
> Thanks a lot for that, I'll give it a try as soon as possible ;)
>
>
> Cheers,
>
> Mario
>
>
> On 04/12/2018 17:07, Jordan Orelli wrote:
>
> ok, I had a little time to look at this, so I wrote you a simple proxy
> server in Go: https://github.com/jordanorelli/tuntun
>
> there's a binary release here if you want to just have a downloadable
> binary to check it out:https://github.com/jordanorelli/tuntun/releases
>
> and there's an example of how you'd use it from Chuck here:
> https://github.com/jordanorelli/tuntun/blob/master/ex/ex.ck
>
> this is a pretty straightforward proxying server to write. If you're not
> familiar with writing network servers, this is a good entry-level project
> for learning about it.
>
> Chuck's subprocess facilities are, ... well, pretty weird. It doesn't seem
> like you get the pid of the spawned process; there's no way to signal it,
> so there's no way to kill it from Chuck. This example leaves you with a
> zombie process.
>
> I'm very confused as to how it operates internally, because it blocks the
> entire Chuck VM ... *unless *you stick an ampersand at the end of your
> command, which seems to put the job in the background. It ... it kinda
> looks like it's calling exec to exec a bash shell in the *current *process?
> I'm ... a bit baffled by the behavior, to be honest, but I found a way to
> work it out.
>
> Running it this way means that Chuck launches the proxying server for you
> and can tell the proxying server the ports to use. You could also run the
> server yourself outside of Chuck to not have to deal with the weirdness of
> Chuck's subprocessing facilities.
>
>
> On Tue, Dec 4, 2018 at 10:59 AM Mark Cerqueira <mark.cerqueira at gmail.com>
> wrote:
>
>> Roger Dannenberg did some real-time music work and compared UDP and TCP.
>> His findings may surprise you! See 4.2 UDP vs. TCP here:
>> https://www.cs.cmu.edu/~rbd/papers/icmc01aura.pdf tl;dr - he ended up
>> going with TCP with some tweaks to make it more timely.
>>
>> Cheers!
>>
>> mc
>>
>> On Sun, Dec 2, 2018 at 12:47 PM Jordan Orelli <jordanorelli at gmail.com>
>> wrote:
>>
>>> I'm like ... 80% sure that there's no TCP-handling facilities built in,
>>> but I may be wrong here. If there is, I surely don't know about it.
>>>
>>> Can you tell us a little more about your use case?
>>>
>>> The design goals of TCP and ChucK are very different with respect to
>>> *time*. ChucK is designed around guarantees about time and timeliness.
>>> TCP is not designed around timeliness or latency in general: TCP is
>>> designed around guarantees about ordering (messages always appear in order)
>>> and delivery (messages are guaranteed to be delivered). Since every byte
>>> sent over a TCP socket has to be eventually acknowledged, and all bytes
>>> have to be processed in order, a stall in the network (a normal occurrence)
>>> could mean later messages being delayed, much like a traffic jam. It's not
>>> possible to have timing *guarantees *with TCP.
>>>
>>> I'm not sure how to find it in the source code, but maybe someone else
>>> here knows: does calling Std.system create a new shell process for each
>>> invocation? That would be a fairly inefficient way to go about things. Also
>>> since you're piping to another process, that's a new netcat process for
>>> each invocation (so it's either one or two processes per invocation), and a
>>> new TCP socket for every invocation. TCP is *especially slow *at *the
>>> very beginning of communication.*
>>>
>>> If I were in your shoes, I would probably write a separate program that
>>> acts as a server and serves an OSC-based protocol. This server would take
>>> your messages as OSC and translate them and then forward them to the
>>> intended recipient over just one TCP connection and continue to use that
>>> one connection the whole time. It may or may not respond to your ChucK
>>> program over OSC, depending on whether you want the ChucK program to get
>>> responses to its requests (probably not?).
>>>
>>>
>>> On Sat, Dec 1, 2018 at 1:25 PM mario buoninfante <
>>> mario.buoninfante at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>>
>>>> It seems like accessing the shell and use "netcat" (on Unix) is a
>>>> possible solution. Quite an *exotic* workaround but better than
>>>> nothing I'd say.
>>>>
>>>> Something like that seems to work:
>>>>
>>>> *// run ChucK with "--caution-to-the-wind"*
>>>>
>>>> *"echo -ne '" => string prefix;*
>>>> *"' | netcat 127.0.0.1 3333 " => string suffix;  // netcat <target ip>
>>>> <target port>*
>>>>
>>>> *while(true)*
>>>> *{*
>>>> *  Math.random2(0,127) => int r;*
>>>> *  prefix + Std.itoa(r) + suffix => string msg;*
>>>> *  Std.system(msg);*
>>>>
>>>> *  second => now;*
>>>> *}*
>>>>
>>>>
>>>> Please, let me know if anyone has a better solution.
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Mario
>>>> On 01/12/2018 18:04, mario buoninfante wrote:
>>>>
>>>> Hi,
>>>>
>>>>
>>>> I also tried opening a file in /dev/tcp/<target ip>/<target port>, but
>>>> it didn't work. I'm on Ubuntu 16.04. Any idea?
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Mario
>>>>
>>>>
>>>> On 30/11/2018 16:52, Mario Buoninfante wrote:
>>>>
>>>> Hi,
>>>>
>>>> Does anyone know if it's possible to use TCP instead of UDP in ChucK?
>>>>
>>>> Cheers,
>>>> Mario
>>>>
>>>>
>>>> --
>>>> Electronic Musician, Creative Coder, QA Engineerhttps://vimeo.com/creativecodingsalernohttp://mbuoninfante.tumblr.com/https://github.com/mariobuoninfantehttps://bitbucket.org/mariobuoninfante/
>>>>
>>>> _______________________________________________
>>>> chuck-users mailing list
>>>> chuck-users at lists.cs.princeton.edu
>>>> https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
>>>>
>>> _______________________________________________
>>> chuck-users mailing list
>>> chuck-users at lists.cs.princeton.edu
>>> https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
>>>
>> _______________________________________________
>> chuck-users mailing list
>> chuck-users at lists.cs.princeton.edu
>> https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
>>
>
> _______________________________________________
> chuck-users mailing listchuck-users at lists.cs.princeton.eduhttps://lists.cs.princeton.edu/mailman/listinfo/chuck-users
>
> --
> Electronic Musician, Creative Coder, QA Engineerhttps://vimeo.com/creativecodingsalernohttp://mbuoninfante.tumblr.com/https://github.com/mariobuoninfantehttps://bitbucket.org/mariobuoninfante/
>
> _______________________________________________
> chuck-users mailing list
> chuck-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/chuck-users/attachments/20181204/d06b6331/attachment-0001.html>