other ways to live-code with time

Tom Lieber

28 Apr 2009 28 Apr '09

9:36 p.m.

impromptu is a live-coding environment for OS X that looks very interesting, definitely something I need to try out once I've escaped the clutches of my end-of-year work. http://impromptu.moso.com.au/ I thought I'd bring it up not only because it's interesting, but because its tutorial on "time" contrasts impromptu's asynchronous model with ChucK's synchronous one: in impromptu, time passes while your code is running. You can read the tutorial here: http://impromptu.moso.com.au/tutorials/time.html Just think... you could load a WAV file without hanging the VM! It's interesting to see what you get when you trade one complication (waiting on slow shreds) for another (having "now" change beneath your feet). Here are some, from the tutorial: "There are a couple of gotcha's to keep in mind when doing schedule and forget programming. The first thing to keep in mind is that (now) is an airy fairy thing. In the example below the two notes may be scheduled to play on the same sample - but then again, they may not - (now) may have moved forward in time between the two calls (even if evaluated at the same time). (play-note (now) piano 60 80 *second*) (play-note (now) piano 72 80 *second*) "Often this lack of precision is fine (i.e. too small a change to be noticeable) but where absolute accuracy is required a time variable should be used. (let ((time (now))) (play-note time piano 60 80 *second*) (play-note time piano 72 80 *second*)) "This inaccuracy becomes more of an issue when amplified over time such as using (now) inside a recursive callback loop. We can avoid the problem by precisely incrementing a time value between each recursive callback (note that any arguments required by the function being called back must also be passed to callback). ;This is bad (define loop (lambda () (play-note (now) piano 60 80 *second*) (callback (+ (now) *second*) 'loop))) (loop) ;This is good (precise time arg is now incremented each recursion) (define loop (lambda (time) (play-note time piano 60 80 *second* ) (callback (+ time *second*) 'loop (+ time *second*)))) (loop (now)) "The second major gotcha in recursive callback loops is that (now) is now. Code requires some time to execute. If you are executing a call to evaluate a note (now) by the time the code is evaluated it will already be late. You should always try to schedule your code execution ahead of the scheduled time of your tasks. ;This is best (callback happens 4100 samples earlier than new time) (define loop (lambda (time) (play-note time piano 60 80 1.0) (callback (+ time 40000) 'loop (+ time 44100)))) (loop (now)) "In the this-is-good version of loop the time sent as an argument to loop is exactly the same time as the scheduled callback time. The problem with this is that the next note needs to be scheduled at exactly the same time that the function is called. The note will always be late. The this-is-best version schedules the callback just ahead of the time that we want to schedule the note. This gives us 4100 samples to execute the code to schedule the note before the note is required to sound." The last sentence reminds me of the thread a while back about benchmarking UGens. :) -- Tom Lieber http://AllTom.com/

Show replies by date

Kassen

29 Apr 29 Apr

9:32 a.m.

Tom;

...

impromptu is a live-coding environment for OS X that looks very interesting, definitely something I need to try out once I've escaped the clutches of my end-of-year work.

Yes! It looks great, not in the least because of the utterly amazing demonstrations Andrew Sorensen gives of it; his videos are awe-inspiring. Impromptu is one of the most interesting things exclusive to the OSX platform, IMHO.

...

I thought I'd bring it up not only because it's interesting, but because its tutorial on "time" contrasts impromptu's asynchronous model with ChucK's synchronous one: in impromptu, time passes while your code is running.

I agree; quite interesting. What's not so clear to me though is how this scheduling of notes interacts with Impromptu mainly being a host for AU plugins. That could become quite relevant, for example if one could have those run on a separate core or cpu.

...

Just think... you could load a WAV file without hanging the VM!

Well... Yes, or compile a more complicated Shred.... but aren't we talking about a trade between a stuttering sound and delaying some processes here? Once the CPU resources run out something, somewhere, will have to give, regardless of the model used. SC users tend to point out that their server-client model has advantages there but I'm not sure I see them. Once the cpu runs out that's that. Maybe loading WAV's is a exception as that doesn't actually cost CPU as much as the it takes memory and HD bandwith but we can load those in chunks, I don't have much experience with that though.

...

"In the this-is-good version of loop the time sent as an argument to loop is exactly the same time as the scheduled callback time. The problem with this is that the next note needs to be scheduled at exactly the same time that the function is called. The note will always be late. The this-is-best version schedules the callback just ahead of the time that we want to schedule the note. This gives us 4100 samples to execute the code to schedule the note before the note is required to sound."

The last sentence reminds me of the thread a while back about benchmarking UGens. :)

Yes, there are certainly similarities there. I do wonder about the exact interaction between Impromptu and the plugins it uses now. I'm also still interested in the interaction between ChucK and the various processes on the host computer that may affect and interact with it's performance; things like the time needed to load or save a file, the MIDI cue and so on. For some we have a model for interacting with it (getting things off the HID input cue) for others they are planned (loading and saving text files) and in other places this hasn't yet been given much attention. While I'm at it; a while ago I posted about timestamps for OSC messages, particularly because of the NetClock initiative. This hasn't yet received much debate. Is that because people aren't interested at all? Is it perhaps unclear why this could be nice? Maybe everybody agreed with me so much that nobody felt a need to add anything? (hey, I can dream!). Yours, Kas.

Tom Lieber

11:36 a.m.

On Wed, Apr 29, 2009 at 9:32 AM, Kassen wrote:

...

...
Just think... you could load a WAV file without hanging the VM!

Well... Yes, or compile a more complicated Shred.... but aren't we talking about a trade between a stuttering sound and delaying some processes here? Once the CPU resources run out something, somewhere, will have to give, regardless of the model used. SC users tend to point out that their server-client model has advantages there but I'm not sure I see them. Once the cpu runs out that's that.

Maybe loading WAV's is a exception as that doesn't actually cost CPU as much as the it takes memory and HD bandwith but we can load those in chunks, I don't have much experience with that though.

I assumed from the model that the reason "now" could change was that user code could be pre-empted to make framerate and sample rate deadlines. I also assumed that loading a WAV file is a lot more about waiting on the disk than the CPU. I could be wrong on both counts..

...

While I'm at it; a while ago I posted about timestamps for OSC messages, particularly because of the NetClock initiative. This hasn't yet received much debate. Is that because people aren't interested at all? Is it perhaps unclear why this could be nice? Maybe everybody agreed with me so much that nobody felt a need to add anything? (hey, I can dream!).

I can see why it would be nice, even though I don't need it. ;p A lot of rehearsal time in PLOrk this past semester was dedicated to synchronizing the performers because we lacked good network synchronization. So having NetClock as an option would have been great. -- Tom Lieber http://AllTom.com/

Daniel Trueman

12:27 p.m.

i've come to view this as a feature, not a bug... ;--} On Apr 29, 2009, at 11:36 AM, Tom Lieber wrote:

...

A lot of rehearsal time in PLOrk this past semester was dedicated to synchronizing the performers because we lacked good network synchronization.

Kassen

12:47 p.m.

Dan;

...

i've come to view this as a feature, not a bug... ;--}

I certainly feel that PLOrk's "conductor" approach is a very interesting idea that does expose a lot of the things we often assume to be given for what they are. Still; it doesn't need to be a either/or thing. In case of emergency you can always add a random offset to timestamps so the amount of jitter can be set to suit the mood. :¬p Here's one I prepared earlier; //returns a random duration in a given range //works like Std.rand2(int, int) and Std.rand2f(float, float) fun dur rand2dur( dur min, dur max) { return min + Std.rand2f(0,1)::(max - min); } Kas.

Daniel Trueman

3:36 p.m.

...

...
i've come to view this as a feature, not a bug... ;--}

I certainly feel that PLOrk's "conductor" approach is a very interesting idea that does expose a lot of the things we often assume to be given for what they are.

Still; it doesn't need to be a either/or thing.

oh, i know this, and we do both. until this year we had excellent success synchronizing the ensemble over the network; i've written about this elsewhere. this year we've been exploring other ways of doing it that i find quite interesting, but we've also had some more difficulties with the network sync, in part (i think) because we've gotten larger; synchronizing 30 laptops seems to be harder than 15. but even clock/timestamp syncing is not quite the same as the syncing we've done in the past, where a single server machine can say to all the other machines, do THIS, NOW. we've been able to do this in the past with 15 machines, and hopefully we'll be able to figure out a way to do it well (wirelessly) with >15. so, while i'm looking forward to having a good timestamp system of some sort, it will be one of several ways we go about "syncing." dt

...

In case of emergency you can always add a random offset to timestamps so the amount of jitter can be set to suit the mood. :¬p

Here's one I prepared earlier;

//returns a random duration in a given range //works like Std.rand2(int, int) and Std.rand2f(float, float) fun dur rand2dur( dur min, dur max) { return min + Std.rand2f(0,1)::(max - min); }

Kas. _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

Kassen

4:19 p.m.

Dan;

...

oh, i know this, and we do both.

I was teasing a bit; the conducting thing is quite inovative, creative, and likely very conductive to expression.... but it's also quite funny to me.

...

until this year we had excellent success synchronizing the ensemble over the network; i've written about this elsewhere.

You wrote about dealing with a wireless network for musical sync? Do you have a link for that? That could be good to send to the NetClock list as well because more people had that issue. I think SLUB simply switched to using cables but then they are just 3 guys. It's probably more convenient to use cables when you have three friends than when you have a group of 30 students.

...

this year we've been exploring other ways of doing it that i find quite interesting, but we've also had some more difficulties with the network sync, in part (i think) because we've gotten larger; synchronizing 30 laptops seems to be harder than 15.

This makes perfect sense. I also wouldn't be surprised if Apple occasionally tweaked their implementation and likely they are more concerned with things like throughput for things like streamed video than with syncing up 30 performers. There are quite a few variables.

...

but even clock/timestamp syncing is not quite the same as the syncing we've done in the past, where a single server machine can say to all the other machines, do THIS, NOW. we've been able to do this in the past with 15 machines, and hopefully we'll be able to figure out a way to do it well (wirelessly) with >15. so, while i'm looking forward to having a good timestamp system of some sort, it will be one of several ways we go about "syncing."

I see. That indeed is different. NetClock is more about sharing a clock that sends pulses (or events in ChucK's case). From the user's perspective I think it would work like the clock I've been using when livecoding lately (I'll post it below to illustrate). That need not be a issue; maybe we can work at interoperability in the interest of cross-platform jamming. I'd also be happy to implement NetClock for ChucK myself. If I'd have access to timestamps I think I could do it entirely in ChucK itself. I hope nobody minds that I'm a bit fanatical about this but I've been very interested in synced livecoding jams since I realised the guy who has a radio show before mine uses SC and the one after me uses PD. Synced jam session would be nice for us; I'm purely being selfish here :¬). Then again it works on SC, Perl and Haskel now, I think, with Fluxus, Impromptu and PD in the works, somebody mentioned Processing as well. To me that's exciting; I don't like MIDI that much but I do like the convenience of MIDI clock and how it allows for jam sessions without having to first agree on some protocol. It's also interesting to me how all of these systems deal with time and timing in quite different ways so they will likely all abstract this shared clock in a different way towards the user; to me that's a beautifull thing. Yours, Kas.

Daniel Trueman

8:12 p.m.

...

...
until this year we had excellent success synchronizing the ensemble over the network; i've written about this elsewhere.

You wrote about dealing with a wireless network for musical sync? Do you have a link for that?

it's not a technical paper, but a paper about the musical effects of the syncing, which at the time was quite amazing. http://music.princeton.edu/~dan/plork/papers/WhyALaptopOrchestra.pdf this was all done with OSC/UDP and a Airport, with 15 laptopists.

...

I think SLUB simply switched to using cables but then they are just 3 guys. It's probably more convenient to use cables when you have three friends than when you have a group of 30 students.

exactly. i've used wires with quartets. i won't with plork.

...

...
this year we've been exploring other ways of doing it that i find quite interesting, but we've also had some more difficulties with the network sync, in part (i think) because we've gotten larger; synchronizing 30 laptops seems to be harder than 15.

This makes perfect sense. I also wouldn't be surprised if Apple occasionally tweaked their implementation and likely they are more concerned with things like throughput for things like streamed video than with syncing up 30 performers. There are quite a few variables.

yep! mark ran some tests and found that at least wiring the server to the airport would noticeably improve performance, as well as turning on Network Interference Robustness, or whatever it's called. at the moment the network seems to be working well, but the difficulties encouraged us to try other strategies, which has been quite interesting. for instance, to sync with Matmos, running a specific BPM, we've had the players simply use their ears (!) to sync. this has worked quite well, and with a distributed audio system like PLOrk, it may well be that this kind of sync is *better* than network sync. dt

...

...
but even clock/timestamp syncing is not quite the same as the syncing we've done in the past, where a single server machine can say to all the other machines, do THIS, NOW. we've been able to do this in the past with 15 machines, and hopefully we'll be able to figure out a way to do it well (wirelessly) with >15. so, while i'm looking forward to having a good timestamp system of some sort, it will be one of several ways we go about "syncing."

I see. That indeed is different. NetClock is more about sharing a clock that sends pulses (or events in ChucK's case). From the user's perspective I think it would work like the clock I've been using when livecoding lately (I'll post it below to illustrate). That need not be a issue; maybe we can work at interoperability in the interest of cross-platform jamming. I'd also be happy to implement NetClock for ChucK myself. If I'd have access to timestamps I think I could do it entirely in ChucK itself.

...

I hope nobody minds that I'm a bit fanatical about this but I've been very interested in synced livecoding jams since I realised the guy who has a radio show before mine uses SC and the one after me uses PD. Synced jam session would be nice for us; I'm purely being selfish here :¬). Then again it works on SC, Perl and Haskel now, I think, with Fluxus, Impromptu and PD in the works, somebody mentioned Processing as well. To me that's exciting; I don't like MIDI that much but I do like the convenience of MIDI clock and how it allows for jam sessions without having to first agree on some protocol. It's also interesting to me how all of these systems deal with time and timing in quite different ways so they will likely all abstract this shared clock in a different way towards the user; to me that's a beautifull thing.

...

Yours, Kas. _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

Kassen

9:01 p.m.

Dan;

...

it's not a technical paper, but a paper about the musical effects of the syncing, which at the time was quite amazing.

Quite interesting indeed.

...

for instance, to sync with Matmos, running a specific BPM, we've had the players simply use their ears (!) to sync. this has worked quite well, and with a distributed audio system like PLOrk, it may well be that this kind of sync is *better* than network sync.

That sounds more or less like mixing two locked groove records using quartz lock on turntables, that's indeed not that hard while it does open up a lot of space for intuitive ways of dealing with things like the contrast between jazz-style brushed hits and techno-style stacato beats. I noticed I neglected to post the code I promissed before (now it's really there). As silly as it may sound; I seriously considered to augment this with a tap-tempo using -say- spacebar and the enter key for intuitive control over global tempo while livecoding. There is certainly a lot more to this topic than what we've seen so far in general applications, I feel. Yours; Kas. public class clk { //beats per minute static float BPM; //beats per bar static int bpb; static dur period; static int beatnr; static int barnr; static Event @ bar; static Event @ beat; fun static void bpm(float value) { if (value > 0) { value => BPM; minute / BPM => period; } else <<<"BPM should be greater than 0">>>; } } new Event @=> clk.beat; new Event @=> clk.bar; 120 => clk.bpm; 4 => clk.bpb; while(1) { if(!clk.beatnr) { clk.barnr++; clk.bar.broadcast(); } clk.beat.broadcast(); clk.period => now; clk.beatnr++; clk.bpb %=> clk.beatnr; }

Kassen

12:28 p.m.

Tom;

...

I assumed from the model that the reason "now" could change was that user code could be pre-empted to make framerate and sample rate deadlines. I also assumed that loading a WAV file is a lot more about waiting on the disk than the CPU. I could be wrong on both counts..

That would be a interesting strategy and indeed quite clever though the price is of course that accuracy might be lost. It's not clear to me from that tutorial how Impromptu deals with concurrency and race conditions and so on. Maybe I need to read it again with more care. I also found it interesting to see Impromptu's strategy of using slightly different code depending on the accuracy needed as compared to ChucK's way of always having extreme accuracy at the expense of cpu load and potential glitches. Another interesting contrast is how Impromptu uses plugins for the sound generation. I don't think that would be as satisfying to me as writing my own ways of creating sounds but it does make a lot of sense in a livecoding environment where the focus might be on the generation of music more than on the technique on a sonological level.

...

I can see why it would be nice, even though I don't need it. ;p A lot of rehearsal time in PLOrk this past semester was dedicated to synchronizing the performers because we lacked good network synchronization. So having NetClock as an option would have been great.

Was that using a wireless network? I'm not sure how the jitter of a wireless network would affect the timeserver protocols NetClock needs to sync the computer clocks. There has been some talk about that on the NetClock list but I'm not sure any conclusions have been reached. I could ask if it's relevant to you. Anyway, I do kinda "need" this, livecoding jams across ChucK, SC and PD would make me very happy :¬). Now that I think of it; beyond NetClock there might be questions along the lines of Hans's recent inquiries on how OSC timestamps would interact with the ChucK Shreduler. I could imagine a implementation where OSC messages with timestamps could be used to schedule commands without the need for a shred per command. Yours, Kas.

Tom Lieber

12:55 p.m.

On Wed, Apr 29, 2009 at 12:28 PM, Kassen wrote:

...

Another interesting contrast is how Impromptu uses plugins for the sound generation. I don't think that would be as satisfying to me as writing my own ways of creating sounds but it does make a lot of sense in a livecoding environment where the focus might be on the generation of music more than on the technique on a sonological level.

Not as satisfying? Well, you could always write the plugins too.

...

Was that using a wireless network? I'm not sure how the jitter of a wireless network would affect the timeserver protocols NetClock needs to sync the computer clocks. There has been some talk about that on the NetClock list but I'm not sure any conclusions have been reached. I could ask if it's relevant to you.

Yeah, a wireless network. And for whatever reason (AirPort updates, OS X updates, variation in laptop hardware), timing issues got worse from the previous year. Actually, I don't know how this was solved in the end, because I think things had improved by the end (or maybe we stopped relying on the network so much and I didn't notice)... On Wed, Apr 29, 2009 at 12:27 PM, Daniel Trueman wrote:

...

On Apr 29, 2009, at 11:36 AM, Tom Lieber wrote:

...
A lot of rehearsal time in PLOrk this past semester was dedicated to synchronizing the performers because we lacked good network synchronization.

i've come to view this as a feature, not a bug... ;--}

I chose my words carefully! There's no way I can view getting everybody to practice being in sync as a bad thing, but having the option of good network synchronization means we can make pieces where players can give their full attention to something else, like making patterns in beepsh. -- Tom Lieber http://AllTom.com/

Mark Cerqueira

1:22 p.m.

I've spent this semester doing some independent work with Perry and Dan on how to get good network synchronization for PLOrk so that we don't need a conductor and can focus on doing other things. I implemented a Time-Tagged OSC protocol similar to the stuff Adrian Freed does. Essentially the protocol first synchronizes everyone's clock via a protocol that is based on the Network Time Protocol (NTP). On the PLOrk network, one machine runs as a NTP Server and all others run as NTP Clients - NTP Clients calculate the offset of their VM time to the server VM time, as well as keeping track of the round-trip time of packets sent to synchronize to account for that in the offset. Once the clocks are synchronized, packets sent over the network include the current time (which should be synchronized pretty closely if the above protocol works) and a time-to-execute offset. When someone receives the packet they add the current time plus the time-to-execute offset and schedule the packet to be opened only at that time. The NTP-like synchronization protocol I described above also keeps track of all RTTs and broadcasts to everyone what a good time-to-execute field is (i.e. make sure we give enough time for the packet to travel over the network). If all works well, everyone should open the packet at the same time. Not sure how successful this implementation will be...I'm testing it tomorrow during PLOrk rehearsal! If anyone's interested in seeing the code/my write-up I can pass around a copy once I *crosses fingers* complete it and hand it in on Monday. Best, Mark Tom Lieber wrote:

...

On Wed, Apr 29, 2009 at 12:28 PM, Kassen wrote:

Yeah, a wireless network. And for whatever reason (AirPort updates, OS X updates, variation in laptop hardware), timing issues got worse from the previous year. Actually, I don't know how this was solved in the end, because I think things had improved by the end (or maybe we stopped relying on the network so much and I didn't notice)...

On Wed, Apr 29, 2009 at 12:27 PM, Daniel Trueman wrote:

...
On Apr 29, 2009, at 11:36 AM, Tom Lieber wrote:

...
A lot of rehearsal time in PLOrk this past semester was dedicated to synchronizing the performers because we lacked good network synchronization.

i've come to view this as a feature, not a bug... ;--}

I chose my words carefully! There's no way I can view getting everybody to practice being in sync as a bad thing, but having the option of good network synchronization means we can make pieces where players can give their full attention to something else, like making patterns in beepsh.

Kassen

1:30 p.m.

Mark, That seems quite similar in approach to this; http://netclock.slab.org/wiki/index.php/FAQ I'd like to invite you to have a look at that and perhaps join the mailinglist on it. If these two could be made inter-operable it could be used to sync ChucK to a range of other open source music and graphics systems. Yours, Kas. 2009/4/29 Mark Cerqueira :

...

I've spent this semester doing some independent work with Perry and Dan on how to get good network synchronization for PLOrk so that we don't need a conductor and can focus on doing other things. I implemented a Time-Tagged OSC protocol similar to the stuff Adrian Freed does. Essentially the protocol first synchronizes everyone's clock via a protocol that is based on the Network Time Protocol (NTP). On the PLOrk network, one machine runs as a NTP Server and all others run as NTP Clients - NTP Clients calculate the offset of their VM time to the server VM time, as well as keeping track of the round-trip time of packets sent to synchronize to account for that in the offset.

Once the clocks are synchronized, packets sent over the network include the current time (which should be synchronized pretty closely if the above protocol works) and a time-to-execute offset. When someone receives the packet they add the current time plus the time-to-execute offset and schedule the packet to be opened only at that time. The NTP-like synchronization protocol I described above also keeps track of all RTTs and broadcasts to everyone what a good time-to-execute field is (i.e. make sure we give enough time for the packet to travel over the network). If all works well, everyone should open the packet at the same time.

Not sure how successful this implementation will be...I'm testing it tomorrow during PLOrk rehearsal! If anyone's interested in seeing the code/my write-up I can pass around a copy once I *crosses fingers* complete it and hand it in on Monday.

Best, Mark

Tom Lieber wrote:

...
On Wed, Apr 29, 2009 at 12:28 PM, Kassen wrote:

Yeah, a wireless network. And for whatever reason (AirPort updates, OS X updates, variation in laptop hardware), timing issues got worse from the previous year. Actually, I don't know how this was solved in the end, because I think things had improved by the end (or maybe we stopped relying on the network so much and I didn't notice)...

On Wed, Apr 29, 2009 at 12:27 PM, Daniel Trueman wrote:

...
On Apr 29, 2009, at 11:36 AM, Tom Lieber wrote:

...
A lot of rehearsal time in PLOrk this past semester was dedicated to synchronizing the performers because we lacked good network synchronization.

i've come to view this as a feature, not a bug... ;--}

I chose my words carefully! There's no way I can view getting everybody to practice being in sync as a bad thing, but having the option of good network synchronization means we can make pieces where players can give their full attention to something else, like making patterns in beepsh.

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

Mark Cerqueira

6 May 6 May

7:46 p.m.

Hello, Last week I sent out an email to the list about a synchronization protocol I was working on with Perry and Dan for PLOrk. I thought I'd share the current product and invite you all to check it out, try it out, and give me feedback if you'd like. All relevant materials can be found at: http://www.princeton.edu/~mcerquei/networking/ The protocol that was rather successful in testing is the Time-Tagged OSC (TOSC) and the source code for all that code is in the appropriately named folder. Apologies for the lack of comments in the source code for the new classes I wrote. The sample client and server programs I wrote have some comments which pretty much covers the main differences from OSC to TOSC. In the final report pdf file, the TOSC section will outline what's going on in general so that's worth checking out. You can also check out a demo that compares a controlled recording/TOSC test/OSC test in the Demo.tar file (it plays in Audacity). I'll be continuing to work on this project next year for my senior thesis. There are a number of things I already want to fix/work on, but recommendations/suggestions are, again, very welcomed! 1. Code everything into the ChucK source code. Everything is written in ChucK right now so it can be made more efficient by coding directly into the source. 2. Improve the NTP protocols. My algorithms to synchronize clocks/ calculate good time-to-open offsets are very, very, very crude and can certainly be improved on. Better NTP = better synchronization. 3. Netclock - Making TOSC interoperable with Netclock would be useful, as Kassen mentioned in an earlier email. Thanks! Best, Mark On Apr 29, 2009, at 1:30 PM, Kassen wrote:

...

Mark,

That seems quite similar in approach to this; http://netclock.slab.org/wiki/index.php/FAQ

I'd like to invite you to have a look at that and perhaps join the mailinglist on it. If these two could be made inter-operable it could be used to sync ChucK to a range of other open source music and graphics systems.

Yours, Kas.

2009/4/29 Mark Cerqueira :

...
I've spent this semester doing some independent work with Perry and Dan on how to get good network synchronization for PLOrk so that we don't need a conductor and can focus on doing other things. I implemented a Time- Tagged OSC protocol similar to the stuff Adrian Freed does. Essentially the protocol first synchronizes everyone's clock via a protocol that is based on the Network Time Protocol (NTP). On the PLOrk network, one machine runs as a NTP Server and all others run as NTP Clients - NTP Clients calculate the offset of their VM time to the server VM time, as well as keeping track of the round-trip time of packets sent to synchronize to account for that in the offset.

Once the clocks are synchronized, packets sent over the network include the current time (which should be synchronized pretty closely if the above protocol works) and a time-to-execute offset. When someone receives the packet they add the current time plus the time-to-execute offset and schedule the packet to be opened only at that time. The NTP-like synchronization protocol I described above also keeps track of all RTTs and broadcasts to everyone what a good time-to-execute field is (i.e. make sure we give enough time for the packet to travel over the network). If all works well, everyone should open the packet at the same time.

Not sure how successful this implementation will be...I'm testing it tomorrow during PLOrk rehearsal! If anyone's interested in seeing the code/my write-up I can pass around a copy once I *crosses fingers* complete it and hand it in on Monday.

Best, Mark

Tom Lieber wrote:

...
On Wed, Apr 29, 2009 at 12:28 PM, Kassen
...
wrote:

Yeah, a wireless network. And for whatever reason (AirPort updates, OS X updates, variation in laptop hardware), timing issues got worse from the previous year. Actually, I don't know how this was solved in the end, because I think things had improved by the end (or maybe we stopped relying on the network so much and I didn't notice)...

On Wed, Apr 29, 2009 at 12:27 PM, Daniel Trueman
...
wrote:

...
On Apr 29, 2009, at 11:36 AM, Tom Lieber wrote:

...
A lot of rehearsal time in PLOrk this past semester was dedicated to synchronizing the performers because we lacked good network synchronization.

i've come to view this as a feature, not a bug... ;--}

I chose my words carefully! There's no way I can view getting everybody to practice being in sync as a bad thing, but having the option of good network synchronization means we can make pieces where players can give their full attention to something else, like making patterns in beepsh.

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

mike clemow

8:41 p.m.

Mark, This is fascinating work. I will definitely try this out as soon as I graduate (one week left). I've spent the last year working with ChucK on a cluster and haven't even tried to attempt synchronization because the nature of the work I was doing required sample-accurate sync. We've given up doing what we were doing (because it was insane) and now I'm excited to test out your implementation in my environment here. I'd be happy to provide you feedback, especially considering you're doing thesis work on it (user testing is key ;-). Good luck! _Mike 2009/5/6 Mark Cerqueira :

...

Hello, Last week I sent out an email to the list about a synchronization protocol I was working on with Perry and Dan for PLOrk. I thought I'd share the current product and invite you all to check it out, try it out, and give me feedback if you'd like. All relevant materials can be found at: http://www.princeton.edu/~mcerquei/networking/ The protocol that was rather successful in testing is the Time-Tagged OSC (TOSC) and the source code for all that code is in the appropriately named folder. Apologies for the lack of comments in the source code for the new classes I wrote. The sample client and server programs I wrote have some comments which pretty much covers the main differences from OSC to TOSC. In the final report pdf file, the TOSC section will outline what's going on in general so that's worth checking out. You can also check out a demo that compares a controlled recording/TOSC test/OSC test in the Demo.tar file (it plays in Audacity). I'll be continuing to work on this project next year for my senior thesis. There are a number of things I already want to fix/work on, but recommendations/suggestions are, again, very welcomed! 1. Code everything into the ChucK source code. Everything is written in ChucK right now so it can be made more efficient by coding directly into the source. 2. Improve the NTP protocols. My algorithms to synchronize clocks/calculate good time-to-open offsets are very, very, very crude and can certainly be improved on. Better NTP = better synchronization. 3. Netclock - Making TOSC interoperable with Netclock would be useful, as Kassen mentioned in an earlier email. Thanks! Best, Mark On Apr 29, 2009, at 1:30 PM, Kassen wrote:

Mark,

That seems quite similar in approach to this; http://netclock.slab.org/wiki/index.php/FAQ

I'd like to invite you to have a look at that and perhaps join the mailinglist on it. If these two could be made inter-operable it could be used to sync ChucK to a range of other open source music and graphics systems.

Yours, Kas.

2009/4/29 Mark Cerqueira :

I've spent this semester doing some independent work with Perry and Dan on

how to get good network synchronization for PLOrk so that we don't need a

conductor and can focus on doing other things. I implemented a Time-Tagged

OSC protocol similar to the stuff Adrian Freed does. Essentially the

protocol first synchronizes everyone's clock via a protocol that is based on

the Network Time Protocol (NTP). On the PLOrk network, one machine runs as a

NTP Server and all others run as NTP Clients - NTP Clients calculate the

offset of their VM time to the server VM time, as well as keeping track of

the round-trip time of packets sent to synchronize to account for that in

the offset.

Once the clocks are synchronized, packets sent over the network include the

current time (which should be synchronized pretty closely if the above

protocol works) and a time-to-execute offset. When someone receives the

packet they add the current time plus the time-to-execute offset and

schedule the packet to be opened only at that time. The NTP-like

synchronization protocol I described above also keeps track of all RTTs and

broadcasts to everyone what a good time-to-execute field is (i.e. make sure

we give enough time for the packet to travel over the network). If all works

well, everyone should open the packet at the same time.

Not sure how successful this implementation will be...I'm testing it

tomorrow during PLOrk rehearsal! If anyone's interested in seeing the

code/my write-up I can pass around a copy once I *crosses fingers* complete

it and hand it in on Monday.

Best,

Mark

Tom Lieber wrote:

On Wed, Apr 29, 2009 at 12:28 PM, Kassen

wrote:

Yeah, a wireless network. And for whatever reason (AirPort updates, OS

X updates, variation in laptop hardware), timing issues got worse from

the previous year. Actually, I don't know how this was solved in the

end, because I think things had improved by the end (or maybe we

stopped relying on the network so much and I didn't notice)...

On Wed, Apr 29, 2009 at 12:27 PM, Daniel Trueman

wrote:

On Apr 29, 2009, at 11:36 AM, Tom Lieber wrote:

A lot

of rehearsal time in PLOrk this past semester was dedicated to

synchronizing the performers because we lacked good network

synchronization.

i've come to view this as a feature, not a bug... ;--}

I chose my words carefully! There's no way I can view getting

everybody to practice being in sync as a bad thing, but having the

option of good network synchronization means we can make pieces where

players can give their full attention to something else, like making

patterns in beepsh.

_______________________________________________

chuck-users mailing list

chuck-users@lists.cs.princeton.edu

https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

-- http://michaelclemow.com http://semiotech.org

Kassen

9:22 p.m.

Mark, I read your PDF and listened to the recordings. I also glanced over most of the code (I'm sure I'm still missing things there). This looks very promising to me; it's clear that the TOSC version is a big improvement and it's good that the exact nature of the issue with the wireless network is now clear. It strikes me that you are doing this research into syncing in a especially challenging environment so that should mean the solutions you find should work in a majority of cases. I thought it quite inventive to emulate the things we are missing -at the moment- in ChucK (time stamps for OSC bundles as use of networked clocks through reading the system's clock) in ChucK itself; it's a nice reminder of what we already have and how powerful it is. If I can/should give some critique; I was momentarily confused as from your PDF it looked like your "TOSC" version was based on "OSC with timestamps added", which would be a slightly odd thing as OSC already has timestamps. What we have here (unless I'm gravely mistaken) is instead ChucK's OSC implementation plus time-stamps readable to your use of it here. This ambiguity isn't very relevant for the topic of your research here but it is quite relevant from a perspective of compatibility and pointing this out might save confusion (I'm not sure what the intended audience of that text is though, maybe this isn't relevant). Overall these are some very promising results; it's looking like this will be a great year for synced live performance. Thanks for the update, Kas.

Mark Cerqueira

8 May 8 May

2:10 a.m.

Kassen, Thanks for the encouraging feedback and sorry for the delayed response - far too much work to do these days! I see what you're saying regarding the confusing wordage in that particular section of my report. Any future written work will certainly take care of the OSC time-stamp vs. TOSC time-tags for synchronization difference explicitly. Thanks again Kas! Best, Mark On May 6, 2009, at 9:22 PM, Kassen wrote:

...

Mark,

I read your PDF and listened to the recordings. I also glanced over most of the code (I'm sure I'm still missing things there). This looks very promising to me; it's clear that the TOSC version is a big improvement and it's good that the exact nature of the issue with the wireless network is now clear. It strikes me that you are doing this research into syncing in a especially challenging environment so that should mean the solutions you find should work in a majority of cases.

I thought it quite inventive to emulate the things we are missing - at the moment- in ChucK (time stamps for OSC bundles as use of networked clocks through reading the system's clock) in ChucK itself; it's a nice reminder of what we already have and how powerful it is.

If I can/should give some critique; I was momentarily confused as from your PDF it looked like your "TOSC" version was based on "OSC with timestamps added", which would be a slightly odd thing as OSC already has timestamps. What we have here (unless I'm gravely mistaken) is instead ChucK's OSC implementation plus time-stamps readable to your use of it here. This ambiguity isn't very relevant for the topic of your research here but it is quite relevant from a perspective of compatibility and pointing this out might save confusion (I'm not sure what the intended audience of that text is though, maybe this isn't relevant).

Overall these are some very promising results; it's looking like this will be a great year for synced live performance.

Thanks for the update, Kas. _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

Hans Aberg

29 Apr 29 Apr

2:35 p.m.

On 29 Apr 2009, at 19:22, Mark Cerqueira wrote:

...

I've spent this semester doing some independent work with Perry and Dan on how to get good network synchronization for PLOrk so that we don't need a conductor and can focus on doing other things. I implemented a Time-Tagged OSC protocol similar to the stuff Adrian Freed does. Essentially the protocol first synchronizes everyone's clock via a protocol that is based on the Network Time Protocol (NTP). On the PLOrk network, one machine runs as a NTP Server and all others run as NTP Clients - NTP Clients calculate the offset of their VM time to the server VM time, as well as keeping track of the round-trip time of packets sent to synchronize to account for that in the offset.

Once the clocks are synchronized, packets sent over the network include the current time (which should be synchronized pretty closely if the above protocol works) and a time-to-execute offset. When someone receives the packet they add the current time plus the time-to-execute offset and schedule the packet to be opened only at that time. The NTP-like synchronization protocol I described above also keeps track of all RTTs and broadcasts to everyone what a good time-to-execute field is (i.e. make sure we give enough time for the packet to travel over the network). If all works well, everyone should open the packet at the same time.

I think somebody made a system for Internet performances (on any kind of instrument) where one does not bother synchronizing the sources, but the outputs are sent to a common computer which mixes them and sends it back to the performers. These wear headsets and need to learn to perform in a situation where the sound is delayed. (Perhaps similar to some pipe organs, with pipes far from the console.) A good synchronization will help, though, and will be a better solution when achievable. Hans

Kassen

1:23 p.m.

Tom;

...

Not as satisfying? Well, you could always write the plugins too.

Of course :¬) I've been thinking about this a fair amount, lately; the amount of assumptions made in livecoding systems and how they speed up the performance while also limiting options. It's a bit of a trade.

...

Yeah, a wireless network. And for whatever reason (AirPort updates, OS X updates, variation in laptop hardware), timing issues got worse from the previous year. Actually, I don't know how this was solved in the end, because I think things had improved by the end (or maybe we stopped relying on the network so much and I didn't notice)...

You only mention aspects of OSX but I heard the exact same thing about trying to sync over a wireless network with Linux as a host. the OS may be a factor but from what I gather the phase of the moon and the proximity of any squirrels that might be around are also quite relevant in wireless network behaviour. I'm not at all sure how that would affect time server protocols; I can't imagine it helps. I think I might've simply used a switch and some CAT5 cables but then I would've lost out on the more organic strategies... Kas.

Robert Poor

11:39 a.m.

On 29 Apr 2009, at 06:32, Kassen wrote:

...

Tom; <>>

I really appreciate ChucK's deterministic timing approach, but I predict that Impromptu's non-deterministic approach is ultimately a significant advantage. Howzzat? ChucK promises deterministic timing on a single processor. The only way ChucK can take advantage of multiple processors is by synching them together, and techniques for synchronizing multiple ChucKs are generally non-deterministic. And it takes extra work to do this. If I understand correctly, Impromptu uses any available processing power, regardless of where it comes from. So I believe that in a world where processing power gets cheaper and faster every year, Impromptu will have an advantage. - Rob [P.S.: It's like the early arguments that Ethernet was useless for real-time processing because it was inherently non-deterministic. As Ethernet got faster (3mbps, 10mbps, 100mbps, 1gbps) that argument largely fell by the wayside.]

Kassen

6:31 p.m.

Rob;

...

Howzzat?

ChucK promises deterministic timing on a single processor. The only way ChucK can take advantage of multiple processors is by synching them together, and techniques for synchronizing multiple ChucKs are generally non-deterministic. And it takes extra work to do this.

Well, you can have multi-core deterministic performance, it just potentially means some cores will be waiting some of the time. This can be ok or terrible, performance-wise, depending on how clever people get and the application. For some applications paralel processing hardly has any benefits at all.

...

If I understand correctly, Impromptu uses any available processing power, regardless of where it comes from.

Likely, yes, and judging from Andrew's videos it's a lovely system with very respectable graphical capabilities as well.

...

So I believe that in a world where processing power gets cheaper and faster every year, Impromptu will have an advantage.

In some regards. I also read and appreciated your note on Ethernet. SC, for example uses the network protocol to communicated between the "server" and the "language", SC is also generally recongnised to be very fast indeed. However, execution order in SC can be a tricky affair that SC specialists sometimes have to spend quite some time and some care on to get it to behave. Neither of those languages will do DSP right in the language itself like ChucK will. This is a trade, you get some nice things for some less nice ones, it might be a good trade or a bad one depending on your needs. I don't think you can compare them purely on CPU usage, like you can't compare a Unimog (A Mercedes-build mountain-truck with 25 gears, 5 of which are reverse) to a Ferrari purely on speed; though both are cars they excell at very different things. I do think it would make sense to start thinking about how ChucK could make use of multi-core or multi-cpu setups (and fortunately this has been hinted at) but these concerns are hardly the be-all end-all of audio programming; quite often precision to the single sample will matter a lot, maybe not often in sequencing but there is more to sound than just sequencing. There are certainly challenges there but I'm still quite happy with the choices ChucK made here. Yours, Kas.

5539

Age (days ago)

5548

Last active (days ago)

List overview

Download

20 comments

7 participants

participants (7)

Daniel Trueman
Hans Aberg
Kassen
Mark Cerqueira
mike clemow
Robert Poor
Tom Lieber