On 21/04/2008, mike clemow wrote:

Chiming in a bit late here...

Count me as +1 in the "swell." ;-) I *heart* MAUI as well. But I also wanted to ask the crowd here what they think about the idea of utilizing/accepting influence from some of the very-accessible graphics platforms out there. There's SwingOSC, Processing, OpenFrameworks, the ixi-software.net people have Mirra... all of which are cross-platform.

I think they are great. I feel there are a lot of very interesting options there, we'd be silly to ignore those, there's a lot there to learn/borrow from idea-wise as well. To me ChucK is mainly a syntax though and I think there are very good reasons to try to incorporate visual elements using this same syntax. This has advantages for ChucK as a educational platform (only one syntax to learn) and it has advantages for artists (less mental switching). This does/would also lead to whole projects being more self-contained which is good for communication. This is not to say I disagree with you in the slightest as the Maui are clearly no alternative for Processing or the like at all. Maybe GlucK would eventually be but GlucK isn't even released yet. I think that an OSC-based communication for GUI elements would be

great. OSC, as pervasive as it is, is in my opinion an under-utilized protocol. MAUI is awesome and convenient, because it's still in the miniAudicle environment, I still usually just use it for debugging. I think that I would rather see a set of GUI-specific OSC objects that can be manipulated with whatever visual front-end you want to use. For anything more complex than the normal slider, knob, led, metaphor, I find myself rolling my own anyway.

Yes, this makes perfect sense. As Inventor ran into on the forum; there are definite questions about the MAUI at the moment because a array (non- CS-sense) of buttons will result in a array of shreds listening for them which in turn results in a array of overhead. Besides that there are no MAOI on two out of three platforms right now. Regardless of what we do I think it would be a good idea to have a round-table of ideas on how to deal with interface elements. I could imagine -for example- a way of abstracting them that could be tied to either MAUI or OSC depending on mood/platform/needs to mention but one idea. If the GUI API could be decoupled from the environment, OSC-based, and

graphics-platform agnostic (and miniAudicle could easily have it's own in-environment 2D implementation), then I think we'd really be onto something.

You mean like a separate program? Right now command-line ChucK doesn't depend on Xserver/Quartz/explorer (that I know of) and I could see reasons for keeping it that way like installations or embedded applications. I wonder if you can have a single application that wouldn't need a graphical shell to run but could still use one when one is available and useful. Yours, Kas.

On Mon, Apr 21, 2008 at 7:18 PM, mike clemow wrote:

right OSC messages. I guess what I'm suggesting is a similar Paradigm for setting up responders in Chuck code for UI elements if they are around. Perhaps...

SinOsc s => dac; OSCResponder resp; resp.type( "f" ); // f is for "float" - OSC has float, int, and string resp.port( 9001 ) resp.address( "/test" ); // the address in OSC namespace resp => s.gain; // plug it in... (if this sort of thing is possible, since right now this attribute wouldn't accept this object)

In my Chuckian fantasy, ;-) if the shred received a /test OSC message on port 9001 with a matching type signature, it would "chuck" the value into s.gain.

This would be perfect for me, I desperately want a simpler (i.e., less code) way to create OSC-enabled parameters. I think it would even be possible to do this easily with Chuck code if it supported references for primitive types. (I'll draw attention yet again to: http://wiki.cs.princeton.edu/index.php/References_for_primitives.) Although my suggestion didn't use the chuck operator => which is a pretty cool idea. I think this would be a fantastic thing to get implemented, I'll have to have a closer look at the Chuck code when I get a chance. Steve

On 22/04/2008, mike clemow wrote:

Hi, Kas,

Hey, Mike (and Stephen and Joe),

This I agree with completely. It would be sweet, for instance, to "plug" a GUI element into a object parameter.

mge => s.gain; //this might work like "SinOsc s => dac;"

I've been thinking about things like that as well. This would mainly be shorthand that would save defining a actual shred but I could imagine that in the (far) future such things (not just interface elements but also Ugens send to parameters) would be dealt with by the compiler who would try to see if that parameter were one where we could do something more clever then using a shred (equivalent). In the case of gain I could imagine assigning the fader's internal value for it's state to the oscillator's internal value for gain. This won't work everywhere; some parameters need to do calculations upon changing but in some places such things might save cpu, I wonder if that would be worth it.

That's off the top of my head, but there are probably a million "chuckian" things one might change about the normal UI paradigm. To be fair, MAUI elements are not created in code in a very novel way. It's still pretty standard UI code. For a live-coding language, this isn't really ideal.

I agree. I'm extremely curious about the syntax GlucK will have because most 3d libraries seem to reason in a way that's not mine. I haven't yet worked with the MAUI elements myself as I don't have a Mac but based on what Inventor found and my own tests there sporking a thousand empty shreds waiting for a event and seeing how much that took of the CPU (quite a lot) I think we could use some streamlining. While that may sound very critical of Spencer I think a syntax is a language and like all languages will have to be formed, changed and developed by the people that speak it.

I think I actually do mean like a separate program. When I run a UI I made in Processing to control Chuck, Chuck doesn't even have to be running on the same machine, since all the control messages are sent via UPD as OSC messages. This is the same kind of paradigm that the SuperCollider language (client) communicates with their synth server.

Yes, this makes a lot of sense to me. I could see a lot of useful applications for example running such a interface client on a mobile phone or on a tablet PC with a modest CPU. I had a idea last night. How about abstracting the interface to a class called something like "Gui", Gui would have member functions for doing things like creating faders and buttons and so on but also (like Hid) a "open" function. This "open" function would be used to either link the gui to a window like the Mini has or to OSC. If OSC is chosen it would be given a port for input and for output and send all commands (standardised) over OSC to be dealt with by Processing or whatever graphical program the user wants to use. This way the choice of what interface to use could even be given as a parameter to the shred. OSC-wise it would just be a abstraction with a convenient naming scheme. Templates for catching these messages could be build in Processing and IXI and so on to get people started. This would also enable us to use a entire graphical interface window as a single event, ChucK that to now and parse that using the same syntax we use for Hid "if (msg.isButtonDown())......." which would save a LOT of shreds for large and complex interfaces. The big weakspot in this plan is that a user may want to define a element (say a sprite) in one of these packages that has no ChucKian gui equivalent. Ideally there would be a way of dealing with that, for example simply passing them on if the gui is linked to OSC and generating a non-fatal error otherwise (to preserve compatibility where possible and fail gracefully). I could also imagine this "open" function getting presets to make it speak the same language as a Lemur or a Monome. That would chiefly be a convenience thing but I could see that getting quite popular.

I mean, I guess we're not that far from this... meaning that we could kind of roll our own here with a clever class that abstracts the OSC functionality of Chuck, but to honest, it's going to be tricky. There are some quirks to its current implementation, and some limitations to the OSC hierarchy due to the way that shreds work. If it were built in functionality, I think that this would certainly pave the way to having a framework upon which one could choose-your-own-UI-platform. After that, I could see a Chuck-syntax 2D/3D drawing kit that would use Chuck to do UI.

Yes, OSC could use some love, I'm not sure I understand what you mean by the link between shreds and OSC hierarchy though, could you elaborate on that, please? I'm just thinking out loud here... :¬) Yours, Kas.

Kas,

Yes, OSC could use some love, I'm not sure I understand what you mean by the link between shreds and OSC hierarchy though, could you elaborate on that, please?

Yes, of course. So, a while back (and I had promised to send video and school took over my life instead) I worked on a project building a trash-sculpture with two bands, who graciously donated audio samples to the project. The trash sculpture had things you could turn, twist, open, close, etc and each of those would trip various sensors connected to an Arduino microcontroller board, which sent ASCII text to Python script, which bundled the information as OSC messages and sent them along to Chuck which would happily scrub away at the poor bands' audio samples and turn them into an audience-driven remix. (That's the backstory.) Between Chuck and the Python script, I tried to create an OSC address hierarchy not unlike the following: /control /birdnoise /gain /filterq /aliensquawk /freq /gain ... /router (this was for controlling the Python script) /shutdown /reset ... Or something like that. While other OSC implementations allowed for the parsing of the addresses and assignment of the address hierarchy to different parts of code. Now, I'm not positive that my knowledge of the OSC specification is on target, but here's what I found frustrating to do in Chuck. I have a message: "/control/birdnoise/freq" for instance. So I took a look at the example OSC receiver code, which is here: http://chuck.cs.princeton.edu/doc/examples/osc/OSC_recv.ck Here's the part of the code that i'm talking about: // create an address in the receiver, store in new variable recv.event( "/sndbuf/buf/rate, f" ) @=> OscEvent oe; First, for some reason, chuck didn't care if i specified it like this either: // create an address in the receiver, store in new variable recv.event( "/sndbuf/buf/rate", f ) @=> OscEvent oe; //notice the difference in where i break the string. what's the deal with this method? And then, also you can listen to two events at once, so if you wanted to also send "/sndbuf/buf/gain" you'd have to set up a new shred... but that also necessarily means a new receiver, a new port, and a new event. The sender will have to know that to send "/sndbuf/buf/gain," they have to send to a different destination. So, what's the point in the hierarchy, right? Everything is flat and everything is it's own net address. Unless I'm wrong about OSC, this seems like a serious limitation. I feel like I should be able to structure my code such that it can route "/sndbuf/buf/rate" messages to the part of code dealing with rate and "/sndbuf/buf/gain" messages to the part of the code dealing with gain *within a single shred.* After all, that's where my SndBuf buf => dac; patch is. The other thing is that, OSC events listening for two different messages on the same port, will respond erroneously if the number and types of data values match, meaning that the OSC "signature" in Chuck has nothing to do with the address. This also seems "wrong" to me. I'm not sure if anyone else has encountered that. Anyway, I'm done ranting for now. I have to admit that I'm a bit obsessed with OSC in general. cheers, mike On Tue, Apr 22, 2008 at 6:26 AM, Kassen wrote:

On 22/04/2008, mike clemow wrote:

...

Hi, Kas,

Hey, Mike (and Stephen and Joe),

...

This I agree with completely. It would be sweet, for instance, to "plug" a GUI element into a object parameter.

mge => s.gain; //this might work like "SinOsc s => dac;"

I've been thinking about things like that as well. This would mainly be shorthand that would save defining a actual shred but I could imagine that in the (far) future such things (not just interface elements but also Ugens send to parameters) would be dealt with by the compiler who would try to see if that parameter were one where we could do something more clever then using a shred (equivalent). In the case of gain I could imagine assigning the fader's internal value for it's state to the oscillator's internal value for gain. This won't work everywhere; some parameters need to do calculations upon changing but in some places such things might save cpu, I wonder if that would be worth it.

...

That's off the top of my head, but there are probably a million "chuckian" things one might change about the normal UI paradigm. To be fair, MAUI elements are not created in code in a very novel way. It's still pretty standard UI code. For a live-coding language, this isn't really ideal.

I agree. I'm extremely curious about the syntax GlucK will have because most 3d libraries seem to reason in a way that's not mine. I haven't yet worked with the MAUI elements myself as I don't have a Mac but based on what Inventor found and my own tests there sporking a thousand empty shreds waiting for a event and seeing how much that took of the CPU (quite a lot) I think we could use some streamlining.

While that may sound very critical of Spencer I think a syntax is a language and like all languages will have to be formed, changed and developed by the people that speak it.

...

I think I actually do mean like a separate program. When I run a UI I made in Processing to control Chuck, Chuck doesn't even have to be running on the same machine, since all the control messages are sent via UPD as OSC messages. This is the same kind of paradigm that the SuperCollider language (client) communicates with their synth server.

Yes, this makes a lot of sense to me. I could see a lot of useful applications for example running such a interface client on a mobile phone or on a tablet PC with a modest CPU.

I had a idea last night. How about abstracting the interface to a class called something like "Gui", Gui would have member functions for doing things like creating faders and buttons and so on but also (like Hid) a "open" function. This "open" function would be used to either link the gui to a window like the Mini has or to OSC. If OSC is chosen it would be given a port for input and for output and send all commands (standardised) over OSC to be dealt with by Processing or whatever graphical program the user wants to use. This way the choice of what interface to use could even be given as a parameter to the shred. OSC-wise it would just be a abstraction with a convenient naming scheme. Templates for catching these messages could be build in Processing and IXI and so on to get people started.

This would also enable us to use a entire graphical interface window as a single event, ChucK that to now and parse that using the same syntax we use for Hid "if (msg.isButtonDown())......." which would save a LOT of shreds for large and complex interfaces.

The big weakspot in this plan is that a user may want to define a element (say a sprite) in one of these packages that has no ChucKian gui equivalent. Ideally there would be a way of dealing with that, for example simply passing them on if the gui is linked to OSC and generating a non-fatal error otherwise (to preserve compatibility where possible and fail gracefully).

I could also imagine this "open" function getting presets to make it speak the same language as a Lemur or a Monome. That would chiefly be a convenience thing but I could see that getting quite popular.

...

I mean, I guess we're not that far from this... meaning that we could kind of roll our own here with a clever class that abstracts the OSC functionality of Chuck, but to honest, it's going to be tricky. There are some quirks to its current implementation, and some limitations to the OSC hierarchy due to the way that shreds work. If it were built in functionality, I think that this would certainly pave the way to having a framework upon which one could choose-your-own-UI-platform. After that, I could see a Chuck-syntax 2D/3D drawing kit that would use Chuck to do UI.

Yes, OSC could use some love, I'm not sure I understand what you mean by the link between shreds and OSC hierarchy though, could you elaborate on that, please?

...

I'm just thinking out loud here...

:¬)

Yours, Kas.

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

-- urls is coming...

I haven't messed around that much with OSC either, my experience is limited to the monome. The monome doesn't need more than one event listener (definitely not 64), because the only osc message that the monome ever sends looks like this: /prefix/press x y state (e.g. /256/press 13 7 1 - the prefix is configurable) which means that only have to listen for "press" and then you do whatever you want. Comparing to another similar protocol (I've worked with something called "Tib/Rendezvous" which has similarities with OSC - both are UDP based), I think that this isn't a bad way to work with incoming UDP messages. If there are two different OSC commands that do different things, it makes sense that you have two seperate entry points in the ChucK code that waits for them. Especially if they can happen simultaneously (gain and buf are sent at the same time). Even if the shred concept isn't exactly mappable to the thread concept, I think it makes sense to keep handling of each asynchronous event in its own shred (like you'd do in a thread based solution). What could make sense would be to listen to something like /sndbuf/buf/* or /sndbuf/buf, but then you have the problem of finding out what the full name of the incoming event is (../gain or ../rate). I have occasionally wished for a possibility to wait on multiple events as well (mixing timing events with message events), but I usually find that I'd have to check which event was triggered anyway, which will lead to an if-statement that means that it's not that much harder to do a solution using two shreds. /Stefan On Tue, Apr 22, 2008 at 9:57 PM, Kassen wrote:

On 22/04/2008, mike clemow wrote:

...

Yes, of course. So, a while back (and I had promised to send video and school took over my life instead) I worked on a project building a trash-sculpture with two bands, who graciously donated audio samples to the project.

Did your life take over from making the video or just from sending it? If just the sending then please know I would be very interested in seeing this, this sounds great!

...

And then, also you can listen to two events at once, so if you wanted to also send "/sndbuf/buf/gain" you'd have to set up a new shred... but that also necessarily means a new receiver, a new port, and a new event. The sender will have to know that to send "/sndbuf/buf/gain," they have to send to a different destination. So, what's the point in the hierarchy, right? Everything is flat and everything is it's own net address.

Yes, I see, and for large amounts of events this would get quite expensive before anything even gets done at all, not to mention spectacularly ugly and hard to edit, I imagine.

I should probably confess that I haven't done that much with OSC yet. I'm greatly in favour of OSC as I find MIDI quite unpleasant but I simply haven't had a situation yet where I had a OSC sender and receiver that would solve some problem I had... but this doesn't look that good to me.

Anyway, yes, I agree this is less then clean. Parsing that hierarchy would depend on string parsing though and we can build strings but ATM we can't pull them apart. I'd also say that this way of dealing with OSC doesn't alleviate some of the practical issues with the native GUI elements at all.

Illustration1; Example of shreds waiting for events taking up CPU. It's not *that* bad but they aren't free either. (see "Taskmanager" or "top" or similar OS tool) -------------------------------------------- Event some_event;

for (0 => int i; i < 1000; i++) { spork ~my_shred(); me.yield(); } minute => now; //clean it all up some_event.broadcast();

fun void my_shred() { //no functionality at all here some_event => now; } ----------------------------------------------

...

Anyway, I'm done ranting for now. I have to admit that I'm a bit obsessed with OSC in general.

If that's "ranting" I'd love to hear about your more serious concerns! :¬)

On some level I think your concerns with OSC here are quite similar to some of the concerns about the MAUI. There ought to be a more powerful and compact way of structuring these things. There have been proposals for shreds waiting for multiple events and I agree that could be a good idea but for even just 64 events (I'm thinking about the Monome here) that would already be a lot of boring monotonous code. I feel we need something compact that would have a single event and more elaborate parsing of it's members. You make it sound like MIDI parsing is more elegant then OSC and considering the MIDI spec I think that means there is a issue... I like MIDI as much as anyone, it's cheap and generally available, but readability and simplicity don't seem to have been high on the list there.

Thanks for "ranting"! Kas.

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

-- Release me, insect, or I will destroy the Cosmos!