Hi Kas and Spencer, As I wanted to comment both of your replies at once, I'm switching quoting style in mid-thread, IF this new one is problematic, let me know in a reply and I'l'll change back. AT least it makes it clear with screen readers who is saying what without having to parse "greater than x 3" in my head, <smile>. Feel free to snip heavily. V, S, K = Veli-Pekka, Spencer, Kas [UI for synth panels] [mini-audicle] VV: I've yet to figure out how to run scripts in it, <embarrassed emoticon>. S: Its easy! (hopefully...) You just have to type or paste some ChucK code into the editor window, start the virtual machine (Alt .), and then add the shred with Alt + . Once the virtual machine is active, V: Yes, that works, thanks. WHIch reminds me of a problem I found with the organ, more about it in the HID section. The step I failed to do here was to start the machine, I thought one could add a shread to it which would implicitly start it, or something like that. I actually tried Audicle before the manual so that explains things, too. S: you can also use the buttons at the top of the document window to add, remove, and replace shreds. V: Ah now that you mentioned them, I used virtual focus and found it. Here the problem is that they are toolbar buttons and in WIndows toolbars are not in the tab order. From an accessibility point of view, buttons close to the start virtual machine button in which you can tab or shift+tab to would be better. But it is good enough to also mirror the choices in the menus so there's at least one accessible, and discoverable route. K: screenshot of the Mac version. This screenshot shows sliders and leds and so on and the idea is of course that this will be extended. V: Ah I see, sounds good to me. A classic accessibility question, does that panel UI have the concept of keyboard focus, tab order and the ability to interact with the controls using the same hotkeys for knobs that Mac and WIndows use for sliders? THese are the most frequently overlooked points in synth UI design, in terms of accessibility, and make life hard if your primary input medium is keyboard and output comes via synthetic speech. K: especially since I imagine that in the long run more visually impaired people will try programing for their electronic music needs V: That's right, and also because sighted folks might like precise keyboard control from time to time. Precise adjustment of values, and the ability to type in values using numbers or modal dialogs would rock. I've seen a zillion synth UIs in which I'd like to type in an exact cutoff value and cannot, though I have 102 keys at my Disposal. VV: was thinking the panel objects could maybe export their serialized state. That info could be then stored in a file directly. K: As I understood it Spencer wants to integrate such things with ChucK itself, not make it into a exception. I think that this would make it more versatile V: Most definitely. If chucK had serialization across the board, you could also store modules efficiently in files and load them directly. Maybe, you could even send objects like Synths over the network. K: For novice users it might be harder to use at first but that trade-off might be worth it V: Yeah, although I like how much ChucK ressembles C-like langs and Java in particular. It made it real easy to learn. I wish the manual would have a quick conversion guide, too. Regarding serialization it need not be that hard. Many languages have one function to turn an object into binary and another to get it back. In scripting langs that's generating a string representation of the data structure and evaluating it on the fly, in simple cases. K: quite right that MIDI learn needs a way to indicate what parameter is meant and right now -without a graphical interface- that would be a rather primitive affair. V: Yes, but could be done. Most old games would have you input controls in exactly the same order every time with no undo or step back facility. That could work for simple MIDI instruments, though is of course, very clumsy. [accessibility] VV: didn't mean the GUI would be customizable in the app <snip> it would use the colors and fonts you had picked for your OS. K: think that I myself run one of the Windows settings meant for disabled people; I use one of the high-contrast B&W ones because I find them easier on the eye and more clear. V: Well, here we go again. ANd one advantage of truetype fonts compared to bitmaps burned in the synth panel is that the text size is changible for highRes or low-vision users, and localizable, too. It's a common observation that sometimes accessibility can improve usability, too, especially for keyboard powr-users. At other times, the two are in conflict, e.g. whether to skip over grayed out menu entries from the keyboard. [screen readers] V: apps that programmatically reduce the graphical user interface to text rendered as speech and or Braille. <snip> following focus changes, heuristically guessing which pieces of text are labels and augmenting the keyboard interface <snip> K: Got it. This is the sort of thing why we need alt tags for images and Flash is of dubious value and so on. V: Yes, although Flash could be made accessible, only people lack the interest and or knowledge. It's sad people think of the Web when they think accessibility, where as it is desktop apps your average Joe or Jane is using most of the time. Often their accessibility is much worse. Nough said. [uses for ChucK] K: it's quite remarkable that nobody -that I know of- pointed out the interesting benefits that livecoding and generally using programming as a musical instrument might have for the visually impaired. V: It's funny you mention this. I didn't think of live performance when I looked into ChucK, although I imagine it could be of great value in that, too. As I input virtually everything via MIDI in my music, and record it live in a seq, the live aspect of chucK wasn't my cup of tea, though. What I'm looking for is a more accessible equivalent to Reaktor eventually, whose UI has been going down and down for years, in terms of accessibility. NAmely the ability to patch modular synths together and process MIDI eventes either in real time or off-lien in MIDi files like you would in Cakewalk application language. Another interest is as a simple testbench for audio and MIDi mangling ideas comparable to VST but much easier and faster to work with. I've noticed that graphical programming, as in Reaktor, has a limit after which it would be faster and more natural to express those ideas in code, particularly if you have to use magnification and speech to begin with. Laying out math formulae as a series of modules is one obvious example. And another is counting stuff. To count a 16-bit integer quantity in Reaktor, I Would have to worry about the range of events and chain too binary counters together to get the precision. Where as in ChucK I can say: 0 => int myCounter; // And that's that. ++myCounter; // to increment, and modulo to wrap. Similarly in computer science, as I have to view graphs one node at a time magnified, it is much easier to render the stuff on paper (laptop) or in my head as matrices, despite being much more a programmer than a mathematician of any kind. Also in Reaktor I have to spend time arranging modules on screen, which has absollutely nothing to do with how the synth operates or even looks to the user. Programming also gives you dynamic control for virtually everything. SAY I want to build an n to 1 audio switch, no problem, as the number of mixers instanciated can be parameterized. I have never used such a tac but howabout a synth which directly alters the patching and set of modules in it based on previous input and its current state, a complex Mealey state machine if you will. K: One thing that I've become very interested in is interface sonification for musical programs. V: I wonder if that's bene studied. As part of my Uni graduation stuff for investigating issues of current screen readers from the user point of view, which is just in its infancy, I've read plenty of stuff on accessibility. I do know auditory icons have been researched. K: so I'm developing my own instrument (a house style sequencer) which is build around two HID gaming devices and is playable without using the screen at all. V: Quite nice, actually. But where do you store the sequences and in what kind of a format? The japanese music macro language (mML) would be especially nice for expressing modulation and monophonic parts in ASCIIbetical notes. I mean, I've used it to compose mini tunes for the 8-bit Nintendo on the PC. I still lack an official spec, though I've written a simple MML parser for myself in Perl. K: found that I tend to input commands rhythmically after listening to a beat for a while regardless of whether those commands have anything to do with the beat directly. V: But what kind of params would you adjust via timing? tap tempo comes very naturally but I don't know about the rest. Of course, switching patterns needs to be done in time, too. [HID API and MIDI] VV: think I found a bug in it. In: .\chuck-1.2.0.7-exe\examples\hid\keyboard-organ.ck The keystrokes get delivered to chucK even when that window has not gotten the focus. I find that counter-intuittive and a potential problem if you multi-task several keyboard instruments <snip> K: Personally I like this. Right now I work with ChucK and the Nord Modular and I find it very convenient to be able to play ChucK with the keyboard V: I see, after choosing file/send message in the menu, I realized it all depends on the context. Most certainly, if you need to work with several music apps at the same time, that's handy. But suppose, if that's possible[?], I'd like to test my keyboard instrument, write some more code for it, and then reload the instrument without closing the chucK VM. In that case I wouldn't want to hear blips and bleeps while I'm coding. But one could remove the shread before coding, admitted. S: This is actually a feature, not a bug. The idea is to abstract the keyboard as just some device with buttons, <snip> it simply uses DirectInput. V: Ah I see, and agree that in this case, as there's an alternative API, the choice is quite sound actually. Of course it would be better to have a uniform API and a boolean flag to decide how to handle the focus. DirectINput is good, I do know those multi axes and button gamepads are well exposed via it. V: A funny thing about the organ. I've tried playing very fast in WIndows and it seems to me As though fast input is auto-quantized to some pretty coarse note value. I suppose this is a limitation of the keybord scanning, right? VV: this laptop has a Synaptics touchpad that can send pressure in addition to the xy coords. So I guess one could use it as a 3D mini Khaos pad if you will. S: ChucK probably won't be able to read the pressure of your touchpad at this point. V: No prob, I really didn't expect it to. I've only looked into this minimally and it seems as though I'll need to download an SDK to begin with, to even use the pressure in my apps. It is no wonder the WIndows mouse API doesn't support a third axis. Here's an odd idea, using pressure to navigate window z-order i.e. switch between the children in a top-level window like an MDI app. [MIDI] V: Lastly, some feedback about MIDI: I mentioned MIDI processing as one potential use and find the current API a bit raw. It would be great if I could transparently handle either off_line MIDI data in a file or stuff that's coming in on the wire in real time. Further more, if it is off-line, would be very cool if the data could be idealized a little: note on/off pairs to note events with length, a bunch of controller messages to 14-bit NRPN and RPN messages and full sysex messages whose checksum would be auto-calculated basedd on the manufacturer ID, you get the picture. I think the current API is about as raw as in VSt which means getting the job done but reinventing wheeels in the process. Ideally, I'd like to work at the musician level without having to know about running status, handle channels by masking off nibbles in binary data and recalling or defining constants for the various event types. A Java-like API with dedicated event classes for each one would be great. Even a more primitive presentation as in MIDI Perl using arrays would be quite all right. -- With kind regards Veli-Pekka Tätilä (vtatila@mail.student.oulu.fi) Accessibility, game music, synthesizers and programming: http://www.student.oulu.fi/~vtatila/