initial patch for 64-bit
Here is an initial attempt at a patch to make Chuck run on 64-bit architectures. Mostly it only modifies chuck_instr.cpp and chuck_emit.cpp. First I'll summarize the results so far and then describe the patches a little. Result: ------- This code seems to allow the Chuck VM to run on 64-bit platforms. I haven't had 100% success with every computer I've tried it on, but I'm able to run several test programs, with while loops. Function calls seem to work, as do classes with member functions. I ran into problems with the real-time audio, which I'll discuss below, however time does seem to advance and using WvOut I'm able to generate a seemingly good .wav file. I've only tested with SinOsc so far. Also, I should note that the only code in chuck_instr.cpp that I have *not touched* yet is the array stuff, which probably still has issues. (Although, it seems I'm able to make int and float arrays okay, but I would expect problems, seeing as the array types are based on byte size.) This has only been tested on Fedora and Ubuntu, it may work differently on other platforms. (Particularly with other compilers if I got the #ifdefs wrong for some particular case.) Oh yes and the patches are generated against today's CVS. Technical: ---------- The technique here is to change all the places that depend on byte count or type assumptions and replace them with word sizes instead. So, for example when the Reg_Push_Imm2 instruction is called, this no longer for "floats", but for "types using 2 words". Correspondingly, the emitter emits the 1-word instruction for floating-point values, so Imm2 is only used for complex types. So you'll see lots of replacements of things like "if ( .. type->size==4)" with "if ( .. type->words()==1 )". In general, word size is calculated by sizeof(type) / sizeof(long). This is done in chuck_def.h, and a series of "wd_" macros are defined similar to the "sz_" macros. There is one special case where I couldn't see any alternative except to use #ifdef, so I defined a macro called CK64BIT based on a few common compiler macros used to define a 64-bit environment. This case was for places that needed to agnostically refer to "some type" of a particular word size. For example, lots of code referred to t_CKUINT for a 1-word type and t_CKFLOAT for a 2-word type. This code didn't actually need to know that it was a floating-point number, since it's just copying memory from one place to another, but it needed a type of the correct size. So I defined t_CK1WORD, t_CK2WORD, and t_CK4WORD which are different depending on the architecture, and replaced references of CKUINT to CK1WORD, and CKFLOAT to CK2WORD, etc. Also, though arguably not necessary, there were also a lot of places where CKUINT is used interchangeably with object pointers. I replaced these with CKVOIDPTR. On both 32- and 64-bit architectures CKUINT and CKVOIDPTR are the same size, but I thought it would be better to make this more semantically accurate regardless. Lastly there are a few places where a "zero" value is placed on the stack. This was the only other place where I needed the CK64BIT macro, which was to define some constants for the zero values, which are of different sizes depending on the architecture. There was no chuck_def.cpp to place these in so I stuck them in chuck_type.cpp instead. Possibly memset() could be used here instead of using the const values. So, this was enough to get things running, but it seemed that if I included a ugen in the test program Chuck would just hang. It turns out this was some problem with RtAudio for ALSA, where snd_pcm_readn() returns EPIPE indicating XRUN, but the XRUN flag is not set. I don't know what the issue is, but I found that if snd_pcm_link() is not called the problem doesn't occur, so I've included here a work-around patch for now. I'm discussing this with Gary Scavone at the moment, so hopefully we'll be able to fix it. (It occurs even with RtAudio/STK 4.3.1, so he's interested in getting it working.) That said, even with this patch it doesn't quite *sound* right. A sine wave sounds all choppy and distorted, so it *might* have something to do with RtAudio's byte swapping code, but I'm still investigating this. The same script sent to a file with WvOut produces a good sine wave, so it's certainly an RtAudio issue and not a deeper problem. Finally, I was getting a segfault whenever an error message was printed, and I think it's due to the handling of variable arguments in the error handler, specifically the ap_list being used more than once after va_start(), so I include here a separate patch for this issue. .... So I've got to take a break from this now and do other things, because this has been somewhat consuming me for a week. I found it an interesting problem to solve, but the RtAudio issues are driving me _crazy_. If anyone wants to try this patch and tell me how it goes, I'd love to hear from you. I'll keep working on it eventually, but I have to catch up now on other stuff.. :) Steve
On Thu, Aug 21, 2008 at 1:01 AM, Stephen Sinclair
Here is an initial attempt at a patch to make Chuck run on 64-bit architectures.
Not to be presumptuous, but is there some more appropriate method for submitting patches, or is this just completely uninteresting? I'd be happy to post a version that applies on top of the last few days' commits. I noticed some work in the CVS related to vectorizing unit generators (tick_system_v), that sounds pretty interesting.. ;-) thanks, Steve
2008/8/29 Stephen Sinclair
or is this just completely uninteresting?
It sounds absolutely great to me. Actually I thought about posting that in response to your original post but wondered if sending "Go Stephen, Go Stephen, Go Stephen!" would be appropriate for the dev-list. ;¬) I don't anticipate 64 bit CPU's will be getting less popular any time soon so I can't imagine how this could be uninteresting. I noticed some work in the CVS related to vectorizing unit generators
(tick_system_v), that sounds pretty interesting.. ;-)
Ah, is this the "context sensitive block processing"? That's great! Three cheers for everybody generously donating their time and expertise! Kas.
On Fri, Aug 29, 2008 at 11:19 AM, Kassen
2008/8/29 Stephen Sinclair
or is this just completely uninteresting?
It sounds absolutely great to me. Actually I thought about posting that in response to your original post but wondered if sending "Go Stephen, Go Stephen, Go Stephen!" would be appropriate for the dev-list. ;¬)
I don't anticipate 64 bit CPU's will be getting less popular any time soon so I can't imagine how this could be uninteresting.
Ha, well i decided to post a partially completed patch for a few reasons: 1) I'm running out of 'summer time' to continue working on it, with the semester starting next week. 2) I hit a bit of a brick wall with the real-time issues, and need help. 3) It's generally better to communicate with other developers throughout a process rather than patch-bombing, and this 64-bit stuff is resulting in somewhat large patches. But I'm currently a bit confused whether the ChucK devs actually read and use this list, since there is obviously work being done on ChucK but I see no information about it, and no one responded to my patch. If not, I'm just wondering if there's a more appropriate place to discuss ChucK development or to at least submit patches. Steve
2008/8/29 Stephen Sinclair
But I'm currently a bit confused whether the ChucK devs actually read and use this list, since there is obviously work being done on ChucK but I see no information about it, and no one responded to my patch. If not, I'm just wondering if there's a more appropriate place to discuss ChucK development or to at least submit patches.
I agree that more communication on this level would be preferable. Another example is that we now have some Uanae (Flip and pilF and some more stuff) that I have no idea at all about what they do, how or why, I'm not sure how I could figure this out, short of diging in the source and reading up on whatever phenomenon they deal with. The examples that use them turn a sine into a the same sine so clearly they are relective but beyond that I don't get it. I pointed this out on the main list yet did not yet get a reply. Now, I realise that my friend Ge is a very busy man and may lack the time to send as many mails as he'd like. He has ChucK to maintain, students to teach to, workshops to give, a partner and who knows how much more. I have been and still am very outspoken in my respect for him and his generosity in sharing ChucK with us all. Still, it's my suspicion that if more time would be spend on open communication then that would make it easier to share the work (not just coding but also documentation, finding bugs and documenting them, answering questions, etc, etc, etc), thus decreasing the load on some individuals. Right now I am contributing a bit but I could do more (and have fun doing it) if we could improve comunication. I'd be happy to try my hand at fixing Sitar, for example, but it's not at all clear to me how Sitar should deal with very low notes (clearly not by outputting text warnings at sample-rate, which is the current behaviour). I'd also be happy to help write the manual but it's not at all clear to me in what form I should donate paragraphs or small sections. Perhaps this need not be said again but I'd like to stress that this is not meant as critisism and should merely be seen as my attempt to help think about how we could improve ChucK and ChucKist culture. Yours, Kas.
On Fri, Aug 29, 2008 at 12:03 PM, Kassen
2008/8/29 Stephen Sinclair
But I'm currently a bit confused whether the ChucK devs actually read and use this list, since there is obviously work being done on ChucK but I see no information about it, and no one responded to my patch. If not, I'm just wondering if there's a more appropriate place to discuss ChucK development or to at least submit patches.
I agree that more communication on this level would be preferable. Another example is that we now have some Uanae (Flip and pilF and some more stuff) that I have no idea at all about what they do, how or why, I'm not sure how I could figure this out, short of diging in the source and reading up on whatever phenomenon they deal with. The examples that use them turn a sine into a the same sine so clearly they are relective but beyond that I don't get it. I pointed this out on the main list yet did not yet get a reply.
Now, I realise that my friend Ge is a very busy man and may lack the time to send as many mails as he'd like. He has ChucK to maintain, students to teach to, workshops to give, a partner and who knows how much more. I have been and still am very outspoken in my respect for him and his generosity in sharing ChucK with us all.
Still, it's my suspicion that if more time would be spend on open communication then that would make it easier to share the work (not just coding but also documentation, finding bugs and documenting them, answering questions, etc, etc, etc), thus decreasing the load on some individuals. Right now I am contributing a bit but I could do more (and have fun doing it) if we could improve comunication. I'd be happy to try my hand at fixing Sitar, for example, but it's not at all clear to me how Sitar should deal with very low notes (clearly not by outputting text warnings at sample-rate, which is the current behaviour). I'd also be happy to help write the manual but it's not at all clear to me in what form I should donate paragraphs or small sections.
Actually you might find that a good place to discuss the Sitar model is on the STK mailing list, which is where it comes from.
Perhaps this need not be said again but I'd like to stress that this is not meant as critisism and should merely be seen as my attempt to help think about how we could improve ChucK and ChucKist culture.
Thanks Kas, I want to be clear that I share the same sentiment. Sorry if I come off being utilitarian in my communications sometimes, but I am usually just trying to be efficient. Obviously I think ChucK is a really cool idea and totally respect Ge for making it happen. But yes, I would love some communication and perhaps slightly more informative commit comments sometimes, too. ;-) I want to avoid situations where two people end up trying to solve the same problem, for example, stepping on each other's toes. For example, right now I have no idea, someone at Princeton or Stanford could already be working hard on the 64-bit problem and is totally annoyed by my patch, but I have no way of knowing this. I respect the idea of working on a product and simply pounding away at it until it works well, but I think a large reason for the success for projects like Pd is that it has fairly easy-to-understand code and everyone *talks* a lot while working on things. Making things easy and friendly to developers can only help a project go places faster. I'm actually fine with the idea of Ge not wanting to work with a larger base of developers, I'm okay with that, but the main reason I keep persisting is that he's said himself that he is interested in contributions. e.g., https://lists.cs.princeton.edu/pipermail/chuck-users/2008-April/002762.html Anyways, I'll just sit tight until I get some kind of answer. ;-) Steve
2008/8/29 Stephen Sinclair
Actually you might find that a good place to discuss the Sitar model is on the STK mailing list, which is where it comes from.
Ah. But does that version print to the screen as well? I suppose the real issue is verifying the frequency we ask it to use is within range so with regard to the fix the performance-wrecking error messages may not be a real issue. I may look into this route.
Thanks Kas, I want to be clear that I share the same sentiment. Sorry if I come off being utilitarian in my communications sometimes, but I am usually just trying to be efficient. Obviously I think ChucK is a really cool idea and totally respect Ge for making it happen. But yes, I would love some communication and perhaps slightly more informative commit comments sometimes, too. ;-)
I thought your sentiments were quite clear and your message "sounded" friendly to me, I just went a step further (or more general) so I wanted to be a 100% clear. It's not my place to tell anybody how to spend his/her time but I still wanted to comment on what's going on and how I think it could be improved (for everybody, I'm actually hoping for this to save Ge time).
I want to avoid situations where two people end up trying to solve the same problem, for example, stepping on each other's toes. For example, right now I have no idea, someone at Princeton or Stanford could already be working hard on the 64-bit problem and is totally annoyed by my patch, but I have no way of knowing this.
Yes, I agree. This is a important issue. Not just here but also with things like Maudi trying to improve the Mini's interface and add to it. I'm actually fine with the idea of Ge not wanting to work with a
larger base of developers, I'm okay with that, but the main reason I keep persisting is that he's said himself that he is interested in contributions.
Yes, it says so in the "TODO" file; "too many to name. Need more developers. Thanks." This line has been there since the dawn of "now" as far as I know. Also; "working on ChucK" is used as a teaching device at Princeton (likely Stanford as well?)... I think it's mainly the procedure that's unclear, I suspect. Of course it would be quite ChucKian for there not to be any set procedure at all.... Yours, Kas.
On Fri, Aug 29, 2008 at 6:23 PM, Stephen Sinclair
Anyways, I'll just sit tight until I get some kind of answer. ;-)
I cannot give you the answer you are waiting for, because I'm not a developer, but I would love to run chuck on my 64bit linux machine. I hope your patch will be committed, then I'll try to compile it and will most probably find some bug. Not that I'm eager to discovering bugs, but it happens all the time to me.
On Fri, Aug 29, 2008 at 1:15 PM, Oliver Oli
On Fri, Aug 29, 2008 at 6:23 PM, Stephen Sinclair
wrote: Anyways, I'll just sit tight until I get some kind of answer. ;-)
I cannot give you the answer you are waiting for, because I'm not a developer, but I would love to run chuck on my 64bit linux machine. I hope your patch will be committed, then I'll try to compile it and will most probably find some bug. Not that I'm eager to discovering bugs, but it happens all the time to me.
Of course it's not a completely working patch so you'll find things don't work. However the VM seems to be able to execute code and time can advance. I'd be interested to hear about crashes, but if only it could produce real-time sound properly I'd be more happy. I was able to generate a time-varying sinusoid in a .wav file using WvOut and it worked okay. Steve
Greetings again, Kassen and Steve! First of all, huge thank you's for all you've done for ChucK and friends!
But I'm currently a bit confused whether the ChucK devs actually read and use this list, since there is obviously work being done on ChucK but I see no information about it, and no one responded to my patch. If not, I'm just wondering if there's a more appropriate place to discuss ChucK development or to at least submit patches.
This is a great point. We haven't been using this list as much as would be helpful. Things have certainly been a bit crazy for everyone these last few months. I just got back from ICMC in Belfast, and a week before that Beijing. Others have had fairly crazy schedules as well.
I agree that more communication on this level would be preferable. Another example is that we now have some Uanae (Flip and pilF and some more stuff) that I have no idea at all about what they do, how or why, I'm not sure how I could figure this out, short of diging in the source and reading up on whatever phenomenon they deal with. The examples that use them turn a sine into a the same sine so clearly they are relective but beyond that I don't get it. I pointed this out on the main list yet did not yet get a reply.
We need an update on the documentation for the new UAnae in ChucK. Flip and pilF are meant to flip and unflip audio sample streams into vectors that work at the UAna level. We have a few more UAnae to add before this becomes useful. Apologies for the confusion and lack of explanation!
Now, I realise that my friend Ge is a very busy man and may lack the time to send as many mails as he'd like. He has ChucK to maintain, students to teach to, workshops to give, a partner and who knows how much more. I have been and still am very outspoken in my respect for him and his generosity in sharing ChucK with us all.
Still, it's my suspicion that if more time would be spend on open communication then that would make it easier to share the work (not just coding but also documentation, finding bugs and documenting them, answering questions, etc, etc, etc), thus decreasing the load on some individuals. Right now I am contributing a bit but I could do more (and have fun doing it) if we could improve comunication. I'd be happy to try my hand at fixing Sitar, for example, but it's not at all clear to me how Sitar should deal with very low notes (clearly not by outputting text warnings at sample-rate, which is the current behaviour). I'd also be happy to help write the manual but it's not at all clear to me in what form I should donate paragraphs or small sections.
Hear hear! I agree with all of this. At the moment, I honestly don't know how to effectively fix this, but please know that I genuinely want to. Feedback in this forum, despite lack of responses from me, have been extremely helpful. We'll need to come up with ways to submit patches, doc's, examples, etc... Working on it, though slowly!
Perhaps this need not be said again but I'd like to stress that this is not meant as critisism and should merely be seen as my attempt to help think about how we could improve ChucK and ChucKist culture.
Kassen, you rock! And as disarrayed as parts of the dev pipeline is, I am genuinely grateful for the ChucKist "culture" that we've all helped to build. The suggestions and criticisms are totally constructive, and even though we are often slow in fixing things, please know that we are deeply thankful for it all. Yours, Ge!
On Sun, Aug 31, 2008 at 2:43 PM, Ge Wang
Greetings again, Kassen and Steve!
First of all, huge thank you's for all you've done for ChucK and friends!
I've really been enjoying hacking on Chuck lately. ;) I find it pretty interesting to see how someone else has attacked the problem of writing a language for real-time, sample-accurate audio with dynamic behaviour and a complete VM. I hope I'm able to contribute something useful to the effort as ChucK makes a great alternative for me to the graphically oriented audio tools like Pd. Thanks for writing it! And I'm really happy you seem to be interested in continuing it even after finishing your thesis. ;-) I would fully understand the desire to leave such a big project behind after an effort like that, at least for a few months of brain-rest. But all the more reason to encourage others to take on the new challenges!
But I'm currently a bit confused whether the ChucK devs actually read and use this list, since there is obviously work being done on ChucK but I see no information about it, and no one responded to my patch. If not, I'm just wondering if there's a more appropriate place to discuss ChucK development or to at least submit patches.
This is a great point. We haven't been using this list as much as would be helpful. Things have certainly been a bit crazy for everyone these last few months. I just got back from ICMC in Belfast, and a week before that Beijing. Others have had fairly crazy schedules as well.
Of course ICMC must have been crazy.. ;-) and Beijing too, no doubt! I guess some people here at McGill have just gotten back from that too, I'll have to ask how it was.. Sorry for being whiny about my patch, I waited a week before complaining.. ;-) I find in open source, sometimes a little nudge helps get people over the "hump" of getting around to examining patches and bug fixing.. I'll continue to use this list as I think a mailing list is a great way to send patches and generally communicate about development, but for this very reason I was just a little surprised at the lack of mail lately. ICMC is a perfectly good excuse of course.. ;-)
Still, it's my suspicion that if more time would be spend on open communication then that would make it easier to share the work (not just coding but also documentation, finding bugs and documenting them, answering questions, etc, etc, etc), thus decreasing the load on some individuals. Right now I am contributing a bit but I could do more (and have fun doing it) if we could improve comunication. I'd be happy to try my hand at fixing Sitar, for example, but it's not at all clear to me how Sitar should deal with very low notes (clearly not by outputting text warnings at sample-rate, which is the current behaviour). I'd also be happy to help write the manual but it's not at all clear to me in what form I should donate paragraphs or small sections.
Hear hear! I agree with all of this. At the moment, I honestly don't know how to effectively fix this, but please know that I genuinely want to. Feedback in this forum, despite lack of responses from me, have been extremely helpful. We'll need to come up with ways to submit patches, doc's, examples, etc... Working on it, though slowly!
Well, one way might be to take advantage of services like sourceforge which provide all these things. Not necessarily SF of course, since not everyone likes it. (Personally i don't mind it, though it's sometimes slow.) My preference is to make use of the mailing list instead of or in addition to a patch tracker, but either way some way of tracking these things is always helpful. Submitting patches against a specific CVS date is pretty much equivalent to sending a tarball. The nice thing about including patches directly in an email is that you can look at them (at least short ones) critically before even trying to compile and test. Another option would be to install something like Trac or Redmine on the web server, but it's a lot more work to administer something like that yourself. I've said it before, but a great way to deal with looking at other people's development trees is to use git (along with gitorious.org or repo.or.cz for hosting), which allows someone to do all sorts of development and then tell the mailing list, 'hey, look I added this feature, come grab it'. That way everyone takes care of their own source repository, and you can easily grab the good parts from what other people are working on when they are ready. (While retaining authorship credits, etc.) I'm currently using git to track the chuck CVS which makes it much easier to port my changes to the latest version. cheers, Steve
Stephen Sinclair wrote:
Well, one way might be to take advantage of services like sourceforge [...]
Google code tends to be preferable these days.
I've said it before, but a great way to deal with looking at other people's development trees is to use git [...]
I agree that ChucK would be a nice case for Git since it would let development continue even when Ge's not around to approve patches. Alternatively, moving to something like Google-hosted-svn with commit access open to a few folks would be nice as well. The couple of times that I sent or offered to send patches didn't get any response either, which kind of sapped my motivation to do further ones. -Scott
Hey, Ge! This is a great point. We haven't been using this list as much as would
be helpful. Things have certainly been a bit crazy for everyone these last few months. I just got back from ICMC in Belfast, and a week before that Beijing. Others have had fairly crazy schedules as well.
But you're back! I'm back as well, even if I just helped some close friends move, then build/re-build some cupboards for them, then practice livecoding in the evening. We need an update on the documentation for the new UAnae in ChucK. Flip and
pilF are meant to flip and unflip audio sample streams into vectors that work at the UAna level. We have a few more UAnae to add before this becomes useful. Apologies for the confusion and lack of explanation!
Ok, that's one step.. But at the risk of sounding like my friend's 3.5 years old son I'd now like to know what a "vector" is in this context (I know what they are in geometry, etc). Generally it's not that clear to me what sort of data the Uanae are sending to each other and from what I've seen so far they only become interesting once you know this. Hear hear! I agree with all of this. At the moment, I honestly don't
know how to effectively fix this, but please know that I genuinely want to. Feedback in this forum, despite lack of responses from me, have been extremely helpful. We'll need to come up with ways to submit patches, doc's, examples, etc... Working on it, though slowly!
I think we need to look at things in their context and find domain speciffic solutions. I think the way of tracking bugs that we've used on the WiKi for the last few version is a improvement, for example. That way makes it much more clear what bugs are open and it gives a sense of progress to move them to "Fixed in version n" once fixed. To me this is much better then the old page which is a bit of a mess. I could see this being extended by developers replying (in bold, for example) to such reports to either request more information or indicate a fix is in the works or in CVS. I could imagine the WiKi being used for co-writing or proposing manual sections as well, perhaps with a similar system for indicating a section has been merged into the PDF. Especially for the non-code aspects of this question I think the main thing is to get the treshold for helping out down (there is, for example, no real need for everybody to learn latex just to contribute to the manual) and increase the feedback (for example by stamping bugs with a label that says you or Perry or Dan or.... is looking into it). These things are relatively independant problems so let's tackle them on a independant basis; I found that treating a set of issues like one huge question is a great way to make sure you'll never solve it ;¬). Kassen, you rock! And as disarrayed as parts of the dev pipeline is, I
am genuinely grateful for the ChucKist "culture" that we've all helped to build. The suggestions and criticisms are totally constructive, and even though we are often slow in fixing things, please know that we are deeply thankful for it all.
I know, but written comunication can so easily be taken the wrong way that I wanted to be 100% clear. Good to see you back, hope you had a great time in Beijing and at the ICMC! Yours, Kas.
Greetings Steve and Kassen!
Here is an initial attempt at a patch to make Chuck run on 64-bit architectures.
This sounds awesome and to be sure it is something that is very much needed. Thank you for making the effort. We are currently in an effort to add some additional features (first passes at adaptive block processing and file I/O). As for submitting patches, sorry for the disarray and lack of systematic methods! Patches are good, or if you wouldn't mind, a snapshot of a source ball that builds might be even better. Rock on, Ge!
participants (5)
-
Ge Wang
-
Kassen
-
Oliver Oli
-
Scott Wheeler
-
Stephen Sinclair