Are there tools or techniques for monitoring memory allocation in ChucK? I've been stress testing my code, and after about five hours ChucK vm stops with a malloc / mmap failure. It would be nice to figure out what's clogging memory. [In a previous life, I was an embedded systems programmer, so I'm happy to create reusable object pools and managing the objects myself, but I need to know what leaks and what doesn't.] On a related note, I notice that ChucK uses the standard library malloc rather centralizing calls through a single chuck_malloc() (or the existing checked_malloc()). Going through a single call would make it easier to monitor heap allocations. Just my $0.25. - Rob
Rob; Are there tools or techniques for monitoring memory allocation in ChucK?
I've been stress testing my code, and after about five hours ChucK vm stops with a malloc / mmap failure. It would be nice to figure out what's clogging memory. [In a previous life, I was an embedded systems programmer, so I'm happy to create reusable object pools and managing the objects myself, but I need to know what leaks and what doesn't.]
From my experience (looking at task manager on Windows and Top on Linux) defining anything will allocate memory and this won't ever be freed. I'm not sure what the current state of garbage collection is but last time I checked I believe this even included the temporary variables in "for" loops. In case of doubt it's currently fairly safe to assume it'll leak though memory allocated for samples should be freed (I think).
For some types of program you can simply make sure to never define anything inside of loops, going as far as to re-use the temporary variables in those "for" loops. I think that can lead to garbage-less code but I'm not sure about function calls. That's the bad news (I think everybody agrees this is quite bad). the good news is that it's being worked on and some GC is (supposedly) already in place. One of the most important techniques I use to deal with this is making sure anything "large" (classes, UGens) of which I need multiple copies will be stored in a array so I can re-use one copy once I'm done with it; basically I try to make sure it never becomes impossible to address. This creates some overhead in code and complexity but at least memory leakage will be contained to reasonable levels. Hope that clarifies, Kas.
I think with careful coding (and this goes down into the ugen level) you should be able to design fairly leak-less code without GC. I ran an RTcmix script -- about 10 scheduled notes/second -- for several days without seeing any significant memory hit. brad http://music.columbia.edu/~brad On Mar 17, 2009, at 11:08 PM, Kassen wrote:
Rob;
Are there tools or techniques for monitoring memory allocation in ChucK? I've been stress testing my code, and after about five hours ChucK vm stops with a malloc / mmap failure. It would be nice to figure out what's clogging memory. [In a previous life, I was an embedded systems programmer, so I'm happy to create reusable object pools and managing the objects myself, but I need to know what leaks and what doesn't.]
From my experience (looking at task manager on Windows and Top on Linux) defining anything will allocate memory and this won't ever be freed. I'm not sure what the current state of garbage collection is but last time I checked I believe this even included the temporary variables in "for" loops. In case of doubt it's currently fairly safe to assume it'll leak though memory allocated for samples should be freed (I think).
For some types of program you can simply make sure to never define anything inside of loops, going as far as to re-use the temporary variables in those "for" loops. I think that can lead to garbage- less code but I'm not sure about function calls.
That's the bad news (I think everybody agrees this is quite bad).
the good news is that it's being worked on and some GC is (supposedly) already in place.
One of the most important techniques I use to deal with this is making sure anything "large" (classes, UGens) of which I need multiple copies will be stored in a array so I can re-use one copy once I'm done with it; basically I try to make sure it never becomes impossible to address. This creates some overhead in code and complexity but at least memory leakage will be contained to reasonable levels.
Hope that clarifies, Kas. _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
Kas (and ChucK community): The code I'm writing really needs a "wait-on-event-or-timeout- whichever-comes-first" function, and the obvious way to write this is to spork a shred at each call. Since I will be using ChucK in live performances, the fact that memory is exhausted after 20K calls to woeotwcf is a concern to me. If shreds are allocated from a pool, then profligate sporking isn't what's causing the system to run out of memory, but without monitoring tools, I have no way to know. The short statement is "I can't optimize what I can't see". It would be nice to have visibility into where the storage is getting eaten up. Or a GC. - Rob On 17 Mar 2009, at 20:08, Kassen wrote:
Rob;
Are there tools or techniques for monitoring memory allocation in ChucK? I've been stress testing my code, and after about five hours ChucK vm stops with a malloc / mmap failure. It would be nice to figure out what's clogging memory. [In a previous life, I was an embedded systems programmer, so I'm happy to create reusable object pools and managing the objects myself, but I need to know what leaks and what doesn't.]
From my experience (looking at task manager on Windows and Top on Linux) defining anything will allocate memory and this won't ever be freed. I'm not sure what the current state of garbage collection is but last time I checked I believe this even included the temporary variables in "for" loops. In case of doubt it's currently fairly safe to assume it'll leak though memory allocated for samples should be freed (I think).
For some types of program you can simply make sure to never define anything inside of loops, going as far as to re-use the temporary variables in those "for" loops. I think that can lead to garbage- less code but I'm not sure about function calls.
That's the bad news (I think everybody agrees this is quite bad).
the good news is that it's being worked on and some GC is (supposedly) already in place.
One of the most important techniques I use to deal with this is making sure anything "large" (classes, UGens) of which I need multiple copies will be stored in a array so I can re-use one copy once I'm done with it; basically I try to make sure it never becomes impossible to address. This creates some overhead in code and complexity but at least memory leakage will be contained to reasonable levels.
Hope that clarifies, Kas. _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
Rob;
The short statement is "I can't optimize what I can't see". It would be nice to have visibility into where the storage is getting eaten up. Or a GC.
I agree. I think you might be able to tell something about memory allocation by turning up the "verbose" setting to something very high but we do need GC as well. I could also imagine that eventually the Audicle and/ or mini might have such diagnostic tools added to them. It's not clear to me why you are discarding those two shreds after you used them though. Why not simply have two shreds that wait for eventA / eventB respectively, signal eventC when their respective event comes in and afterwards resume waiting? That would merge two events into one and would only require your time-out function to be stopped or rest in case the event arrives before the timer times out. Yours, Kas.
I haven't read enough to be sure, but there are signs that while Chuck frees some dead memory, it has no general garbage collection. For example, consider these definitions in chuck_vm.cpp (which, btw, aren't called anywhere): //----------------------------------------------------------------------------- // name: gc // desc: ... //----------------------------------------------------------------------------- void Chuck_VM::gc( t_CKUINT amount ) { } //----------------------------------------------------------------------------- // name: gc // desc: ... //----------------------------------------------------------------------------- void Chuck_VM::gc( ) { } On Tue, 17 Mar 2009, Robert Poor wrote:
Are there tools or techniques for monitoring memory allocation in ChucK? I've been stress testing my code, and after about five hours ChucK vm stops with a malloc / mmap failure. It would be nice to figure out what's clogging memory. [In a previous life, I was an embedded systems programmer, so I'm happy to create reusable object pools and managing the objects myself, but I need to know what leaks and what doesn't.]
On a related note, I notice that ChucK uses the standard library malloc rather centralizing calls through a single chuck_malloc() (or the existing checked_malloc()). Going through a single call would make it easier to monitor heap allocations. Just my $0.25.
- Rob
-- Tom Duff. System clock out of range.
participants (4)
-
Brad Garton -
Kassen -
Robert Poor -
Tom Duff