Efficient analysis & resynthesis for game audio
Hello! I've been using Pure Data and the Heavy compiler (hvcc) to make procedural audio for games/interactive media pieces the last couple of years. I just started going through the videos of the Kadenze course. The first couple of lessons, I did what Mr. Cook was doing in his ChucK software using Pure Data... but then I got to Session 3 where he showcases a method of resynthesis using two scripts- one to analyze a sound file and output a list of peaks (frequency and amplitude), and one to perform the resynthesis. I was very impressed with how quickly this goes as you can just copy and paste the list from the 1st list into the 2nd one and you already have a working model which you can then begin to tweak. With Pure Data, this would be an exponentially more arduous process of manually looking at a spectral analysis, filling out boxes, connecting them with wires, copying & pasting for each frequency... I'm curious as to what the possibilities of integrating these ChucK algorithms into standard programming languages like C++/C# for use in game engines. On the download page of ChuCK it says that the newest versions are made to be embedded in other systems. Is there documentation detailing this somewhere? -- *S・エリオット・ペレスwww.selliotp.com http://www.selliotp.com*Sound Designer for * enenインタラクティブサウンドディザイン* [image: http://enengame.com/] https://youtu.be/xYpc9agfil8 *「エンエン」は日本のAppStoreでリリースされました!* [image: https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8] https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8
Hello!
There's documentation for the Unity embedding here:
http://chuck.stanford.edu/chunity/
Compared to Heavy, your ChucK code is compiled at runtime, so you can even
assemble ChucK scripts on the fly with string manipulation while your game
is currently running.
If you're looking to embed ChucK in another program, you will want to use
the ChucK class in the core/chuck.cpp file as the interface. You will need
to include all of the core/ files and none of the host/ files. The changes
we made to ChucK involved separating its core logic (most of it) from its
host -- command line and RtAudio interface, as well as reducing its
dependencies on global variables so that multiple ChucK VM instances can
coexist.
Then, there's also the new "global" variables in ChucK. If you add the
global keyword to a variable, it's accessible by name across the entire VM
(ChucK instance), and can also be set and gotten from outside ChucK (e.g.
from Unity).
~Jack
On Fri, May 1, 2020 at 5:01 AM S. Elliot Perez
Hello!
I've been using Pure Data and the Heavy compiler (hvcc) to make procedural audio for games/interactive media pieces the last couple of years. I just started going through the videos of the Kadenze course. The first couple of lessons, I did what Mr. Cook was doing in his ChucK software using Pure Data... but then I got to Session 3 where he showcases a method of resynthesis using two scripts- one to analyze a sound file and output a list of peaks (frequency and amplitude), and one to perform the resynthesis. I was very impressed with how quickly this goes as you can just copy and paste the list from the 1st list into the 2nd one and you already have a working model which you can then begin to tweak. With Pure Data, this would be an exponentially more arduous process of manually looking at a spectral analysis, filling out boxes, connecting them with wires, copying & pasting for each frequency...
I'm curious as to what the possibilities of integrating these ChucK algorithms into standard programming languages like C++/C# for use in game engines. On the download page of ChuCK it says that the newest versions are made to be embedded in other systems. Is there documentation detailing this somewhere?
--
*S・エリオット・ペレスwww.selliotp.com http://www.selliotp.com*Sound Designer for * enenインタラクティブサウンドディザイン* [image: http://enengame.com/] https://youtu.be/xYpc9agfil8 *「エンエン」は日本のAppStoreでリリースされました!* [image: https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8] https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8 _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
Hi Jack,
That's super interesting, especially the bit about global variables.
How's that suppose to work? I don't seem to be able to make it work on
ChucK 1.4.0.0 . Just tried on MacOS High Sierra 10.13.6 .
Cheers,
Mario
On Fri, 1 May 2020 at 15:34, Jack Atherton
Hello!
There's documentation for the Unity embedding here: http://chuck.stanford.edu/chunity/ Compared to Heavy, your ChucK code is compiled at runtime, so you can even assemble ChucK scripts on the fly with string manipulation while your game is currently running.
If you're looking to embed ChucK in another program, you will want to use the ChucK class in the core/chuck.cpp file as the interface. You will need to include all of the core/ files and none of the host/ files. The changes we made to ChucK involved separating its core logic (most of it) from its host -- command line and RtAudio interface, as well as reducing its dependencies on global variables so that multiple ChucK VM instances can coexist.
Then, there's also the new "global" variables in ChucK. If you add the global keyword to a variable, it's accessible by name across the entire VM (ChucK instance), and can also be set and gotten from outside ChucK (e.g. from Unity).
~Jack
On Fri, May 1, 2020 at 5:01 AM S. Elliot Perez
wrote: Hello!
I've been using Pure Data and the Heavy compiler (hvcc) to make procedural audio for games/interactive media pieces the last couple of years. I just started going through the videos of the Kadenze course. The first couple of lessons, I did what Mr. Cook was doing in his ChucK software using Pure Data... but then I got to Session 3 where he showcases a method of resynthesis using two scripts- one to analyze a sound file and output a list of peaks (frequency and amplitude), and one to perform the resynthesis. I was very impressed with how quickly this goes as you can just copy and paste the list from the 1st list into the 2nd one and you already have a working model which you can then begin to tweak. With Pure Data, this would be an exponentially more arduous process of manually looking at a spectral analysis, filling out boxes, connecting them with wires, copying & pasting for each frequency...
I'm curious as to what the possibilities of integrating these ChucK algorithms into standard programming languages like C++/C# for use in game engines. On the download page of ChuCK it says that the newest versions are made to be embedded in other systems. Is there documentation detailing this somewhere?
--
*S・エリオット・ペレスwww.selliotp.com http://www.selliotp.com*Sound Designer for * enenインタラクティブサウンドディザイン* [image: http://enengame.com/] https://youtu.be/xYpc9agfil8 *「エンエン」は日本のAppStoreでリリースされました!* [image: https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8] https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8 _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
In ChucK 1.4.0.0 the "global" keyword got released as "external" -- we since switched to global because we thought it made more intuitive sense. You can use "external" for now and then when you upgrade ChucK in the future, "external" will still work, but it will print a message saying it's deprecated and to use "global" instead. ~Jack On Fri, May 1, 2020 at 9:10 AM Mario Buoninfante < mario.buoninfante@gmail.com> wrote:
Hi Jack,
That's super interesting, especially the bit about global variables. How's that suppose to work? I don't seem to be able to make it work on ChucK 1.4.0.0 . Just tried on MacOS High Sierra 10.13.6 .
Cheers, Mario
On Fri, 1 May 2020 at 15:34, Jack Atherton
wrote: Hello!
There's documentation for the Unity embedding here: http://chuck.stanford.edu/chunity/ Compared to Heavy, your ChucK code is compiled at runtime, so you can even assemble ChucK scripts on the fly with string manipulation while your game is currently running.
If you're looking to embed ChucK in another program, you will want to use the ChucK class in the core/chuck.cpp file as the interface. You will need to include all of the core/ files and none of the host/ files. The changes we made to ChucK involved separating its core logic (most of it) from its host -- command line and RtAudio interface, as well as reducing its dependencies on global variables so that multiple ChucK VM instances can coexist.
Then, there's also the new "global" variables in ChucK. If you add the global keyword to a variable, it's accessible by name across the entire VM (ChucK instance), and can also be set and gotten from outside ChucK (e.g. from Unity).
~Jack
On Fri, May 1, 2020 at 5:01 AM S. Elliot Perez
wrote: Hello!
I've been using Pure Data and the Heavy compiler (hvcc) to make procedural audio for games/interactive media pieces the last couple of years. I just started going through the videos of the Kadenze course. The first couple of lessons, I did what Mr. Cook was doing in his ChucK software using Pure Data... but then I got to Session 3 where he showcases a method of resynthesis using two scripts- one to analyze a sound file and output a list of peaks (frequency and amplitude), and one to perform the resynthesis. I was very impressed with how quickly this goes as you can just copy and paste the list from the 1st list into the 2nd one and you already have a working model which you can then begin to tweak. With Pure Data, this would be an exponentially more arduous process of manually looking at a spectral analysis, filling out boxes, connecting them with wires, copying & pasting for each frequency...
I'm curious as to what the possibilities of integrating these ChucK algorithms into standard programming languages like C++/C# for use in game engines. On the download page of ChuCK it says that the newest versions are made to be embedded in other systems. Is there documentation detailing this somewhere?
--
*S・エリオット・ペレスwww.selliotp.com http://www.selliotp.com*Sound Designer for * enenインタラクティブサウンドディザイン* [image: http://enengame.com/] https://youtu.be/xYpc9agfil8 *「エンエン」は日本のAppStoreでリリースされました!* [image: https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8] https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8 _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
thanks Jack, can't wait to give it a try :) On 01/05/2020 17:45, Jack Atherton wrote:
In ChucK 1.4.0.0 the "global" keyword got released as "external" -- we since switched to global because we thought it made more intuitive sense. You can use "external" for now and then when you upgrade ChucK in the future, "external" will still work, but it will print a message saying it's deprecated and to use "global" instead.
~Jack
On Fri, May 1, 2020 at 9:10 AM Mario Buoninfante
mailto:mario.buoninfante@gmail.com> wrote: Hi Jack,
That's super interesting, especially the bit about global variables. How's that suppose to work? I don't seem to be able to make it work on ChucK 1.4.0.0 . Just tried on MacOS High Sierra 10.13.6 .
Cheers, Mario
On Fri, 1 May 2020 at 15:34, Jack Atherton
mailto:lja@ccrma.stanford.edu> wrote: Hello!
There's documentation for the Unity embedding here: http://chuck.stanford.edu/chunity/ Compared to Heavy, your ChucK code is compiled at runtime, so you can even assemble ChucK scripts on the fly with string manipulation while your game is currently running.
If you're looking to embed ChucK in another program, you will want to use the ChucK class in the core/chuck.cpp file as the interface. You will need to include all of the core/ files and none of the host/ files. The changes we made to ChucK involved separating its core logic (most of it) from its host -- command line and RtAudio interface, as well as reducing its dependencies on global variables so that multiple ChucK VM instances can coexist.
Then, there's also the new "global" variables in ChucK. If you add the global keyword to a variable, it's accessible by name across the entire VM (ChucK instance), and can also be set and gotten from outside ChucK (e.g. from Unity).
~Jack
On Fri, May 1, 2020 at 5:01 AM S. Elliot Perez
mailto:s.elliot.perez@gmail.com> wrote: Hello!
I've been using Pure Data and the Heavy compiler (hvcc) to make procedural audio for games/interactive media pieces the last couple of years. I just started going through the videos of the Kadenze course. The first couple of lessons, I did what Mr. Cook was doing in his ChucK software using Pure Data... but then I got to Session 3 where he showcases a method of resynthesis using two scripts- one to analyze a sound file and output a list of peaks (frequency and amplitude), and one to perform the resynthesis. I was very impressed with how quickly this goes as you can just copy and paste the list from the 1st list into the 2nd one and you already have a working model which you can then begin to tweak. With Pure Data, this would be an exponentially more arduous process of manually looking at a spectral analysis, filling out boxes, connecting them with wires, copying & pasting for each frequency...
I'm curious as to what the possibilities of integrating these ChucK algorithms into standard programming languages like C++/C# for use in game engines. On the download page of ChuCK it says that the newest versions are made to be embedded in other systems. Is there documentation detailing this somewhere?
-- *S・エリオット・ペレス www.selliotp.com http://www.selliotp.com *Sound Designer for*enen インタラクティブサウンドディザイン* http://enengame.com/ https://youtu.be/xYpc9agfil8 _*「エンエン」は日本のAppStoreでリリースされました!*_ https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8 https://itunes.apple.com/us/app/enen/id1164297889?ls=1&mt=8 _______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu mailto:chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu mailto:chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu mailto:chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users
-- electronic musician, sound artist, creative coder, QA engineer https://vimeo.com/creativecodingsalerno http://mbuoninfante.tumblr.com/ https://github.com/mariobuoninfante https://bitbucket.org/mariobuoninfante/
participants (4)
-
Jack Atherton
-
mario buoninfante
-
Mario Buoninfante
-
S. Elliot Perez