[chuck-users] DSP question: accumulating frequency info using FFT?

Jascha Narveson jnarveson at wesleyan.edu
Sat Oct 14 14:10:54 EDT 2017


Hi, Eric -

Interesting!  This is definitely a step closer.

On my own, I also tried running my original code with a constant mag of 0.1 in every bin, but with the phase updating live, and voila: I was hearing a dirty and distant version of the original signal, attacks and all.  So, you’re right about that: phase info is important, and can sound like the source even if the mags are all kept steady.

Your averaging idea seems like a good route to try - maybe what I’m after is more of an average of a series of successive FFT-freezes, or something…

…back to the drawing board…

Thanks!

-j-





> On Oct 14, 2017, at 1:59 PM, Eric Heep <ericheep at gmail.com> wrote:
> 
> I have a feeling that the phase information may be important to keep around.
> 
> I added a moving average filter into your code, and then added that average into the accumulation spectrum. I'm also including the phase information, and while it all sounds more "smeary", it is also introducing windowing artifacts (I think that's what those are). 
> 
> I would be interested in a way to clean those up.
> 
> SinOsc synth => FFT fft => blackhole;
> 
> UAnaBlob blob;
> 
> IFFT ifft => dac;
> 
> 
> 
> synth.freq(440);
> 
> synth.gain(0.1);
> 
> 
> 
> // set parameters
> 
> 1024 => fft.size;
> 
> Windowing.hamming(fft.size() / 2) => fft.window;
> 
> Windowing.hamming(fft.size() / 2) => ifft.window;
> 
> 
> 
> // place to hold the FFT data as it comes in
> 
> complex spec[fft.size()/2];
> 
> 
> 
> // place to store the accumulated FFT data, preload with zeroes
> 
> complex acc[fft.size()/2];
> 
> for(0 => int i; i < acc.cap(); i++) {
> 
>     #(0, 0) => acc[i];
> 
> }
> 
> 
> 
> // do it!
> 
> spork ~ sawTune();
> 
> spork ~ accumulateFrequenciesViaFFT(fft.size()/2);
> 
> 
> 
> while (true) {
> 
>     1::second => now;
> 
> }
> 
> 
> 
> fun void accumulateFrequenciesViaFFT(int HALF_FFT_SIZE) {
> 
>     4 => int FILTER_SIZE;
> 
>     float movingAverageMag[FILTER_SIZE][HALF_FFT_SIZE];
> 
>     float movingAveragePhase[FILTER_SIZE][HALF_FFT_SIZE];
> 
>     
>     while( true )
> 
>     {
> 
>         fft.upchuck() @=> blob;
> 
>         // get the data
> 
>         blob.cvals() @=> spec;
> 
>         
>         // for every bin...
> 
>         for(0 => int i; i < spec.cap(); i++) {
> 
>             // get the mag/phase for each bin by converting to polar
> 
>             spec[i] $ polar => polar newBin;
> 
>             
>             // and get our running totals from acc
> 
>             acc[i] $ polar => polar accBin;
> 
>             
>             // move old values down the line
> 
>             for (FILTER_SIZE - 1 => int j; j > 0; j--) {
> 
>                 movingAverageMag[j - 1][i] => movingAverageMag[j][i];
> 
>                 movingAveragePhase[j - 1][i] => movingAveragePhase[j][i];
> 
>             }          
> 
>               
>             // add in new value
> 
>             newBin.mag => movingAverageMag[0][i];
> 
>             newBin.phase => movingAveragePhase[0][i];
> 
>             
>             // get sums for averaging
> 
>             float magSum, phaseSum;
> 
>             for (0 => int j; j < FILTER_SIZE; j++) {
> 
>                 movingAverageMag[j][i] +=> magSum;
> 
>                 movingAveragePhase[j][i] +=> phaseSum;
> 
>             }
> 
>             
>             // average
> 
>             magSum/FILTER_SIZE => float magAvg;
> 
>             phaseSum/FILTER_SIZE => float phaseAvg;
> 
> 
> 
>             magAvg * 0.005 +=> accBin.mag;
> 
>             phaseAvg * 0.005 +=> accBin.phase;
> 
>             
>             // let phase pass through
> 
>             //newBin.phase => accBin.phase;
> 
>             /*Math.random2f(0, 2*pi) => accBin.phase;*/
> 
>             
>             // convert back to complex and put back in the acc array
> 
>             accBin $ complex => acc[i];
> 
>         }
> 
>         
>         ifft.transform(acc);
> 
>         (fft.size() / 4)::samp => now;
> 
>     }
> 
> }
> 
> 
> 
> fun void sawTune() {
> 
>     while (true) {
> 
>         Math.random2(48, 92) => int midinote;
> 
>         Math.mtof(midinote) => synth.freq;
> 
>         333::ms => now;
> 
>     }
> 
> }
> 
> 
> On Sat, Oct 14, 2017 at 10:31 AM, Jascha Narveson <jnarveson at wesleyan.edu <mailto:jnarveson at wesleyan.edu>> wrote:
> 
> Here’s the same thing as my original questions, but with some polite windowing - same problems persist: I’m hearing attacks (why?) and it’s pretty noisy - I guess because letting magnitudes creep up across the spectrum I’m just ultimately aiming for a big loud saw wave.
> 
> So I guess I’m looking to do two things:
> 
> - avoid hearing attacks in the resynthesized sound from my “accumulated magnitudes” spectrum
> 
> - artfully avoid the inevitable sawtooth sound by changing my approach to something the follows the spirit of the original idea and not the actual idea.
> 
> Thoughts?
> 
> cheers,
> 
> - jascha
> 
> 
> 
> - - - - same thing, a bit better maybe - - - -
> 
> SinOsc synth => FFT fft => blackhole;
> UAnaBlob blob;
> IFFT ifft => dac;
> 
> synth.freq(440);
> synth.gain(0.5);
> 
> 
> second / samp => float srate;
> 
> // set parameters
> 1024 => fft.size;
> Windowing.hamming(fft.size() / 2) => fft.window;
> Windowing.hamming(fft.size() / 2) => ifft.window;
> 
> // hold the FFT data as it comes in
> complex spec[fft.size()/2];
> 
> // store the accumulated FFT data
> complex acc[fft.size()/2];
> 
> // fill the accumulated complex spectral array with zeroes
> for(0 => int i; i < acc.cap(); i++) {
>   #(0, 0) => acc[i];
> }
> 
> // spork shreds
> spork ~ synthTune();
> spork ~ accumulateMagnitudesViaFFT();
> 
> while (true) {
>   1::second => now;
> }
> 
> fun void accumulateMagnitudesViaFFT() {
>   while( true )
>   {
>       fft.upchuck() @=> blob;
>       // get the data
>       blob.cvals() @=> spec;
> 
>       // for every bin...
>       for(0 => int i; i < spec.cap(); i++) {
>         // get the mag/phase for each bin by converting to polar
>         spec[i] $ polar => polar newBin;
> 
>         // and get our running totals from acc
>         acc[i] $ polar => polar accBin;
> 
>         // scale the inocming mag and add it to acc
>         newBin.mag * 0.001 +=> accBin.mag;
> 
>         // let phase pass through
>         newBin.phase => accBin.phase;
> 
>         // conver back to complex and put back in the acc array
>         accBin $ complex => acc[i];
>       }
> 
>       ifft.transform(acc);
>       (fft.size() / 4)::samp => now;
>   }
> }
> 
> fun void synthTune() {
>   while (true) {
>     Math.random2(48, 72) => int midinote;
>     Math.mtof(midinote) => synth.freq;
>     333::ms => now;
>     <<<acc[6] $ polar >>>; // just to check - yep: they're getting bigger...
>   }
> }
> 
> 
> 
> > On Oct 14, 2017, at 12:26 PM, Jascha Narveson <jnarveson at wesleyan.edu <mailto:jnarveson at wesleyan.edu>> wrote:
> >
> >
> > Chuck list! Help!
> >
> > I have the following idea and I’d like to hear what it sounds like:
> >
> > - play a sound into an FFT
> > - as the FFT runs, for each bin:
> > - - look at the magnitude info
> > - - take a small fraction of this info and store it in a running total
> > - - let the phase pass through unchanged
> > - use the spectral data in the running total for the IFFT
> >
> > What I was hoping for was a slow emerging smear of all the frequencies from the input sound, kind of like a weird version of some kind of “infinite sustain” reverb.
> >
> > What I’m getting is the attacks of the original sound coming through, although fading in, and covered with noise.
> >
> > I think I’m doing something wrong by not doing something with the phase, but I’m not sure what.
> >
> > I’d love some advice…
> >
> > cheers,
> >
> > j
> >
> >
> > - - - - c o d e  e x a m p l e - - - -
> >
> > SinOsc synth => FFT fft => blackhole;
> > UAnaBlob blob;
> > IFFT ifft => dac;
> >
> > synth.freq(440);
> > synth.gain(0.1);
> >
> > // set parameters
> > 2048 => fft.size;
> >
> > // place to hold the FFT data as it comes in
> > complex spec[fft.size()/2];
> >
> > // place to store the accumulated FFT data, preload with zeroes
> > complex acc[fft.size()/2];
> > for(0 => int i; i < acc.cap(); i++) {
> >  #(0, 0) => acc[i];
> > }
> >
> > // do it!
> > spork ~ sawTune();
> > spork ~ accumulateFrequenciesViaFFT();
> >
> > while (true) {
> >  1::second => now;
> > }
> >
> > fun void accumulateFrequenciesViaFFT() {
> >  while( true )
> >  {
> >      fft.upchuck() @=> blob;
> >      // get the data
> >      blob.cvals() @=> spec;
> >
> >      // for every bin...
> >      for(0 => int i; i < spec.cap(); i++) {
> >        // get the mag/phase for each bin by converting to polar
> >        spec[i] $ polar => polar newBin;
> >
> >        // and get our running totals from acc
> >        acc[i] $ polar => polar accBin;
> >
> >        // scale the inocming mag and add it to acc
> >        newBin.mag * 0.001 +=> accBin.mag;
> >
> >        // let phase pass through
> >        newBin.phase => accBin.phase;
> >                               /*Math.random2f(0, TWO_PI) => accBin.phase;*/
> >
> >        // convert back to complex and put back in the acc array
> >        accBin $ complex => acc[i];
> >      }
> >
> >      ifft.transform(acc);
> >      fft.size()::samp => now;
> >  }
> > }
> >
> > fun void sawTune() {
> >  while (true) {
> >    Math.random2(48, 72) => int midinote;
> >    Math.mtof(midinote) => synth.freq;
> >    333::ms => now;
> >  }
> > }
> >
> > _______________________________________________
> > chuck-users mailing list
> > chuck-users at lists.cs.princeton.edu <mailto:chuck-users at lists.cs.princeton.edu>
> > https://lists.cs.princeton.edu/mailman/listinfo/chuck-users <https://lists.cs.princeton.edu/mailman/listinfo/chuck-users>
> 
> _______________________________________________
> chuck-users mailing list
> chuck-users at lists.cs.princeton.edu <mailto:chuck-users at lists.cs.princeton.edu>
> https://lists.cs.princeton.edu/mailman/listinfo/chuck-users <https://lists.cs.princeton.edu/mailman/listinfo/chuck-users>
> 
> _______________________________________________
> chuck-users mailing list
> chuck-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/chuck-users/attachments/20171014/9a83a9aa/attachment-0001.html>


More information about the chuck-users mailing list