I have a feeling that the phase information may be important to keep around. I added a moving average filter into your code, and then added that average into the accumulation spectrum. I'm also including the phase information, and while it all sounds more "smeary", it is also introducing windowing artifacts (I think that's what those are). I would be interested in a way to clean those up. SinOsc synth => FFT fft => blackhole; UAnaBlob blob; IFFT ifft => dac; synth.freq(440); synth.gain(0.1); // set parameters 1024 => fft.size; Windowing.hamming(fft.size() / 2) => fft.window; Windowing.hamming(fft.size() / 2) => ifft.window; // place to hold the FFT data as it comes in complex spec[fft.size()/2]; // place to store the accumulated FFT data, preload with zeroes complex acc[fft.size()/2]; for(0 => int i; i < acc.cap(); i++) { #(0, 0) => acc[i]; } // do it! spork ~ sawTune(); spork ~ accumulateFrequenciesViaFFT(fft.size()/2); while (true) { 1::second => now; } fun void accumulateFrequenciesViaFFT(int HALF_FFT_SIZE) { 4 => int FILTER_SIZE; float movingAverageMag[FILTER_SIZE][HALF_FFT_SIZE]; float movingAveragePhase[FILTER_SIZE][HALF_FFT_SIZE]; while( true ) { fft.upchuck() @=> blob; // get the data blob.cvals() @=> spec; // for every bin... for(0 => int i; i < spec.cap(); i++) { // get the mag/phase for each bin by converting to polar spec[i] $ polar => polar newBin; // and get our running totals from acc acc[i] $ polar => polar accBin; // move old values down the line for (FILTER_SIZE - 1 => int j; j > 0; j--) { movingAverageMag[j - 1][i] => movingAverageMag[j][i]; movingAveragePhase[j - 1][i] => movingAveragePhase[j][i]; } // add in new value newBin.mag => movingAverageMag[0][i]; newBin.phase => movingAveragePhase[0][i]; // get sums for averaging float magSum, phaseSum; for (0 => int j; j < FILTER_SIZE; j++) { movingAverageMag[j][i] +=> magSum; movingAveragePhase[j][i] +=> phaseSum; } // average magSum/FILTER_SIZE => float magAvg; phaseSum/FILTER_SIZE => float phaseAvg; magAvg * 0.005 +=> accBin.mag; phaseAvg * 0.005 +=> accBin.phase; // let phase pass through //newBin.phase => accBin.phase; /*Math.random2f(0, 2*pi) => accBin.phase;*/ // convert back to complex and put back in the acc array accBin $ complex => acc[i]; } ifft.transform(acc); (fft.size() / 4)::samp => now; } } fun void sawTune() { while (true) { Math.random2(48, 92) => int midinote; Math.mtof(midinote) => synth.freq; 333::ms => now; } } On Sat, Oct 14, 2017 at 10:31 AM, Jascha Narveson wrote:

Here’s the same thing as my original questions, but with some polite windowing - same problems persist: I’m hearing attacks (why?) and it’s pretty noisy - I guess because letting magnitudes creep up across the spectrum I’m just ultimately aiming for a big loud saw wave.

So I guess I’m looking to do two things:

- avoid hearing attacks in the resynthesized sound from my “accumulated magnitudes” spectrum

- artfully avoid the inevitable sawtooth sound by changing my approach to something the follows the spirit of the original idea and not the actual idea.

Thoughts?

cheers,

- jascha

- - - - same thing, a bit better maybe - - - -

SinOsc synth => FFT fft => blackhole; UAnaBlob blob; IFFT ifft => dac;

synth.freq(440); synth.gain(0.5);

second / samp => float srate;

// set parameters 1024 => fft.size; Windowing.hamming(fft.size() / 2) => fft.window; Windowing.hamming(fft.size() / 2) => ifft.window;

// hold the FFT data as it comes in complex spec[fft.size()/2];

// store the accumulated FFT data complex acc[fft.size()/2];

// fill the accumulated complex spectral array with zeroes for(0 => int i; i < acc.cap(); i++) { #(0, 0) => acc[i]; }

// spork shreds spork ~ synthTune(); spork ~ accumulateMagnitudesViaFFT();

while (true) { 1::second => now; }

fun void accumulateMagnitudesViaFFT() { while( true ) { fft.upchuck() @=> blob; // get the data blob.cvals() @=> spec;

// for every bin... for(0 => int i; i < spec.cap(); i++) { // get the mag/phase for each bin by converting to polar spec[i] $ polar => polar newBin;

// and get our running totals from acc acc[i] $ polar => polar accBin;

// scale the inocming mag and add it to acc newBin.mag * 0.001 +=> accBin.mag;

// let phase pass through newBin.phase => accBin.phase;

// conver back to complex and put back in the acc array accBin $ complex => acc[i]; }

ifft.transform(acc); (fft.size() / 4)::samp => now; } }

fun void synthTune() { while (true) { Math.random2(48, 72) => int midinote; Math.mtof(midinote) => synth.freq; 333::ms => now; <<>>; // just to check - yep: they're getting bigger... } }

...

On Oct 14, 2017, at 12:26 PM, Jascha Narveson wrote:

Chuck list! Help!

I have the following idea and I’d like to hear what it sounds like:

- play a sound into an FFT - as the FFT runs, for each bin: - - look at the magnitude info - - take a small fraction of this info and store it in a running total - - let the phase pass through unchanged - use the spectral data in the running total for the IFFT

What I was hoping for was a slow emerging smear of all the frequencies from the input sound, kind of like a weird version of some kind of “infinite sustain” reverb.

What I’m getting is the attacks of the original sound coming through, although fading in, and covered with noise.

I think I’m doing something wrong by not doing something with the phase, but I’m not sure what.

I’d love some advice…

cheers,

j

- - - - c o d e e x a m p l e - - - -

SinOsc synth => FFT fft => blackhole; UAnaBlob blob; IFFT ifft => dac;

synth.freq(440); synth.gain(0.1);

// set parameters 2048 => fft.size;

// place to hold the FFT data as it comes in complex spec[fft.size()/2];

// place to store the accumulated FFT data, preload with zeroes complex acc[fft.size()/2]; for(0 => int i; i < acc.cap(); i++) { #(0, 0) => acc[i]; }

// do it! spork ~ sawTune(); spork ~ accumulateFrequenciesViaFFT();

while (true) { 1::second => now; }

fun void accumulateFrequenciesViaFFT() { while( true ) { fft.upchuck() @=> blob; // get the data blob.cvals() @=> spec;

// for every bin... for(0 => int i; i < spec.cap(); i++) { // get the mag/phase for each bin by converting to polar spec[i] $ polar => polar newBin;

// and get our running totals from acc acc[i] $ polar => polar accBin;

// scale the inocming mag and add it to acc newBin.mag * 0.001 +=> accBin.mag;

// let phase pass through newBin.phase => accBin.phase; /*Math.random2f(0, TWO_PI) => accBin.phase;*/

// convert back to complex and put back in the acc array accBin $ complex => acc[i]; }

ifft.transform(acc); fft.size()::samp => now; } }

fun void sawTune() { while (true) { Math.random2(48, 72) => int midinote; Math.mtof(midinote) => synth.freq; 333::ms => now; } }

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

Okay, a little closer: I’ve switched my approach to taking a running average of the mags and phases - it’s sounding better, but is plagued by clicking at a rate related to my hop size. I thought windowing might’ve smoothed that out - any further ideas? - - - - taking averages of mag and phase - - - - - SinOsc synth => FFT fft => blackhole; IFFT ifft => dac; synth.gain(1); synth.freq(430); // set parameters 16384 => fft.size; fft.size() / 2 => int HALF_FFT_SIZE; Windowing.hamming(fft.size() / 2) => fft.window; Windowing.hamming(fft.size() / 2) => ifft.window; 1 => int FRAME_COUNT; // for monitoring the bin for middle-C second / samp / fft.size() => float binWidth; Math.ceil(Math.mtof(60) / binWidth) $ int => int middleCBinNum; // place to hold the FFT data as it comes in complex spec[HALF_FFT_SIZE]; // place to store the averaged FFT data, preload with an analysis frame complex avg[HALF_FFT_SIZE]; // do it! spork ~ synthTune(); spork ~ averageFrequenciesViaFFT(HALF_FFT_SIZE / 4); while (true) { 1::second => now; } fun void averageFrequenciesViaFFT(int HOP_SIZE) { while( true ) { FRAME_COUNT++; 1.0 / FRAME_COUNT => float FRAME_RECIPROCAL; fft.upchuck(); fft.spectrum(spec); // for every bin... for(0 => int i; i < HALF_FFT_SIZE; i++) { // get the mag/phase for each bin by converting to polar spec[i] $ polar => polar newBin; // and get our running average from avg avg[i] $ polar => polar avgBin; ( (avgBin.mag * (FRAME_COUNT - 1)) + newBin.mag) * FRAME_RECIPROCAL => float newAvgMag; ( ( (avgBin.phase * (FRAME_COUNT - 1)) + newBin.phase) * FRAME_RECIPROCAL) % Math.TWO_PI => float newAvgPhase; polar newAvgBin; newAvgMag => newAvgBin.mag; newAvgPhase => newAvgBin.phase; // convert back to complex and put back in the acc array newAvgBin $ complex => avg[i]; } <<<"note 60's bin: live: ", spec[middleCBinNum] $ polar, "avg: ", avg[middleCBinNum] $ polar>>>; // how's midinote 60 doing? ifft.transform(avg); HOP_SIZE::samp => now; } } fun void synthTune() { while (true) { Math.random2(48, 92) => int midinote; Math.mtof(midinote) => synth.freq; 333::ms => now; } }

On Oct 14, 2017, at 2:10 PM, Jascha Narveson wrote:

Hi, Eric -

Interesting! This is definitely a step closer.

On my own, I also tried running my original code with a constant mag of 0.1 in every bin, but with the phase updating live, and voila: I was hearing a dirty and distant version of the original signal, attacks and all. So, you’re right about that: phase info is important, and can sound like the source even if the mags are all kept steady.

Your averaging idea seems like a good route to try - maybe what I’m after is more of an average of a series of successive FFT-freezes, or something…

…back to the drawing board…

Thanks!

-j-

...

On Oct 14, 2017, at 1:59 PM, Eric Heep mailto:ericheep@gmail.com> wrote:

I have a feeling that the phase information may be important to keep around.

I added a moving average filter into your code, and then added that average into the accumulation spectrum. I'm also including the phase information, and while it all sounds more "smeary", it is also introducing windowing artifacts (I think that's what those are).

I would be interested in a way to clean those up.

SinOsc synth => FFT fft => blackhole;

UAnaBlob blob;

IFFT ifft => dac;

synth.freq(440);

synth.gain(0.1);

// set parameters

1024 => fft.size;

Windowing.hamming(fft.size() / 2) => fft.window;

Windowing.hamming(fft.size() / 2) => ifft.window;

// place to hold the FFT data as it comes in

complex spec[fft.size()/2];

// place to store the accumulated FFT data, preload with zeroes

complex acc[fft.size()/2];

for(0 => int i; i < acc.cap(); i++) {

#(0, 0) => acc[i];

}

// do it!

spork ~ sawTune();

spork ~ accumulateFrequenciesViaFFT(fft.size()/2);

while (true) {

1::second => now;

}

fun void accumulateFrequenciesViaFFT(int HALF_FFT_SIZE) {

4 => int FILTER_SIZE;

float movingAverageMag[FILTER_SIZE][HALF_FFT_SIZE];

float movingAveragePhase[FILTER_SIZE][HALF_FFT_SIZE];

while( true )

{

fft.upchuck() @=> blob;

// get the data

blob.cvals() @=> spec;

// for every bin...

for(0 => int i; i < spec.cap(); i++) {

// get the mag/phase for each bin by converting to polar

spec[i] $ polar => polar newBin;

// and get our running totals from acc

acc[i] $ polar => polar accBin;

// move old values down the line

for (FILTER_SIZE - 1 => int j; j > 0; j--) {

movingAverageMag[j - 1][i] => movingAverageMag[j][i];

movingAveragePhase[j - 1][i] => movingAveragePhase[j][i];

}

// add in new value

newBin.mag => movingAverageMag[0][i];

newBin.phase => movingAveragePhase[0][i];

// get sums for averaging

float magSum, phaseSum;

for (0 => int j; j < FILTER_SIZE; j++) {

movingAverageMag[j][i] +=> magSum;

movingAveragePhase[j][i] +=> phaseSum;

}

// average

magSum/FILTER_SIZE => float magAvg;

phaseSum/FILTER_SIZE => float phaseAvg;

magAvg * 0.005 +=> accBin.mag;

phaseAvg * 0.005 +=> accBin.phase;

// let phase pass through

//newBin.phase => accBin.phase;

/*Math.random2f(0, 2*pi) => accBin.phase;*/

// convert back to complex and put back in the acc array

accBin $ complex => acc[i];

}

ifft.transform(acc);

(fft.size() / 4)::samp => now;

}

}

fun void sawTune() {

while (true) {

Math.random2(48, 92) => int midinote;

Math.mtof(midinote) => synth.freq;

333::ms => now;

}

}

On Sat, Oct 14, 2017 at 10:31 AM, Jascha Narveson mailto:jnarveson@wesleyan.edu> wrote:

Here’s the same thing as my original questions, but with some polite windowing - same problems persist: I’m hearing attacks (why?) and it’s pretty noisy - I guess because letting magnitudes creep up across the spectrum I’m just ultimately aiming for a big loud saw wave.

So I guess I’m looking to do two things:

- avoid hearing attacks in the resynthesized sound from my “accumulated magnitudes” spectrum

- artfully avoid the inevitable sawtooth sound by changing my approach to something the follows the spirit of the original idea and not the actual idea.

Thoughts?

cheers,

- jascha

- - - - same thing, a bit better maybe - - - -

SinOsc synth => FFT fft => blackhole; UAnaBlob blob; IFFT ifft => dac;

synth.freq(440); synth.gain(0.5);

second / samp => float srate;

// set parameters 1024 => fft.size; Windowing.hamming(fft.size() / 2) => fft.window; Windowing.hamming(fft.size() / 2) => ifft.window;

// hold the FFT data as it comes in complex spec[fft.size()/2];

// store the accumulated FFT data complex acc[fft.size()/2];

// fill the accumulated complex spectral array with zeroes for(0 => int i; i < acc.cap(); i++) { #(0, 0) => acc[i]; }

// spork shreds spork ~ synthTune(); spork ~ accumulateMagnitudesViaFFT();

while (true) { 1::second => now; }

fun void accumulateMagnitudesViaFFT() { while( true ) { fft.upchuck() @=> blob; // get the data blob.cvals() @=> spec;

// for every bin... for(0 => int i; i < spec.cap(); i++) { // get the mag/phase for each bin by converting to polar spec[i] $ polar => polar newBin;

// and get our running totals from acc acc[i] $ polar => polar accBin;

// scale the inocming mag and add it to acc newBin.mag * 0.001 +=> accBin.mag;

// let phase pass through newBin.phase => accBin.phase;

// conver back to complex and put back in the acc array accBin $ complex => acc[i]; }

ifft.transform(acc); (fft.size() / 4)::samp => now; } }

fun void synthTune() { while (true) { Math.random2(48, 72) => int midinote; Math.mtof(midinote) => synth.freq; 333::ms => now; <<>>; // just to check - yep: they're getting bigger... } }

...

On Oct 14, 2017, at 12:26 PM, Jascha Narveson mailto:jnarveson@wesleyan.edu> wrote:

Chuck list! Help!

I have the following idea and I’d like to hear what it sounds like:

- play a sound into an FFT - as the FFT runs, for each bin: - - look at the magnitude info - - take a small fraction of this info and store it in a running total - - let the phase pass through unchanged - use the spectral data in the running total for the IFFT

What I was hoping for was a slow emerging smear of all the frequencies from the input sound, kind of like a weird version of some kind of “infinite sustain” reverb.

What I’m getting is the attacks of the original sound coming through, although fading in, and covered with noise.

I think I’m doing something wrong by not doing something with the phase, but I’m not sure what.

I’d love some advice…

cheers,

j

- - - - c o d e e x a m p l e - - - -

SinOsc synth => FFT fft => blackhole; UAnaBlob blob; IFFT ifft => dac;

synth.freq(440); synth.gain(0.1);

// set parameters 2048 => fft.size;

// place to hold the FFT data as it comes in complex spec[fft.size()/2];

// place to store the accumulated FFT data, preload with zeroes complex acc[fft.size()/2]; for(0 => int i; i < acc.cap(); i++) { #(0, 0) => acc[i]; }

// do it! spork ~ sawTune(); spork ~ accumulateFrequenciesViaFFT();

while (true) { 1::second => now; }

fun void accumulateFrequenciesViaFFT() { while( true ) { fft.upchuck() @=> blob; // get the data blob.cvals() @=> spec;

// for every bin... for(0 => int i; i < spec.cap(); i++) { // get the mag/phase for each bin by converting to polar spec[i] $ polar => polar newBin;

// and get our running totals from acc acc[i] $ polar => polar accBin;

// scale the inocming mag and add it to acc newBin.mag * 0.001 +=> accBin.mag;

// let phase pass through newBin.phase => accBin.phase; /*Math.random2f(0, TWO_PI) => accBin.phase;*/

// convert back to complex and put back in the acc array accBin $ complex => acc[i]; }

ifft.transform(acc); fft.size()::samp => now; } }

fun void sawTune() { while (true) { Math.random2(48, 72) => int midinote; Math.mtof(midinote) => synth.freq; 333::ms => now; } }

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu mailto:chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu mailto:chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu mailto:chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users

_______________________________________________ chuck-users mailing list chuck-users@lists.cs.princeton.edu https://lists.cs.princeton.edu/mailman/listinfo/chuck-users