# Using fluid.noveltyslice~ to find formal changes
**URL:** https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271
**Category:** Usage Questions
**Created:** [July 11, 2019, 12:04am UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271 "2019-07-11T00:04:12Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [July 11, 2019, 12:04am UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/1 "2019-07-11T00:04:12Z")
After seeing @weefuzzy’s talk at the plenary followed by some interesting discussion in [this thread](https://discourse.flucoma.org/t/difficult-segmentation/222/6), I wanted to take a stab and using `fluid.noveltyslice~` to segment live audio formally, rather than by events/onsets.
After playing with `@fftsettings` and `@kernelsize` (and getting some [hearty crashes](https://discourse.flucoma.org/t/fluid-noveltyslice-instacrash/270/3) along the way), I found that `@fftsettings 8192 128 -1` seemed to work ok, but was still reporting clearer onsets (as opposed to shifts in form) depending on the type of material used.
`@threshold` seemed to kind of be all-or-nothing as well, with a lot of activity around 0.2, and then \>0.4, getting nothing really.
(as an aside the helpfile/refence says that `@threshold` should be a value between 0-1, but the `@attribute` is not clamped at all. Is that intentional?)
Latency isn’t super critical with what I have in mind, but keeping a smallish `hopsize` would be good in that something along the lines of 300-500ms latency would be passable for my use case.
I vaguely remember from @weefuzzy’s talk, that he did a whole load of downsampling ([the bane of me at the moment!](https://discourse.flucoma.org/t/fluid-bufnmf-cv-splitter-processor/135/47)) and iteration to “roughly” and “vaguely” tell where changes in material happened. Is there a FluCoMa-able approximation of this idea/approach possible?
I’m attaching an example audio file to show the kinds of sound I’m working with, but I (musically) think there are around 3-4 types of material in here that I’d like to be able to differentiate between. I’ve mainly tested with `@feature 0`, as that seemed to work best in practice, although I thought `@feature 1` would be better for this kind of noisy material.
[turntable snare extract.mp3.zip](https://discourse.flucoma.org/uploads/short-url/6HAmGR3eeMsBYuMiRNyPBsI3xIS.zip) (936.0 KB)
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 11, 2019, 8:20am UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/2 "2019-07-11T08:20:54Z")
things to try to find larger chuncks:
- larger hop size
- downsampling
- autothresh (check the examples for an iterative way of doing that)
now, spectral novelty is very likely, at small resolution, to be attacks, so you need to blur time resolution and swallow them, hence proposal 1 and 2 above. as for thresh, it is context dependant, so autothresh is a good idea to explore
I’ll investigate the crashes of the other thread as we did not get them with our test material. Non-modulatable is easy to implement as it is everywhere else, so it might either be a real bug or an omission.
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [July 11, 2019, 8:29am UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/3 "2019-07-11T08:29:55Z")
> [@tremblap](#):
>
> autothresh (check the examples for an iterative way of doing that)
Where is this? I didn’t see anything on that in the helpfile(s) or /examples folder. (or do you mean the “filter size” tab in `fluid.bufnoveltyslice~`?)
> [@tremblap](#):
>
> larger hop size
Would that also impact the chunk size, and not just latency? I guess impacts how soon a new chunk can be assessed?
> [@tremblap](#):
>
> I’ll investigate the crashes of the other thread as we did not get them with our test material.
I didn’t even load material. I got those crashes just adjusting the params in the helpfile as-is. With audio off or on.
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 11, 2019, 8:37am UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/4 "2019-07-11T08:37:09Z")
> [@rodrigo.constanzo](#):
>
> Where is this? I
/examples/segmenting/nb\_of\_slices.maxpat
> [@rodrigo.constanzo](#):
>
> Would that also impact the chunk size, and not just latency?
yes, so the plan is for you to find with super large windows in large file where large changes happen. then, rerun on that chunk only with whatever other parameter.
for instance:
You could do an ampseg looking for minimum up time of 1sec and min silences of 500ms on a 10 minutes file. you get large chunks
then you take those and do fine tuning.
how sexy and flexible is that?
😉
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [July 11, 2019, 8:45am UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/5 "2019-07-11T08:45:10Z")
Oh yeah, I remember seeing that.
The problem is…
> [@tremblap](#):
>
> in large file
I want to do it on real-time audio.
Unless you mean using that process to figure out what a suitable thresh might be, but it still doesn’t help with the fft/kernel/filter sizes, which are probably the critical things here.
> [@tremblap](#):
>
> now, spectral novelty is very likely, at small resolution, to be attacks, so you need to blur time resolution and swallow them, hence proposal 1 and 2 above.
Oh, I forgot to ask about this in my previous response. Would that then mean I also want to look at different `@feature`s as well, particularly if downsampled?
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 11, 2019, 8:47am UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/6 "2019-07-11T08:47:56Z")
> [@rodrigo.constanzo](#):
>
> I want to do it on real-time audio.
I should have guessed. then you need to spy on your ears and brains: how do you actually segment in real-time? The answer is you don’t. you just remember and make sense of the past. So having some sort of past that is analysed bluntly to set automagically some threshold is something you can now, thanks to flucoma, explore 😉
> [@rodrigo.constanzo](#):
>
> Oh, I forgot to ask about this in my previous response. Would that then mean I also want to look at different `@feature` s as well, particularly if downsampled?
segmenting on amplitude change, and on MFCCs, would be the 2 I’d choose, for loudness and spectra, respectively.
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [July 11, 2019, 9:11am UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/7 "2019-07-11T09:11:46Z")
> [@tremblap](#):
>
> I should have guessed.
Hehe, hence the thread title. 😉
> [@tremblap](#):
>
> So having some sort of past that is analysed bluntly to set automagically some threshold is something you can now
By this do you mean keeping a running buffer and running ‘offline’ analysis (in lieu of the real-time version), JIT style? Or is that what having larger FFT sizes enables? Or downsampling? Or both?
> [@tremblap](#):
>
> segmenting on amplitude change, and on MFCCs, would be the 2 I’d choose, for loudness and spectra, respectively.
As in using either, or using two instances, one with each `@feature`?
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 11, 2019, 12:27pm UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/8 "2019-07-11T12:27:46Z")
> [@rodrigo.constanzo](#):
>
> By this do you mean keeping a running buffer
I imagine wanting to know about the past, and if I’m different enough to be elsewhere or back. That really depends on your definition of past and section, so you need to experiment. but let say I want to know every 10 seconds where I am, I’d analyse at that frequency and get a reading of the past 30 and see if there in one in the last 10… I have not done anything like this but this is all possible. So I go back to my hint:
> [@tremblap](#):
>
> you need to spy on your ears and brains
> [@rodrigo.constanzo](#):
>
> either,
either, in relation to the above.
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [July 11, 2019, 12:45pm UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/9 "2019-07-11T12:45:10Z")
Right, doing a running buffer offline reading and updating based on that (rather than having a massive 30s fft inside the realtime version).
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 11, 2019, 2:16pm UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/10 "2019-07-11T14:16:29Z")
or having a massive 30s in RT, but downsampled 32 times (let’s see if you could guess the fft size now 😉
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [July 11, 2019, 2:31pm UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/11 "2019-07-11T14:31:12Z")
Is it as simple of figuring out a 30s FFT size and then dividing it by 32?
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [July 11, 2019, 6:29pm UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/12 "2019-07-11T18:29:56Z")
Ok, got something kind of working.
I’m testing it with smaller fft sizes for now, since it’s quicker to hear the changes.
Also, what _units_ is `@filtersize` in? Samples, bins? More specifically, if I’m working at extreme downsampling, should I adjust my expectations with the `@filtersize` accordingly?
[novelty.zip](https://discourse.flucoma.org/uploads/short-url/A138QunkJfzBnbcBFsoKZkEPgjg.zip) (3.8 KB)
---
### Author:  [@tremblap](https://discourse.flucoma.org/u/tremblap)
#### Post date: [July 11, 2019, 7:37pm UTC](https://discourse.flucoma.org/t/using-fluid-noveltyslice-to-find-formal-changes/271/13 "2019-07-11T19:37:58Z")
IIRC: kernel size is the window of time (in hops) that the difference is computed
filtersize is in hop too, and is a smoothing of the novelty curve (because @weefuzzy said famously that it is jittery as \*\*\*)