# Making sense of fluid.mlpregressor~s autoencoder
**URL:** https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840
**Category:** Pre-Release Toolbox2 Usage
**Created:** [March 17, 2021, 11:53pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840 "2021-03-17T23:53:09Z")
**Posts on this page:** 14
**Page:** 2
### Author:  [@jamesbradbury](https://discourse.flucoma.org/u/jamesbradbury)
#### Post date: [May 20, 2021, 4:46pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/21 "2021-05-20T16:46:24Z")
That is a cute dog đ
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [May 22, 2021, 8:41pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/22 "2021-05-22T20:41:06Z")
> [@weefuzzy](#):
>
> Data augmentation is certainly worth a try, but not everything will pay dividends: changing the time-domain phase, for example, wonât do a great deal if youâre then just using Mel bands or MFCCs as your principle feature: basically, the key is to think of augmentations that make some sort of sense for the data the model is likely to encounter at run time. For your percussive hits, perhaps subtle time stretches would be useful, and maybe even small amounts of saturation.
I was thinking about this today as I was planning on recording a much more comprehensive set of âsounds I can make with my snareâ. Knowing that the corpus would be so specific to the snare, the head, the tuning, and the room (to a certain extent), and that it wouldnât necessarily translate if I went to a gig and used another snare, or even just had my head drift in tuning over time is a bit of a bummer.
So that led me down a couple paths of thinking.
- creating the minimum viable corpus for any given snare (maximum variety/dynamics, with a generous helping of data augmentation to fill in the gaps)
- create a monolithic corpus for each setup I have and just streamline that process
- thinking about the viability of having a mega-chunky-corpus, that is continuously fed new snares/setups/tunings and keeps getting bigger every time I use the system with a new drum
- if itâs somehow possible to train a NN on some kind of archetypical aspects of the sounds (within the word of âshort attacks on a snareâ), which is then made a bit more specific with samples of the exact snare in any given setup
Part of that last example was remembering the topology if the [machine learning snare thing](https://discourse.flucoma.org/t/reverse-engineering-real-time-mlmatching/121) that I was looking into a while back:
It could just be that this makes sense for the purposes of the [patent application](https://patentimages.storage.googleapis.com/1b/dc/9a/bc2924d1c0f461/US20160093278A1.pdf) but from the looks of it, the NN is trained on data that is distinct from the user generated and trained aspects. In fact, remember when I last used the software, you would go into a training mode, and give it around 50 hits of any given zone (âsnare centerâ, âsnare edgeâ, etcâŚ), and then come out of training mode and it worked immediately. There was never any computation that went along with it (unless it happened as you went and was super super super fast). You literally toggled in and out of training mode ala a classifier. But thereâs an NN involved somewhere/somehow. How?
---
### Author:  [@jamesbradbury](https://discourse.flucoma.org/u/jamesbradbury)
#### Post date: [May 22, 2021, 10:33pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/23 "2021-05-22T22:33:36Z")
> [@rodrigo.constanzo](#):
>
> and then come out of training mode and it worked immediately. There was never any computation that went along with it (unless it happened as you went and was super super super fast).
its likely its just super optimised for one specific purpose. No fucking around with memory allocation - its probably all pre-allocated for everything that is needed. Itâs certainly not implausible that it trains _fast enough_ with such small datasets too with the level of optimisation they could give toward a small use case
---
### Author:  [@weefuzzy](https://discourse.flucoma.org/u/weefuzzy)
#### Post date: [May 22, 2021, 10:35pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/24 "2021-05-22T22:35:52Z")
Itâs a patent, so purposefully quite vague on certain details â if you read the text, thereâs a lot of âmayâ going on. However, if one perseveres, then some general impression seems to come out.
For example, one of the things they âmayâ be doing is using a âSiameseâ network architecture (like @jamesbradbury posted the other day, and I can see him replying now) in such a way as to learn a distance function from labelled data which can then be applied to unlabelled data. I donât know how much that specific trick is still popular, but thereâs still a lot of active research into metric learning (i.e. learning a distance function) and transfer learning (training a general model first, and then making it â quickly â more specific with some extra examples).
---
### Author:  [@weefuzzy](https://discourse.flucoma.org/u/weefuzzy)
#### Post date: [May 22, 2021, 10:38pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/25 "2021-05-22T22:38:08Z")
It looks like some of the ongoing research with UMAP looks at these cases: [UMAP for Supervised Dimension Reduction and Metric Learning â umap 0.5 documentation](https://umap-learn.readthedocs.io/en/latest/supervised.html)
(but thatâs not in ours)
---
### Author:  [@jamesbradbury](https://discourse.flucoma.org/u/jamesbradbury)
#### Post date: [May 23, 2021, 12:51am UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/26 "2021-05-23T00:51:10Z")
Itâs referenced at the end of the umap article even so it must be cool still đ
>
---
### Author:  [@rodrigo.constanzo](https://discourse.flucoma.org/u/rodrigo.constanzo)
#### Post date: [May 23, 2021, 12:42pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/27 "2021-05-23T12:42:08Z")
> [@weefuzzy](#):
>
> For example, one of the things they âmayâ be doing is using a âSiameseâ network architecture (like @jamesbradbury posted the other day, and I can see him replying now) in such a way as to learn a distance function from labelled data which can then be applied to unlabelled data.
Most interesting.
Yeah, the way the system behaves is like a vanilla classification thing where you give it examples, and then thatâs it. So I was never sure how an NN fit into the equation.
> [@weefuzzy](#):
>
> metric learning (i.e. learning a distance function)
Iâm assuming itâs not nearly as simple as this, but would the idea be that (using my multiple snares example/context), that I could train a large set on given sounds, and then when presented with new variants, the overall âdistancesâ would hold up and still be relevant useful, by (I guess) somehow transposing/stretching the existing points?
> [@weefuzzy](#):
>
> transfer learning (training a general model first, and then making it â quickly â more specific with some extra examples)
Aha! This sounds more like what Iâm thinking.
Also, are both of these (metric/transfer) limited to classification or does the paradigm apply for regression as well?
---
### Author:  [@tedmoore](https://discourse.flucoma.org/u/tedmoore)
#### Post date: [May 30, 2021, 10:27am UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/28 "2021-05-30T10:27:15Z")
In response to @spluta:
> [@weefuzzy](#):
>
> Yes, in principle you should be able to use weights from a MLP model trained elsewhere, so long as you can jam them into the appropriate JSON format for our object (which isnât super documented yet, but certainly not at all impossible). Bear in mind that youâll need to limit yourself to the activation functions that we support.
[https://github.com/tedmoore/FluCoMa-stuff/blob/master/sklearn\_mlp\_to\_fluid\_mlp.py](https://github.com/tedmoore/FluCoMa-stuff/blob/master/sklearn_mlp_to_fluid_mlp.py)
Should still work if youâre looking at sklearn. If it doesnât, let me know and I can poke at it. Youâre probably looking at Keras though @spluta?
EDIT: Also note that in the output activation will always be identity. This is the default (and only?) setting for sklearnâs MLP.
---
### Author:  [@weefuzzy](https://discourse.flucoma.org/u/weefuzzy)
#### Post date: [May 30, 2021, 11:30am UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/29 "2021-05-30T11:30:03Z")
> [@tedmoore](#):
>
> ```auto
> if mlp.activation == 'identity':
> activation = 0
> elif mlp.activation == 'logistic':
>
> ```
I donât think thatâs quite right, because `relu` and `tanh` wonât ever get used, even though theyâre valid. Perhaps something like
```python
activation_map = {'identity':0, 'logistic':1, 'relu':2, 'tanh':3}
acvtivation = activation_map[mlp.activation]
```
?
---
### Author:  [@tedmoore](https://discourse.flucoma.org/u/tedmoore)
#### Post date: [May 30, 2021, 11:37am UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/30 "2021-05-30T11:37:10Z")
Can one not string together multiple elifs in Python? Regardless your implementation is much more elegant. Will edit.
---
### Author:  [@weefuzzy](https://discourse.flucoma.org/u/weefuzzy)
#### Post date: [May 30, 2021, 11:49am UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/31 "2021-05-30T11:49:43Z")
You can; sorry, I hadnât noticed that the preview was truncated. I am, however, allergic to big `if` trees đ
---
### Author:  [@tedmoore](https://discourse.flucoma.org/u/tedmoore)
#### Post date: [May 30, 2021, 11:54am UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/32 "2021-05-30T11:54:48Z")
That is a good allergy to develop. đ¤§
---
### Author:  [@spluta](https://discourse.flucoma.org/u/spluta)
#### Post date: [May 30, 2021, 2:07pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/33 "2021-05-30T14:07:37Z")
This is awesome! No. I was looking sklearn. Thanks for this.
Sam
---
### Author:  [@jamesbradbury](https://discourse.flucoma.org/u/jamesbradbury)
#### Post date: [May 30, 2021, 3:09pm UTC](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840/34 "2021-05-30T15:09:56Z")
The better way to do this is to create a class for each activation with its own custom type. You can then use isinstance() to check which type of class it is.
/s đ
[Previous page](https://discourse.flucoma.org/t/making-sense-of-fluid-mlpregressor-s-autoencoder/840.md?page=1)