FR: Probability output of mlpclassifier

[rodrigo.constanzo]

Bit of a late-in-the-game FR for adding probability output to fluid.mlpclassifier~.

Basically the predict_proba from scikit-learn (link).

I don’t know if this is, essentially, what @a.harker did for his custom build of fluid.mlpclassifier~ as discussed in this thread when he said:

@a.harker

When I briefly tried I was unable to get what I needed out of the regressor from the same data, and even then I’d need to recode the hotest output finder (but I didn’t try that hard), so yes what I have in my custom version outputs two further values (I forget what they both are) that are designed to give some indication of confidence. I think those values would be derivable from the regressor but it was easier to add them to the output for what I needed.

But having a flag to (somehow) output this information would be useful for all sorts of things (cascaded classifiers, or double-checking onset detection etc…).

I imagine that the value is being computed internally, but it would be a matter of adding an attribute and/or way to output it.

[tremblap]

there is already a thread where I show how to retrieve that by loading the model in the mlpregressor… check it out

[tremblap]

In case you didn’t find it: Using MLPregressor to get the confidence vector of MLPclassifier

[rodrigo.constanzo]

Very handy.

I (super) vaguely remember that.

Sadly the syntax has changed, and not knowing exactly what it should be doing, it’s not clear what goes where.

[tremblap]

quickly because on the road:

1. train your classifier
2. dump
3. extract the “mlp” key
4. load that in a regressor

done!

[rodrigo.constanzo]

Also, would be handy to have an actual probability vector where the values sum up to 1.

For my intended purposes the workaround is useful, but having an actual probability vector would be handy, as an FR. (I can add it as an issue to git if it’s easier to maintain a history there).

Why You Should Use Cross-Entropy Error Instead Of Classification Error Or...

[tremblap]

it is a little more complicated than that. There is a thread with @danieleghisi and @weefuzzy about providing (eventually) a tool to get softMax on arrays, maybe buffer-style à la FluCoMa

[weefuzzy]

Just to chime in and say

1. This stuff will eventually happen.
2. This thread is in danger of conflating some different stuff. tl;dr be wary of too much belief that something that looks like a probability or a distribution can be reliably treated as such. There be dragons, as ever.
3. If you’re on Max 9, then you can use arrays sum the contents of a buffer to 1 relatively painlessly. Untested: