# Memory hole problem?

**URL:** <https://discourse.flucoma.org/t/memory-hole-problem/226>\
**Category:** Bug Reports and Issues\
**Created:** [June 9, 2019, 7:40pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226 "2019-06-09T19:40:45Z")\
**Posts on this page:** 15\
**Page:** 2

<div class="post-metadata">

**Author:** ![tutschku](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/tutschku/32/22_2.png) [@tutschku](https://discourse.flucoma.org/u/tutschku)\
**Post date:** [July 11, 2020, 2:45am UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/22 "2020-07-11T02:45:52Z")

</div>

Continuing to work on the ‘over-night’ analysis engine and made some interesting discovery today. As mentioned previously, I’m running several instances of the patch, as all the poly~ and parallel processing did not lead to faster calculations.  
My setup for the past weeks was: copying the Max application several times and run 4 or 8 Max versions. Each of the apps loads one copy of the patch and I give each a portion of the large sound folder to analyze.  
What I noticed was that memory creeped up considerably and the initial quick calculation would slow down to a point that one single short sound file would take 15-20 seconds to complete. The same file would analyze in less than one second if Max was just freshly started.

SOLUTION: I packed the entire analysis portion of the patch into an abstraction. With scripting I’m tossing it after each analyzed subfolder and re-instantiate it before the next folder gets loaded. Memory behaves much better. Calculation time went down from a full night to about 2 hours.

---

<div class="post-metadata">

**Author:** ![tremblap](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/tremblap/32/5_2.png) [@tremblap](https://discourse.flucoma.org/u/tremblap)\
**Post date:** [July 11, 2020, 8:58am UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/23 "2020-07-11T08:58:09Z")

</div>

@weefuzzy is it linked to what we were talking about yesterday?

---

<div class="post-metadata">

**Author:** ![weefuzzy](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/weefuzzy/32/362_2.png) [@weefuzzy](https://discourse.flucoma.org/u/weefuzzy)\
**Post date:** [July 12, 2020, 11:32am UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/24 "2020-07-12T11:32:13Z")

</div>

I’m afriad not (unless I’ve missiagnosed that particular problem).

Probably the only way to account for @tutschku’s issue here is to do runs with his whole patch on data sets of a comprable size, using a build with debug info so that once performance has started to degrade we can attach a debugger and actually see what’s happening. When I last looked at this, [here](https://discourse.flucoma.org/t/max-poly-and-parallel-processing/505/33), I wasn’t able to reproduce on a smaller set of data, which I why I think we’ll need to go all in.

---

<div class="post-metadata">

**Author:** ![tutschku](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/tutschku/32/22_2.png) [@tutschku](https://discourse.flucoma.org/u/tutschku)\
**Post date:** [July 12, 2020, 8:52pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/25 "2020-07-12T20:52:41Z")

</div>

Here is the analysis portion of my patch.  
[https://drive.google.com/file/d/1XvoBcb1JvEuGjtxpSa4FNI-dxB\_Ii1r1/view?usp=sharing](https://drive.google.com/file/d/1XvoBcb1JvEuGjtxpSa4FNI-dxB_Ii1r1/view?usp=sharing)

Hope you get it to work as I needed to extract it from my larger work environment.

I also made a short video to explain.  
https://player.vimeo.com/video/437651576

---

<div class="post-metadata">

**Author:** ![Gabriel](https://discourse.flucoma.org/letter_avatar_proxy/v4/letter/g/9de0a6/32.png) [@Gabriel](https://discourse.flucoma.org/u/Gabriel)\
**Post date:** [January 26, 2022, 3:44pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/26 "2022-01-26T15:44:14Z")

</div>

Hello, I’m bumping this discussion because I recently experienced something similar to what happens in this thread.

When analyzing a large buffer with [fluid.bufnoveltyslice~], the object seems to take a big chunk of memory and not releasing it, even when the analyzing process is done. I did a little testing patch, here’s an example of what happens:

- I open the testing patch / Max takes 172 MB of memory
- I load 337 MB of audio in a buffer / Max takes 860 MB of memory
- I analyze the buffer with [fluid.bufnoveltyslice~] / Max takes 3652 MB of memory

Here’s my little chunk of code:

```

----------begin_max5_patcher----------
613.3oc0VssiaCBD8YGo7Of3Y2UwFma6S4+XU0JrMNgUDvBiSSzpte6EXbRb
2j33F4tU8EiXXFl4b7bg2GOJ.mp1ypvnmQufBBd2JIvKyII3nf.7V59LAsxq
HNs1XTRbXyYpZifYLGJYv0fSox0Xz2OdNO2akJ8sus3jQkTS1Ftb8qZVlArK
dwSSBQKl69Fmb7646QVukKsdxGDQmuHMcKyvzuxjzTgOFlzxDH5.a7R+43Qt
U6RXuQrj8Ca3eJ3Mr8F.SkLYNmJTqQEJQNSeaRAG1EwDcehIZVreYRjmZhuO
076nOdnP+NpVZob+QEhZd9SYJYF0Pqy4pBtvlNcAQcc8PqxYUFtjZ3JIJSoK
qqtMClyyb5Q0GtNGlbeNLlrDVHOFGN7YP.wblR3cTWccXS5ArgblkIe0nlZL
5Z9eJhl2IjlOyihkP0vzIWhImW8WlYilUswVXh+Zf6i2XLYdO9KBP9+jVio0
EEL8GnJAOiUp3RSGk1EBE0zcGxjocwPjo9JZBIA5QN8ebOxOUeaICoZGSXN3
oiOPqpT05LVSOOzJtcLRlsgXuXKOIE1h0tAi0iFhMDFI9AaHR9qk2buoA8Hk
gzMALizZjvhAJiArFK3xKdUkOFbG7IZAREZfUyqjPsBiVyHaoTDnzs4+d6NW
6166OW1Tas1xyaxS85X4uoPdjaY9oMCRD55N1+Hb.bX+XjAycQ8wcjq5tl7M
ZY4NltpwBvS15s2T9AgKBg8bIrOF1qY63GMARuwTssLwXqQp0vC71OqoHBuU
YeeqzNMuYbgCqNW6KocOHrpjBnxW5OdjUgeAm241gC
-----------end_max5_patcher-----------

```

---

<div class="post-metadata">

**Author:** ![DominicThibault](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/dominicthibault/32/1239_2.png) [@DominicThibault](https://discourse.flucoma.org/u/DominicThibault)\
**Post date:** [January 26, 2022, 4:01pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/27 "2022-01-26T16:01:52Z")

</div>

@weefuzzy @jamesbradbury @tremblap  
FYI, @Gabriel is on Windows 10.  
I can reproduce the problem on an Apple M1 Max running Monterey.

Dom

---

<div class="post-metadata">

**Author:** ![tremblap](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/tremblap/32/5_2.png) [@tremblap](https://discourse.flucoma.org/u/tremblap)\
**Post date:** [January 26, 2022, 4:06pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/28 "2022-01-26T16:06:26Z")

</div>

Can you reproduce with the new nightlies? If so I’ll investigate more.

---

<div class="post-metadata">

**Author:** ![jamesbradbury](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/jamesbradbury/32/1252_2.png) [@jamesbradbury](https://discourse.flucoma.org/u/jamesbradbury)\
**Post date:** [January 26, 2022, 4:20pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/29 "2022-01-26T16:20:32Z")

</div>

The nightlies @tremblap talks about are [here](https://github.com/flucoma/flucoma-max/releases/tag/nightly) @DominicThibault

---

<div class="post-metadata">

**Author:** ![Gabriel](https://discourse.flucoma.org/letter_avatar_proxy/v4/letter/g/9de0a6/32.png) [@Gabriel](https://discourse.flucoma.org/u/Gabriel)\
**Post date:** [January 26, 2022, 5:04pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/30 "2022-01-26T17:04:35Z")

</div>

I just reproduced with the new nightlies on Windows 10 and I get the same results:

- I open the testing patch / Max takes 185 MB of memory
- I load 337 MB of audio in a buffer / Max takes 866 MB of memory
- I analyze the buffer with [fluid.bufnoveltyslice~] / Max takes 3566 MB of memory

---

<div class="post-metadata">

**Author:** ![jamesbradbury](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/jamesbradbury/32/1252_2.png) [@jamesbradbury](https://discourse.flucoma.org/u/jamesbradbury)\
**Post date:** [January 26, 2022, 6:31pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/31 "2022-01-26T18:31:44Z")

</div>

What happens if you repeat those steps and then:

- Delete the buffer from your patch

or

- Send it the the message `clear, size 11`

?

---

<div class="post-metadata">

**Author:** ![Gabriel](https://discourse.flucoma.org/letter_avatar_proxy/v4/letter/g/9de0a6/32.png) [@Gabriel](https://discourse.flucoma.org/u/Gabriel)\
**Post date:** [January 26, 2022, 6:52pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/32 "2022-01-26T18:52:40Z")

</div>

Ok so this time once the audio files are loaded into a buffer and [fluid.bufnoveltyslice~] has done the analysis on it, Max is taking 3554 MB of memory.

If I delete the audio files and/or slice points buffers objects, Max is still taking 3554 MB of memory.

If instead of deleting the buffers I send a “clear, size 11” message I get:

- Send “clear, size 11” message to audio files buffer : Max takes 2883 MB of memory
- and then, send “clear, size 11” message to slice points buffer : Max takes 2883 MB of memory

I also tried:

- Send “clear, size 11” message to slice points buffer : Max takes 3554 MB of memory
- and then, send “clear, size 11” message to audio files buffer : Max takes 2883 MB of memory

---

<div class="post-metadata">

**Author:** ![weefuzzy](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/weefuzzy/32/362_2.png) [@weefuzzy](https://discourse.flucoma.org/u/weefuzzy)\
**Post date:** [January 26, 2022, 6:53pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/33 "2022-01-26T18:53:43Z")

</div>

Hi @Gabriel

Thanks for the simple patch to look at this with. I’ve had a nose about in Visual Studio to see what’s going on memory wise. Happy to say that it’s not a leak; however, you will see the memory grow markedly, and quite possibly descend again although it still holds on to a bunch. This is because the internal buffers used by the `fluid.buf~` objects aren’t junked after the first run, but kept alive for subsequent invocations, which is useful if hitting them often.

Medium-term I think I can reduce some of this memory usage, because it does seem excessive.

---

<div class="post-metadata">

**Author:** ![jamesbradbury](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/jamesbradbury/32/1252_2.png) [@jamesbradbury](https://discourse.flucoma.org/u/jamesbradbury)\
**Post date:** [January 26, 2022, 6:55pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/34 "2022-01-26T18:55:17Z")

</div>

Just to clarify this is explicitly a windows-centric issue? I can’t seem to reproduce on my macOS end.

---

<div class="post-metadata">

**Author:** ![weefuzzy](https://discourse.flucoma.org/user_avatar/discourse.flucoma.org/weefuzzy/32/362_2.png) [@weefuzzy](https://discourse.flucoma.org/u/weefuzzy)\
**Post date:** [January 26, 2022, 6:58pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/35 "2022-01-26T18:58:26Z")

</div>

No, it’s everything, but the OSes might differ in how much memory gets returned from Max to the OS once it’s been freed. In both cases though, there will be a marked net increase.

---

<div class="post-metadata">

**Author:** ![Gabriel](https://discourse.flucoma.org/letter_avatar_proxy/v4/letter/g/9de0a6/32.png) [@Gabriel](https://discourse.flucoma.org/u/Gabriel)\
**Post date:** [January 26, 2022, 7:22pm UTC](https://discourse.flucoma.org/t/memory-hole-problem/226/36 "2022-01-26T19:22:44Z")

</div>

Ok great, that make sense.

I just tried something else that feels like good news too (at least regarding what I want to do). I’ve try loading almost 2 GB of audio instead of 337 MB just to see how it reacts. After the analyzing process (and a huge peak of memory), Max is taking 4084 MB (only 500 MB more) and everything still feels very sharp.

So I guess that somehow, these objects are managing to keep their memory usage reasonable.

[Previous page](https://discourse.flucoma.org/t/memory-hole-problem/226.md?page=1)