Okay, all good now! I’ll document my final setup here, in case someone tries to do the same.
First, since I have the 4GB RAM version of the Pi 4, I needed to dial in some swap so that the compilation could work, otherwise it would produce the fatal error right around trying to compile fluid_libmanipulation (see above). I added 4 GB to swap, but looking back probably 1 GB could have been enough.
Then I did everything according to the flucoma-pd docs & compile guide.
And it worked without errors (once I had enough RAM/swap). But somehow Pd v0.55 did NOT like it. And so somewhat hastily I did not test what @weefuzzy suggested above (sorry!), but just wiped the build folder and tried again, adding the flag (ARM version) mentioned here, which is -DCMAKE_CXX_FLAGS=-mcpu=native. My final build command looked like this:
cmake -DPD_PATH="/home/dietpi/pure-data" -DDOCS=OFF -DCMAKE_CXX_FLAGS=-mcpu=native ..
Now comes the extra-messy part: earlier, as a 0th step I actually just naively tried the pre-compiled linux distribution of flucoma-pd from here for lolz which did not work hence I tried to compile it locally. So I still had that FluidCorpusManipulation folder with all the abstractions, docs, data, and the incompatible externals. I noticed that this folder structure is slightly different from the one I got from the cloned repo (and not knowing whether Pd would be annoyed by that), notably that the externals do not live in the top-level folder, but under the pd_objects folder. So I decided to just copy over the contents of this pd_objects folder to that above-mentioned FluidCorpusManipulation folder to overwrite those incompatible externals with the freshly compiled ones like this:
cp -a /home/dietpi/flucoma-pd-rpi/flucoma-pd/pd_objects/. /home/dietpi/FluidCorpusManipulation/
Finally, as a test I wanted to load the overview patch and see if everything gets loaded successfully. From my /home/dietpi user folder:
pd -stderr -nogui -verbose -audiodev 0 -midiindev 2 -midioutdev 2 -path "./FluidCorpusManipulation" -lib "fluid_libmanipulation" ./FluidCorpusManipulation/FluidCorpusManipulationOverview.pd
This worked, but I haven’t tested anything else yet. But I guess this is a very good sign! 
If you do all this like I did, don’t panic in the end (like I did not), when you load the patch and seemingly get lots of failure logs like so:
EDIT: reached the comment character limit with this one, so will try to post it in the next comment.
As I have definitely noticed on the first try, you too can notice that Pd tries to load various extensions for the externals in a certain order and will fail until it tries the .pd_linux extension, or just the .pd one in case of an abstraction.
Another notable thing is that after it successfully loaded fluid.plotter and fluid.waveform, in both cases it printed out what looks like the source code for those? Curious. I don’t plan to try using GUI stuff on the RPI, so I can’t verify if this is anything to be afraid of, but it’s documented now.
Thanks for the help, I am super curious to see the potential of Flucoma on a small computer like this.