Reduce memory consumption of the meta backup procedure

[polyrabbit]

What would you like to be added:

Refine code of meta backup procedure, so it won't load everything into memory and marshal all at once into json.

Why is this needed:

OOM happens when backing up ~20 million inodes in a 5GiB memory mount pod.

We can setup a dedicated large container to do the backup, but it's not a good solution - still hoping the code can do some streaming.

Version: 1.1
Meta Engine: redis

[SandyXSD]

Already fixed in the main branch by #4253 .

[polyrabbit]

Just did a quick test using juicefs dump. It seems the issue still exists:

And here is the heap profiling data: pprof.alloc_objects.alloc_space.inuse_objects.inuse_space.005.pb.gz

[davies]

@polyrabbit I can't reproduce his behavior with 10 millions of files generated by juicefs mdtest. Is there a huge directory with tens of millions of files in your case?