Using FSDP

[Yeon07]

Hi, thanks for sharing the amazing work.

I was wondering if you could provide more details on how to train OSM with FSDP.
I set args.fsdp to True and get the following error:
AttributeError: 'OmniScientModel' object has no attribute 'wrap_fsdp'

Are there more configurations to be set?

Thanks in advance!

[cornettoyu]

Hi,

This code-base is built upon open-flamingo and I believe FSDP may not be fully tested in current code-base, you may refer to their implementations for more details regarding training with FSDP.

Best,