FMEngine is a utility library for training large foundation models. It is inspired by and built on top of many existing libraries and frameworks, such as DeepSpeed, torchtitan, torchtune.
FMEngine is primarily implemented and maintained by the Efficient Architecture and Systems Labs @ ETH Zurich.
We thank our friends for their generous support:
- @xzyaoi
- @LorrinWWW
- @fishiu: Scaling up Low Rank Adapters + Longer Contexts
- @Taishi-N324: Benchmarking and Scaling Experiments
- @chiennv2000: Mistral Integration