Flower AMA: Pre-training 7B LLMs made easy with Flower
Date:
On the 20th of November 2024, I presented our work on federated learning of large language models at the Flower AMA session. With the help of Prof. Nicholas D. Lane, Alexandru-Andrei Iacob and Preslav Aleksandrov, we discussed some critical aspects of pre-training language models and their relevancy to the federated approach.
Link to the Flower AMA session.