posts
Introducing FlowerLLM
Flower Labs and Cambridge’s CaMLSys lab trained a 1.3-billion-parameter language model using federated learning, surpassing the previous record set by Google DeepMind by more than a factor of three. This shows that LLM pre-training can scale across distributed, heterogeneous hardware and private data sources, opening the door to foundation model development that taps into previously unused compute and data.