posts
Photon: Federated Pre-training of Large Language Models
A blog write-up of Photon, an extension to the Flower framework that enables decentralized pre-training of large language models across multiple independent data centers. Photon reaches training speeds comparable to or faster than centralized approaches while cutting communication overhead by 64-512x, letting institutions collaboratively train billion-parameter models without sharing sensitive data.