4.5M views · 6.3K reactions | What are the mega projects of today that are pushing civil engineering to its limits? 樂 #engineering | Interesting Engineering | Facebook

In this presentation, I will discuss the design, implementation and engineering experience in building and deploying MegaScale, a production system for training large language models (LLMs) at the scale of more than 10,000 GPUs. Training LLMs at this scale brings unprecedented challenges to training efficiency and stability. Maintaining high efficiency throughout the training process (i.e., stability) is an important consideration in production given the long extent of LLM training jobs. Many ha

516K views · 6.7K reactions | ⚙️ Extreme Engineering Machines Building The Most Amazing Megastructures ⚙️ There’s something about watching a colossal machine at work that fills us with awe. But is there anything more satisfying than watching something of that scale complete an even more gargantuan task? Let’s check out some of the world’s most extreme engineering machines building amazing megastructures. | Be Amazed | Facebook