The original post: /r/localllama by /u/FullstackSensei on 2025-01-27 21:13:50.

From the article: "Of the four war rooms Meta has created to respond to DeepSeek’s potential breakthrough, two teams will try to decipher how High-Flyer lowered the cost of training and running DeepSeek with the goal of using those tactics for Llama, the outlet reported citing one anonymous Meta employee.

Among the remaining two teams, one will try to find out which data DeepSeek used to train its model, and the other will consider how Llama can restructure its models based on attributes of the DeepSeek models, The Information reported."

I am actually excited by this. If Meta can figure it out, it means Llama 4 or 4.x will be substantially better. Hopefully we’ll get a 70B dense model that’s on part with DeepSeek.