Distillation is the process where the outputs of a giant, smart model are used to train compact, fast, and cheap local networks (SLMs). Current Terms of Service for the largest APIs expressly forbid using their outputs to train competitors. Nadella opposes this barrier. Microsoft, despite its partnership with OpenAI, is actively developing its own small models (the Phi family). The ban on distillation prevents corporations from creating efficient Edge AI solutions. The dispute goes beyond engineering: who owns the knowledge generated by a machine? If monopolists maintain the distillation ban, the market will forever remain hostage to expensive cloud computing.
Source: Microsoft / X / AI Magazine
R&DMicrosoftDistillationComplianceSLM