The encoder–decoder approach was significantly faster than LLMs such as Microsoft’s Phi-3.5, which is a decoder-only model.

As demand grows for faster, more capable large language models (LLMs), researchers have introduced a new approach that ...

Microsoft reports that on Qualcomm’s Hexagon NPU, Mu achieves a 47% reduction in first-token latency and nearly five times faster decoding compared to decoder-only models of similar size.

The Mu small language model enables an AI agent to take action on hundreds of system settings. It’s now in preview for some Windows Insiders.