Microsoft has recently released a new small language model called Phi-3 mini, which is optimized for modern smartphones and offers performance comparable to OpenAI’s GPT-3.5. This updated version of Microsoft’s lightweight language model was trained with 3.3 billion tokens from larger and more advanced datasets compared to its predecessor, Phi-2, which was trained with 1.4 billion tokens.
The researchers tested Phi-3 mini on an iPhone 14 with an A16 Bionic chip, where it ran natively and offline at a speed of over 12 tokens per second. The overall performance of this model is comparable to larger models like Mixtral 8x7B and GPT-3.5. It uses a transformer decoder architecture that supports 4K text length and is based on a block structure similar to Meta’s Llama 2, supporting packages developed for Llama 2.
Phi-3 mini is designed for conversational chat formats and aligns with Microsoft’s values of robustness and security. In addition to Phi-3 mini, Microsoft has also trained two other models in the same family: Phi-3 medium with 14 billion parameters and Phi-3 small with 7 billion parameters, both trained with 4.8 billion tokens. The company is focused on providing efficient and secure language models for a variety of applications.
Microsoft’s latest language model innovation highlights the company’s commitment to advancing natural language processing technology and delivering cutting-edge solutions for businesses across various industries. With the growing demand for conversational AI systems, Microsoft’s Phi-3 family of models provides organizations with powerful tools to enhance customer interactions and streamline operations through intelligent automation.
The launch of Phi-3 mini represents an important milestone in the evolution of language models, as it demonstrates how advances in hardware technology can enable smaller but equally capable models that can run natively on smartphones without compromising performance or accuracy.
Overall, the release of Phi-3 mini underscores Microsoft’s ongoing efforts to push the boundaries of what is possible in natural language processing and position itself as a leader in this rapidly evolving field.
