Introduction to Apple’s Multilingual, Multimodal Foundation Language Models

Introducing Apple’s Multilingual, Multimodal Foundation Language Models

At Apple, we are committed to pushing the boundaries of technology to enhance user experience across our diverse range of devices and services. Today, we are excited to introduce two groundbreaking multilingual, multimodal foundation language models that power Apple Intelligence features: a robust on-device model optimized for Apple silicon and a scalable server model built on cutting-edge technology.

Optimized On-Device Model

The first of our innovations is a compact, approximately 3 billion-parameter model designed specifically for on-device use. This model leverages architectural innovations such as KV-cache sharing and 2-bit quantization-aware training, allowing it to operate efficiently on Apple silicon. This on-device model ensures that features work seamlessly, even in environments with limited connectivity, providing a consistently high-quality user experience.

Scalable Server Model

In addition, we introduce our scalable server model, which employs a unique Parallel-Track Mixture-of-Experts (PT-MoE) transformer architecture. This novel construction combines track parallelism with sparse computation and interleaved global–local attention mechanisms, all designed to deliver exceptional quality while maintaining competitive costs. Hosted on Apple’s Private Cloud Compute platform, this model can efficiently handle complex tasks and support high-concurrency user requirements.

Training and Data Sources

Both models have undergone extensive training on large-scale multilingual and multimodal datasets. We have sourced our data responsibly through web crawling, licensed corpora, and high-quality synthetic data. To ensure a high standard of performance, we have further refined these models using supervised fine-tuning and reinforcement learning on an asynchronous platform. The result? Models that not only support an expanded range of languages but also understand images and execute tool calls effectively.

Performance Highlights

In public benchmarks and human evaluations, both the on-device and server models have demonstrated performance that matches or surpasses comparably sized open baselines. This is a testament to the power and efficiency of our advanced machine learning techniques, reaffirming Apple’s position at the forefront of technology innovation.

Swift-Centric Framework for Developers

To empower developers, we have introduced a new Swift-centric Foundation Models framework that exposes capabilities for guided generation, constrained tool calling, and LoRA adapter fine-tuning. This intuitive framework allows developers to seamlessly integrate advanced AI functionalities into their applications with just a few lines of code, facilitating a smooth development process and rapid deployment.

Commitment to Responsible AI

Our advancements in Apple Intelligence are grounded in a commitment to Responsible AI. We prioritize user safety with mechanisms like content filtering and locale-specific evaluations. Furthermore, innovations such as Private Cloud Compute reinforce our dedication to protecting user privacy, ensuring that sensitive data remains secure and confidential.

Conclusion

The introduction of our multilingual, multimodal foundation language models marks a significant milestone in our ongoing journey to enhance user experience across Apple devices and services. By combining sophisticated technology with a commitment to responsible practices, we aim to revolutionize how users interact with AI.

Stay tuned for further updates as we continue to refine our models and push the limits of what’s possible in AI. For a deeper dive into the technical details of these foundational language models, be sure to check out our comprehensive report released on June 9, 2025.