NVIDIA has announced the release of new open-weight models in collaboration with OpenAI, optimized specifically for NVIDIA RTX AI PCs. These groundbreaking gpt-oss models are designed to enhance AI applications, enabling smarter and faster inference capabilities that range from web searches to in-depth research.
The newly launched models, gpt-oss-20b and gpt-oss-120b, are now accessible to millions of users. They leverage NVIDIA GPUs to provide impressive performance, capable of processing up to 256 tokens per second on the powerful GeForce RTX 5090 GPU. Jensen Huang, NVIDIA’s CEO, emphasized the importance of this release, noting that it underscores NVIDIA’s leadership in the AI sector and advances innovation in open-source software.
The gpt-oss models are both flexible and advanced, featuring chain-of-thought capabilities and adjustable reasoning effort levels utilizing a mixture-of-experts architecture. This makes them particularly effective for a range of tasks, including coding assistance, document comprehension, and comprehensive web search functions. With the capacity to manage context lengths of up to 131,072, these models can reason through complex context-dependent problems, setting a new standard in local inference.
AI enthusiasts and developers can easily integrate these open-weight models using the Ollama app, which is designed to work effortlessly on RTX AI PCs equipped with at least 24GB of VRAM. The app simplifies the user experience, allowing for fast and efficient communication with the models without requiring extensive configurations.
Additionally, NVIDIA plans to continue its partnership with the open-source community, optimizing performance across various applications and frameworks. This ongoing collaboration signifies a commitment to enhancing the capabilities of RTX GPUs through the development of libraries like llama.cpp.
For those interested, the models can also be accessed via Microsoft AI Foundry Local, which is currently in public preview, providing another avenue for developers to incorporate the new models into their workflows.
The launch of these open-source models marks a significant advancement in AI technology, presenting exciting opportunities for developers and users alike to explore the potential of personal AI on their devices. With the ongoing support and community-driven innovations highlighted in NVIDIA’s RTX AI Garage blog series, the future of AI-powered applications on Windows is indeed bright.