OpenAI has rolled out real-time video capabilities for ChatGPT, a feature it initially showcased seven months ago. During a livestream event on Thursday, the company introduced Advanced Voice Mode with vision, enabling ChatGPT to respond in near real time when users point their phones at objects. This feature is part of the ChatGPT app and is accessible to subscribers of ChatGPT Plus, Team, or Pro.
In addition to recognizing physical objects, the Advanced Voice Mode can also interpret what is displayed on a user’s screen through screen-sharing. For instance, it can provide explanations for various settings or assist with math problems. To use this feature, users simply tap the voice icon next to the ChatGPT chat bar, followed by the video icon to initiate video mode. Screen sharing can be activated by selecting the appropriate option in a menu.
The rollout of this feature began on Thursday and is expected to be completed within a week, although not all users will have immediate access. Particularly, ChatGPT Enterprise and Edu subscribers will have to wait until January for their access, while a timeline for users in the EU, Switzerland, Iceland, Norway, and Liechtenstein remains undetermined.
During a recent appearance on CNN’s “60 Minutes,” OpenAI President Greg Brockman demonstrated the new mode with host Anderson Cooper, who tested ChatGPT’s ability to understand drawings of anatomical structures. While ChatGPT accurately identified certain elements, it did make an error during a geometry question, reminding viewers that it is still not infallible.
The launch of Advanced Voice Mode with vision faced several delays, attributed to OpenAI announcing the feature prematurely. Following earlier rollouts of the voice-only version, the company shifted focus to enhance both platforms and accessibility for users in the EU.
As competitors like Google and Meta develop similar functionalities, Google has recently offered its real-time video analysis feature to a select group of testers on Android. Additionally, OpenAI has introduced a festive “Santa Mode,” allowing users to interact with ChatGPT in Santa’s voice, further enriching the user experience during the holiday season.
This advancement marks a significant step forward in making AI more interactive and user-friendly, showcasing OpenAI’s commitment to continually enhancing its features and engaging with users creatively.
In summary, OpenAI’s Advanced Voice Mode with vision introduces groundbreaking capabilities in real-time interaction, showing promise for a future where AI can assist users in more dynamic and meaningful ways.