‘New ChatGPT with Listening, Viewing, and Speaking Capabilities Introduced by OpenAI’

OpenAI, an artificial intelligence startup based in San Francisco, has unveiled a new version of its ChatGPT chatbot that can now receive and respond to voice commands, images, and videos. The new app, based on the AI system GPT-4o, is said to handle audio, images, and videos significantly faster than previous versions of the technology. This app is available for free on both smartphones and desktop computers.

Mira Murati, the company’s chief technology officer, stated that this development represents a glimpse into the future of human-machine interaction. OpenAI’s new app is part of a wider effort to merge conversational chatbots like ChatGPT with voice assistants like Google Assistant and Apple’s Siri. As technology continues to advance, Google is integrating its Gemini chatbot with Google Assistant, while Apple is enhancing Siri to be more conversational.

The company plans to gradually share this technology with users over the coming weeks, marking the first time ChatGPT is available as a desktop application. Previously, OpenAI offered similar technologies through various free and paid products, but now they have consolidated them into one system accessible across all platforms.

During a recent online event, Murati and her team showcased the new app as it responded to voice commands, analyzed math problems through live video feed, and narrated stories it had written on the spot. Although the app cannot generate video content, it can create still images representing frames of a video.

OpenAI initially introduced ChatGPT in late 2022 to demonstrate how machines can handle requests in a more human-like manner. By analyzing vast amounts of text from the internet, including Wikipedia articles and chat logs, ChatGPT could answer questions, write papers, and generate code without relying on predefined rules. This approach, known as “multimodal AI,” merges chatbots with AI image, audio, and video generators.

While advancements in technology are promising, challenges persist. Chatbots, which learn from internet data, can sometimes provide inaccurate or made-up information. Known as “hallucination,” these flaws can transfer to voice assistants as well. Companies like OpenAI are actively working to enhance chatbots into reliable “AI agents” capable of performing tasks like scheduling meetings and booking flights.

The latest version of ChatGPT is built on a singular AI technology, GPT-4o, which processes text, sounds, and images efficiently. This streamlined approach reduces latency and enables OpenAI to offer the app for free to users. Murati emphasized the importance of creating a natural dialogue experience for users, which is now achievable with this integrated technology.

In conclusion, OpenAI is at the forefront of technology innovation, bridging the gap between chatbots and voice assistants to create a more seamless and interactive user experience.