Anyone paying attention to Google’s big AI announcement on Wednesday may have been left with two main questions: how is it different from ChatGPT, and is it any good?
What happened: Google introduced Gemini, the large language model that will power AI applications across its services and products.
What it is: Gemini is actually a family of three different models: flagship model Ultra, general purpose model Pro, and slimmed-down Nano, meant to run on mobile devices.
- Google claimed Ultra beat other state-of-the-art systems on nearly every academic benchmark, but its results were only slightly better than ChatGPT-4 on math, reading comprehension, image recognition and reasoning.
- Nano being contained within a phone means it can use data that can’t be sent to an external server, like private info and encrypted messages.
Why it matters: Unlike OpenAI, Google has a whole family of products it can slot this model into, even before it turns it over to developers. Some of these products are dominant in their particular market, so Gemini could be the main AI model most people interact with.
- Google already has plans to roll Gemini into its Search, Workspace, Chrome, and Ads platforms.
Yes, but: We have to take Google’s word about Gemini’s performance, as it limited access prior to the reveal event and didn’t provide hands-on demos. The company also didn’t provide many details on how its model was different from ChatGPT or what data it was trained on.
- Gemini Ultra is what developers are really interested in, but that’s not going to be widely available until the new year.
- Some who have tried the Pro version in the Bard chatbot haven’t been impressed, reporting factual errors, out-of-date info, and easily circumvented safety guardrails. Others say the way demo videos were edited may be overstating Gemini’s capabilities.
Bottom line: There’s a feeling that Gemini was possibly rushed to catch up with OpenAI. A series of live events were cancelled and replaced with a virtual preview, which some reports claim was due to Gemini not handling certain prompts reliably.