Google has announced that Gemini 1.5 Pro will be available in 180+ countries via the Gemini API in a public preview, with a first-ever native audio (speech) understanding capability and a new File API to make it easy to handle files.
Google is also launching new features, like system instructions and JSON mode, to give developers more control over the model’s output. In addition, Google is releasing its next-generation text embedding model that outperforms comparable models.
Also Read: What Is Google Vids? All About Fourth Big Productivity App For Workspace
Google is also expanding the input modalities for Gemini 1.5 Pro to include audio (speech) understanding in both the Gemini API and Google AI Studio. Additionally, Gemini 1.5 Pro is now able to reason across both image (frames) and audio (speech) for videos uploaded in Google AI Studio, and Google might add API support for this in the future.
What are Gemini API Improvements?
Google is addressing the requests by top developers by giving out below mentioned Gemini API improvements:
1. System instructions: Guide the model’s responses with system instructions, now available in Google AI Studio and the Gemini API. Define roles, formats, goals, and rules to steer the model’s behaviour for your specific use case.
2. JSON mode: Instruct the model to only output JSON objects. This mode enables structured data extraction from text or images. You can get started with cURL, and Python SDK support is coming soon.
3. Improvements to function calling: You can now select modes to limit the model’s outputs, improving reliability. Choose text, function call, or just the function itself.
Also Read: Facebook Meta AI Tool Fails to Create Accurate Images like Google Gemini
As per the official blog, developers can access Google’s next-generation text embedding model via the Gemini API. The new model, text-embedding-004, (text-embedding-preview-0409 in Vertex AI), achieves a stronger retrieval performance and outperforms existing models with comparable dimensions, on the MTEB benchmarks. Here is a screenshot report of the Gecko model.
These are the first set of improvements to Gemini 1.5 pro, Google promises to come up with many more improvements as the technology advances.
Also Read: What is Gemini Nano AI Feature Drop in the Google Pixel 8?
