Philipp Schmid's Recent LinkedIn Posts

One of the most underrated features of Gemini is that i can ace minutes/hour of video understanding in seconds! Below is an example of how to analyze Youtube Videos with a single API call using the Gemini Interactions API! Give it a try! You will be surprised how much progress we made. Docs: https://lnkd.in/dhNzRmGB

Gemma 4 is here! 4️⃣ Our most capable, agentic open model, built on the same research as Gemini 3. Reasoning. Multimodal. Four sizes (2B to 31B). Base + Instruct. ✨ Released under Apache 2.0. Runs on your phone, laptop, or servers. All you need to know about Gemma 4: 4️⃣ 4 sizes (E2B, E4B, 26B4A, 31B) 🪟 Up to 256K context window 🛠️ Native function-calling, structured JSON output 👁️ + audio on edge models (E2B/E4B) 🌍 Trained on 140+ languages 🏆 31B ranks #3 open model on Arena AI 🪪 Apache 2.0 license 1️⃣ Fits on a single GPU 🚀 Gemma E4B == Gemma 3 27B All versions support native function-calling and structured JSON output to build agents that can run locally. The small models (E2B, E4B) can run entirely offline on mobile supporting vision, audio, everything on-device. Start building with Gemma 4 now. Try in Google AI Studio → https://lnkd.in/ddQhAsCs Hugging Face → https://lnkd.in/dhNBspd5 Kaggle → https://lnkd.in/dffxshtG or in your favorite ecosystem tool! Blog → https://lnkd.in/d_EXTGCn

Veo 3.1 and Nano Banana models are now available through the OpenAI compatibility layer, no SDK swap needed. 🎬 Generate videos via `/v1/videos` using Veo 3.1 🍌 Generate images with Nano Banana via `images.generate` 🔌 Drop-in compatible with OpenAI Python and JS SDKs 🔀 Switch to Gemini API by changing just 3 lines of code Get Started: https://lnkd.in/d7uQjJFY

We just published a blog on how we built the Gemini API skill. LLMs have fixed knowledge cutoffs, so we need to teach them about our newest models and how to use the SDK. In our evaluations, it helped Gemini 3.1 Pro pass 95% of 117 eval tests. Blog: https://lnkd.in/dztS-RWV Skill: https://lnkd.in/d6gDcXsW

What if you could optimize a model overnight without any ML experience? What if an AI agent runs hundreds of training experiments autonomously, keeping only the improvements? That is the idea behind autoresearch from Andrej Karpathy.    Yes, the early results are small scale, GPT-2 speedups, a 0.8B model beating a 1.6B. But the unlock is real:  You have a domain task (search ranking, fraud scoring, clinical NER). You have labeled data and an eval metric. You hand it to an agent loop. You go home. By morning you have a small, fine-tuned model that's measurably better.  All will depend on "how good is our eval." Learn more: https://lnkd.in/d-MYhnsv

Things you might have missed from the Gemma 4 launch today! ⬇️ - You can use Gemma 4 as your agent for building Android apps in Android Studio, offline! https://lnkd.in/dNpKVtu4 - You can use LiteRT to load Gemma in Android and iOS. https://lnkd.in/djgwKJ3v - You can download Gallery App (which uses Gemma) in the Playstore, and try a cool new agentic experience https://lnkd.in/dhQSeFiP - You can use the model in Vertex Model Garden, fine-tune in VTC, try with ADK, Cloud Run, GKE, GKE Agent Sandbox, MaxText, VLLM with TPUs, and Sovereign Cloud solutions https://lnkd.in/dK9wPUbH - The license changed to Apache 2.0 https://lnkd.in/dQW3GKe5 - You can try the it in the AICore Developer Preview. https://lnkd.in/dPY-rvfE

Read the technical reports on how Kimi (Moonshot AI), Cursor, and Chroma train vertical agentic models with RL. Same underlying recipe, strong base model, train inside the production harness, outcome-based rewards. - Kimi K2.5 learns to spawn parallel sub-agents through RL. -Cursor uses the same production Harness (same tools, same prompts..) and leanrs self-summarization during RL. - Chroma's 20B retrieval model learns to prune its own context mid-search. Full write-up 👇 https://lnkd.in/dGp7_SxU

We shipped one of the most requested Gemini API features! 🥳 You can now combine built-in tools (Google Search, URL Context,…) with your own functions custom in a single API call. Gemini orchestrates everything: 🔧 Combine Google Search, Google Maps, File Search or Url Context with custom tools in one API call 🔄 Built-in tool context is circulated with signatures. 🤖 Gemini decides tool order and chains results. 🆕 Google Maps is now available for Gemini 3 models. Available natively in the Interactions API and opt-in via generate_content. Learn More below 🔽 Blog: https://lnkd.in/d44twa4k Example: https://lnkd.in/dzivDQVC Docs: https://lnkd.in/d7sXU7Hc

We just launched Gemini 3.1 Flash Live! Our fastest, most natural real-time voice AI model for building Agents. - Scores 90.8% on ComplexFuncBench Audio for tool use. - 70 languages, Video streaming, Audio transcriptions, 128k context - Comes with Agent Skill for building live voice agents. - All generated audio is watermarked with SynthID. Blog: https://lnkd.in/de-j3xCT Skill: https://lnkd.in/dtdKiuRx Docs: https://lnkd.in/d9Wu8PjA

Turn prompts into production-ready apps with Google AI Studio newly upgraded vibe coding experience! - Powered by the new Google Antigravity coding agent. - Integrates Firebase for secure sign-in and Cloud Firestore. - Supports Next.js, React, and Angular out of the box. - Stores API keys safely in the new Secrets Manager. Examples and more: https://lnkd.in/du7pKM7n

Today, we’re releasing Lyria 3 music generation models (Pro/Clip) in Google AI Studio and Gemini API! 🎵 - Lyria 3 Pro generates full songs (minutes, controllable via prompt), $0.08/song. - Lyria 3 Clip creates 30-second audio clips, $0.04/song. - Control tempo, time-aligned lyrics, and use image-to-music inputs. - Uses SynthID digital watermark for content identification. Docs: https://lnkd.in/dxVd2BQs Blog: https://lnkd.in/draMhPnK

What if one embedding model could understand text, images, video, audio, and PDFs all at once? Excited to share Gemini Embedding 2 our first fully multimodal embedding model. 🖼️ 5 modalities in a single unified embedding space 🌍 Supports up to 8,192 input tokens, 100+ languages 🎧 Embeds audio natively, no transcription step needed 📐 Flexible output dimensions: 3,072 / 1,536 / 768 via MRL 📎 Up to 6 images, 120s video, and 6-page PDFs per request Now in Public Preview via Gemini API & Vertex AI. Docs: https://lnkd.in/dutRFSqH Blog: https://lnkd.in/d_YpkZq5

Google Colab now has an open-source MCP server that lets you use Colab runtimes with GPUs from any local AI agent. 🔧 Tools to execute_code, connect, notebook editing ☁️ Run Python on cloud GPUs directly from agents 📝 Can create .ipynb files and add code/markdown  🔌 Works with Gemini CLI, Antigravity or any MCP-compatible client https://lnkd.in/e9DrAYwr

Agent skills are powerful but they are often AI-generated and not tested. Here is a practical guide to evaluating agent skills with code, prompts, and real results. 📋 Define success criteria (outcome, style, and efficiency). 🧪 Create 10-12 prompts with deterministic checks. 🤖 Add LLM-as-judge with for qualitative checks. 🔁 Iterate on the skill using eval failures. Blog: https://lnkd.in/dvHjB_Mf

Big QoL! Spend caps are rolling out for the Gemini API!!  Please go set a cap and send us any feedback as you use them! - Spend caps can have up to a 10 minute delay before taking effect - We are working to bring this latency down over time - We are shortly rolling out email notifications when you hit caps  - Spend caps are experimental so pls send us feedback on things you want here Set a spend cap: https://lnkd.in/dQnkabsC

New Google DeepMind Research to help the industry understand and measure AI manipulation risks in the real world. The team conducted nine studies involving over 10,000 participants across three countries to measure harmful manipulation. Finding that AI manipulation was highly effective in the finance domain. Big kudos to the Responsibility team for this important work. Paper below: https://lnkd.in/d2C-JiDT

Great beginner-friendly guide on vibe-coding with Google AI Studio, covers everything from first prompt to deployment. 🔒 Apps are private by default. 🗄️ Firebase databases with auth in one click. 🎨 Draw directly on your app's UI to give feedback. ☁️ Publish to Cloud Run in a few clicks. 👉 https://lnkd.in/d7BuX8s7

Veo 3.1 Lite now available in Gemini API and @GoogleAIStudio. Designed for rapid prototyping and high-volume video generation, starting at $0.05/sec. 🪶 - ~1/2 the cost of Veo 3.1 Fast (starting $0.05/s). - Text-to-Video (T2V) & Image-to-Video (I2V). - Landscape (16:9) and Portrait (9:16) format - 4s, 6s, and 8s clips. Try in AIS: https://lnkd.in/dQ_ARRC8  Blog: https://lnkd.in/dCgyCp4E

Hey Gemini make a website presenting yourself using the skill below. (Gemini 3.1 Pro Preview) + AI Studio + design-taste-frontend skill.

Nano Banana Pro vs Nano Banana 2 blog: https://lnkd.in/d9effiU4

Wrote a developer guide for Nano Banana 2 with the Gemini Interactions API. The guide walks you through 4 use cases. 📝 Text → Image: Generate a photorealistic Kyoto travel poster. 🔍 + Web Search: Ground images with real landmark facts. 🆕 + Image Search: Retrieve real photos for visual accuracy. 🧑 + Reference Photo: Composite a real person into scenes. Guide: https://lnkd.in/d5C35TSS

We added a new a skill for building with the Gemini Interaction API! The Gemini Interactions API is a unified interface for building advanced agentic applications with Gemini models. install it with the Context 7 or @vercel CLIs: ``` # Vercel skills npx skills add google-gemini/gemini-skills --skill gemini-interactions-api --global # Context7 skills npx ctx7 skills install /google-gemini/gemini-skills gemini-interactions-api ``` https://lnkd.in/dJHcRGHA

Last week, we released tool combination in the Gemini API: Google Search and your own functions in a single request. Gemini picks the tools, the order, and circulates context between them. I believe this is especially useful for building agents, where you combine the power of built-in tools like Google Search with custom functions. Short guide with examples → https://lnkd.in/dBgqZYcN

PSA: Starting April 1, Gemini API billing tier gets a monthly spending cap. Hit it → API pauses till next month or tier upgrade. No surprise bills! Upgrading Tiers is now automated and faster too. You can also set your own per-project spend caps in AI Studio. Check tier: aistudio.google.com/spend Tier docs: https://lnkd.in/dm-AK2XT Questions? Ask below or DM me.

Tau Bench got an update! Tau Bench is one of the most adopted Agentic Benchmarks. They now added “Banking” a fintech-inspired customer support domain built around a realistic knowledge base of 698 documents across 21 product categories. Tasks require agents to search this corpus, reason over what they find, and execute multi-step tool calls. "There's this transaction I want to dispute. I also want to file a credit limit increase request." The best model achieve 25% success of tasks and ~< 10% on pass^4 Leaderboard: https://lnkd.in/dyeTsTeF  Paper: https://lnkd.in/dkqZeWbc