Gadget World | Site-Wide Activity

admin wrote a new post

Sun, 12 Apr 2026 12:07:25 +0000

GLM-5.1: Architecture, Benchmarks, Capabilities & How to Use ItZ.ai is out with its next-generation flagship AI model and has named it GLM-5.1. With […]

admin wrote a new post

Sun, 12 Apr 2026 12:07:20 +0000

MiniMax Just Open Sourced MiniMax M2.7: A Self-Evolving Agent Model that Scores 56.22% on SWE-Pro and 57.0% on Terminal Bench 2MiniMax has […]

admin wrote a new post

Sun, 12 Apr 2026 12:01:07 +0000

Liquid AI Releases LFM2.5-VL-450M: a 450M-Parameter Vision-Language Model with Bounding Box Prediction, Multilingual Support, and Sub-250ms Edge Inference Liquid AI just released LFM2.5-VL-450M, an updated version of its earlier LFM2-VL-450M vision-language model. The new release introduces bounding box prediction, improved instruction following, expanded multilingual understanding, and function calling support — all within a 450M-parameter footprint designed to run directly on edge hardware ranging from embedded AI modules like NVIDIA Jetson Orin, to mini-PC APUs like AMD Ryzen AI Max+ 395, to flagship phone SoCs like the Snapdragon 8 Elite inside the Samsung S25 Ultra. What is a Vision-Language Model and Why Model Size Matters Before going deeper, it helps to understand what a vision-language model (VLM) is. A VLM is a model that can process both images and text together — you can send it a photo and ask questions about it in natural language, and it will respond. Most large VLMs require substantial GPU memory and cloud infrastructure to run. That’s a problem for real-world deployment scenarios like warehouse robots, smart glasses, or retail shelf cameras, where compute is limited and latency must be low. LFM2.5-VL-450M is Liquid AI’s answer to this constraint: a model small enough to fit on edge hardware while still supporting a meaningful set of vision and language capabilities. Architecture and Training LFM2.5-VL-450M uses LFM2.5-350M as its language model backbone and SigLIP2 NaFlex shape-optimized 86M as its vision encoder. The context window is 32,768 tokens with a vocabulary size of 65,536. For image handling, the model supports native resolution processing up to 512×512 pixels without upscaling, preserves non-standard aspect ratios without distortion, and uses a tiling strategy that splits large images into non-overlapping 512×512 patches while including thumbnail encoding for global context. The thumbnail encoding is important: without it, tiling would give the model only local patches with no sense of the overall scene. At inference time, users can tune the maximum image tokens and tile count for a speed/quality tradeoff without retraining, which is useful when deploying across hardware with different compute budgets. The recommended generation parameters from Liquid AI are temperature=0.1, min_p=0.15, and repetition_penalty=1.05 for text, and min_image_tokens=32, max_image_tokens=256, and do_image_splitting=True for vision inputs. On the training side, Liquid AI scaled pre-training from 10T to 28T tokens compared to LFM2-VL-450M, followed by post-training using preference optimization and reinforcement learning to improve grounding, instruction following, and overall reliability across vision-language tasks. New Capabilities Over LFM2-VL-450M The most significant addition is bounding box prediction. LFM2.5-VL-450M scored 81.28 on RefCOCO-M, up from zero on the previous model. RefCOCO-M is a visual grounding benchmark that measures how accurately a model can locate an object in an image given a natural language description. In practice, the model outputs structured JSON with normalized coordinates identifying where objects are in a scene — not just describing what is there, but also locating it. This is meaningfully different from pure image captioning and makes the model directly usable in pipelines that need spatial outputs. Multilingual support also improved substantially. MMMB scores improved from 54.29 to 68.09, covering Arabic, Chinese, French, German, Japanese, Korean, Portuguese, and Spanish. This is relevant for global deployments where local-language prompts must be understood alongside visual inputs, without needing separate localization pipelines. Instruction following improved as well. MM-IFEval scores went from 32.93 to 45.00, meaning the model more reliably adheres to explicit constraints given in a prompt — for example, responding in a particular format or restricting output to specific fields. Function calling support for text-only input was also added, measured by BFCLv4 at 21.08, a capability the previous model did not include. Function calling allows the model to be used in agentic pipelines where it needs to invoke external tools — for instance, calling a weather API or triggering an action in a downstream system. Benchmark Performance Across vision benchmarks evaluated using VLMEvalKit, LFM2.5-VL-450M outperforms both LFM2-VL-450M and SmolVLM2-500M on most tasks. Notable scores include 86.93 on POPE, 684 on OCRBench, 60.91 on MMBench (dev en), and 58.43 on RealWorldQA. Two benchmark gains stand out beyond the headline numbers. MMVet — which tests more open-ended visual understanding — improved from 33.85 to 41.10, a substantial relative gain. CountBench, which evaluates the model’s ability to count objects in a scene, improved from 47.64 to 73.31, one of the largest relative improvements in the table. InfoVQA held roughly flat at 43.02 versus 44.56 on the prior model. On language-only benchmarks, IFEval improved from 51.75 to 61.16 and Multi-IF from 26.21 to 34.63. The model does not outperform on all tasks — MMMU (val) dropped slightly from 34.44 to 32.67 — and Liquid AI notes the model is not well-suited for knowledge-intensive tasks or fine-grained OCR. Edge Inference Performance LFM2.5-VL-450M with Q4_0 quantization runs across the full range of target hardware, from embedded AI modules like Jetson Orin to mini-PC APUs like Ryzen AI Max+ 395 to flagship phone SoCs like Snapdragon 8 Elite. The latency numbers tell a clear story. On Jetson Orin, the model processes a 256×256 image in 233ms and a 512×512 image in 242ms — staying well under 250ms at both resolutions. This makes it fast enough to process every frame in a 4 FPS video stream with full vision-language understanding, not just detection. On Samsung S25 Ultra, latency is 950ms for 256×256 and 2.4 seconds for 512×512. On AMD Ryzen AI Max+ 395, it is 637ms for 256×256 and 944ms for 512×512 — under one second for the smaller resolution on both consumer devices, which keeps interactive applications responsive. Real-World Use Cases LFM2.5-VL-450M is especially well suited to real-world deployments where low latency, compact structured outputs, and efficient semantic reasoning matter most, including settings where offline operation or on-device processing is important for privacy. In industrial automation, compute-constrained environments such as passenger vehicles, agricultural machinery, and warehouses often limit perception models to bounding-box outputs. LFM2.5-VL-450M goes further, providing grounded scene understanding in a single pass — enabling richer outputs for settings like warehouse aisles, including worker actions, forklift movement, and inventory flow — while still fitting existing edge hardware like a Jetson Orin. For wearables and always-on monitoring, devices such as smart glasses, body-worn assistants, dashcams, and security or industrial monitors cannot afford large perception stacks or constant cloud streaming. An efficient VLM can produce compact semantic outputs locally, turning raw video into useful structured understanding while keeping compute demands low and preserving privacy. In retail and e-commerce, tasks like catalog ingestion, visual search, product matching, and shelf compliance require more than object detection, but richer visual understanding is often too expensive to deploy at scale. LFM2.5-VL-450M makes structured visual reasoning practical for these workloads. Key Takeaways LFM2.5-VL-450M adds bounding box prediction for the first time, scoring 81.28 on RefCOCO-M versus zero on the previous model, enabling the model to output structured spatial coordinates for detected objects — not just describe what it sees. Pre-training was scaled from 10T to 28T tokens, combined with post-training via preference optimization and reinforcement learning, driving consistent benchmark gains across vision and language tasks over LFM2-VL-450M. The model runs on edge hardware with sub-250ms latency, processing a 512×512 image in 242ms on NVIDIA Jetson Orin with Q4_0 quantization — fast enough for full vision-language understanding on every frame of a 4 FPS video stream without cloud offloading. Multilingual visual understanding improved significantly, with MMMB scores rising from 54.29 to 68.09 across Arabic, Chinese, French, German, Japanese, Korean, Portuguese, and Spanish, making the model viable for global deployments without separate localization models. Check out the Technical details and Model Weight. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Liquid AI Releases LFM2.5-VL-450M: a 450M-Parameter Vision-Language Model with Bounding Box Prediction, Multilingual Support, and Sub-250ms Edge Inference appeared first on M […]

admin wrote a new post

Sun, 12 Apr 2026 00:00:41 +0000

Researchers from MIT, NVIDIA, and Zhejiang University Propose TriAttention: A KV Cache Compression Method That Matches Full Attention at 2.5× Higher Throughput Long-chain reasoning is one of the most compute-intensive tasks in modern large language models. When a model like DeepSeek-R1 or Qwen3 works through a complex math problem, it can generate tens of thousands of tokens before arriving at an answer. Every one of those tokens must be stored in what is called the KV cache — a memory structure that holds the Key and Value vectors the model needs to attend back to during generation. The longer the reasoning chain, the larger the KV cache grows, and for many deployment scenarios, especially on consumer hardware, this growth eventually exhausts GPU memory entirely. A team of researchers from MIT, NVIDIA, and Zhejiang University proposed a method called TriAttention that directly addresses this problem. On the AIME25 mathematical reasoning benchmark with 32K-token generation, TriAttention matches Full Attention accuracy while achieving 2.5× higher throughput or 10.7× KV memory reduction. Leading baselines achieve only about half the accuracy at the same efficiency level. The Problem with Existing KV Cache Compression To understand why TriAttention is important, it helps to understand the standard approach to KV cache compression. Most existing methods — including SnapKV, H2O, and R-KV — work by estimating which tokens in the KV cache are important and evicting the rest. Importance is typically estimated by looking at attention scores: if a key receives high attention from recent queries, it is considered important and kept. The catch is that these methods operate in what the research team calls post-RoPE space. RoPE, or Rotary Position Embedding, is the positional encoding scheme used by most modern LLMs including Llama, Qwen, and Mistral. RoPE encodes position by rotating the Query and Key vectors in a frequency-dependent way. As a result, a query vector at position 10,000 looks very different from the same semantic query at position 100, because its direction has been rotated by the position encoding. This rotation means that only the most recently generated queries have orientations that are ‘up to date’ for estimating which keys are important right now. Prior work has confirmed this empirically: increasing the observation window for importance estimation does not help — performance peaks at around 25 queries and declines after that. With such a tiny window, some keys that will become important later get permanently evicted. This problem is especially acute for what the research team calls retrieval heads — attention heads whose function is to retrieve specific factual tokens from long contexts. The relevant tokens for a retrieval head can remain dormant for thousands of tokens before suddenly becoming essential to the reasoning chain. Post-RoPE methods, operating over a narrow observation window, see low attention on those tokens during the dormant period and permanently evict them. When the model later needs to recall that information, it is already gone, and the chain of thought breaks. The Pre-RoPE Observation: Q/K Concentration The key insight in TriAttention comes from looking at Query and Key vectors before RoPE rotation is applied — the pre-RoPE space. When the research team visualized Q and K vectors in this space, they found something consistent and striking: across the vast majority of attention heads and across multiple model architectures, both Q and K vectors cluster tightly around fixed, non-zero center points. The research team terms this property Q/K concentration, and measures it using the Mean Resultant Length R — a standard directional statistics measure where R → 1 means tight clustering and R → 0 means dispersion in all directions. On Qwen3-8B, approximately 90% of attention heads exhibit R > 0.95, meaning their pre-RoPE Q/K vectors are nearly perfectly concentrated around their respective centers. Critically, these centers are stable across different token positions and across different input sequences — they are an intrinsic property of the model’s learned weights, not a property of any particular input. The research team further confirm that Q/K concentration is domain-agnostic: measuring Mean Resultant Length across Math, Coding, and Chat domains on Qwen3-8B yields nearly identical values of 0.977–0.980. This stability is what post-RoPE methods cannot exploit. RoPE rotation disperses these concentrated vectors into arc patterns that vary with position. But in pre-RoPE space, the centers remain fixed. From Concentration to a Trigonometric Series The research team then show mathematically that when Q and K vectors are concentrated around their centers, the attention logit — the raw score before softmax that determines how much a query attends to a key — simplifies dramatically. Substituting the Q/K centers into the RoPE attention formula, the logit reduces to a function that depends only on the Q-K distance (the relative positional gap between query and key), expressed as a trigonometric series: logit(Δ)≈∑f‖q‾f‖‖k‾f‖⏟amplitudecos⁡(ωfΔ+ϕ‾f⏟phase)=∑f[afcos⁡(ωfΔ)+bfsin⁡(ωfΔ)] text{logit}(Delta) approx sum_{f} underbrace{|bar{q}_f| |bar{k}_f|}_{text{amplitude}} cos(omega_f Delta + underbrace{bar{phi}_f}_{text{phase}}) = sum_{f} [a_f cos(omega_f Delta) + b_f sin(omega_f Delta)] Here, Δ is the positional distance, ωf are the RoPE rotation frequencies for each frequency band f, and the coefficients af and bf are determined by the Q/K centers. This series produces a characteristic attention-vs-distance curve for each head. Some heads prefer nearby keys (local attention), others prefer very distant keys (attention sinks). The centers, computed offline from calibration data, fully determine which distances are preferred. The research team validated this experimentally across 1,152 attention heads in Qwen3-8B and across Qwen2.5 and Llama3 architectures. The Pearson correlation between the predicted trigonometric curve and the actual attention logits has a mean above 0.5 across all heads, with many heads achieving correlations of 0.6–0.9. The research team further validates this on GLM-4.7-Flash, which uses Multi-head Latent Attention (MLA) rather than standard Grouped-Query Attention — a meaningfully different attention architecture. On MLA, 96.6% of heads exhibit R > 0.95, compared to 84.7% for GQA, confirming that Q/K concentration is not specific to one attention design but is a general property of modern LLMs. How TriAttention Uses This TriAttention is a KV cache compression method that uses these findings to score keys without needing any live query observations. The scoring function has two components: The Trigonometric Series Score (Strig) uses the Q center computed offline and the actual cached key representation to estimate how much attention the key will receive, based on its positional distance from future queries. Because a key may be attended to by queries at many future positions, TriAttention averages this score over a set of future offsets using geometric spacing. Strig(k,Δ)=∑f‖𝔼[qf]‖⋅‖kf‖⋅cos⁡(ωfΔ+ϕf)S_{text{trig}}(k, Delta) = sum_{f} |mathbb{E}[q_f]| cdot |k_f| cdot cos(omega_f Delta + phi_f) The Norm-Based Score (Snorm) handles the minority of attention heads where Q/K concentration is lower. It weights each frequency band by the expected query norm contribution, providing complementary information about token salience beyond distance preference alone. Snorm(0)(k)=∑f𝔼[‖qf‖]⋅‖kf‖S_{text{norm}}^{(0)}(k) = sum_{f} mathbb{E}[|q_f|] cdot |k_f| The two scores are combined using the Mean Resultant Length R as an adaptive weight: when concentration is high, Strig dominates; when concentration is lower, Snorm contributes more. Every 128 generated tokens, TriAttention scores all keys in the cache and retains only the top-B, evicting the rest. Results on Mathematical Reasoning On AIME24 with Qwen3-8B, TriAttention achieves 42.1% accuracy against Full Attention’s 57.1%, while R-KV achieves only 25.4% at the same KV budget of 2,048 tokens. On AIME25, TriAttention achieves 32.9% versus R-KV’s 17.5% — a 15.4 percentage point gap. On MATH 500 with only 1,024 tokens in the KV cache out of a possible 32,768, TriAttention achieves 68.4% accuracy against Full Attention’s 69.6%. The research team also introduces a Recursive State Query benchmark based on recursive simulation using depth-first search. Recursive tasks stress memory retention because the model must maintain intermediate states across long chains and backtrack to them later — if any intermediate state is evicted, the error propagates through all subsequent return values, corrupting the final result. Under moderate memory pressure up to depth 16, TriAttention performs comparably to Full Attention, while R-KV shows catastrophic accuracy degradation — dropping from approximately 61% at depth 14 to 31% at depth 16. This indicates R-KV incorrectly evicts critical intermediate reasoning states. On throughput, TriAttention achieves 1,405 tokens per second on MATH 500 against Full Attention’s 223 tokens per second, a 6.3× speedup. On AIME25, it achieves 563.5 tokens per second against 222.8, a 2.5× speedup at matched accuracy. Generalization Beyond Mathematical Reasoning The results extend well beyond math benchmarks. On LongBench — a 16-subtask benchmark covering question answering, summarization, few-shot classification, retrieval, counting, and code tasks — TriAttention achieves the highest average score of 48.1 among all compression methods at a 50% KV budget on Qwen3-8B, winning 11 out of 16 subtasks and surpassing the next best baseline, Ada-KV+SnapKV, by 2.5 points. On the RULER retrieval benchmark at a 4K context length, TriAttention achieves 66.1, a 10.5-point gap over SnapKV. These results confirm that the method is not tuned to mathematical reasoning alone — the underlying Q/K concentration phenomenon transfers to general language tasks. Key Takeaways Existing KV cache compression methods have a fundamental blind spot: Methods like SnapKV and R-KV estimate token importance using recent post-RoPE queries, but because RoPE rotates query vectors with position, only a tiny window of queries is usable. This causes important tokens — especially those needed by retrieval heads — to be permanently evicted before they become critical. Pre-RoPE Query and Key vectors cluster around stable, fixed centers across nearly all attention heads: This property, called Q/K concentration, holds regardless of input content, token position, or domain, and is consistent across Qwen3, Qwen2.5, Llama3, and even Multi-head Latent Attention architectures like GLM-4.7-Flash. These stable centers make attention patterns mathematically predictable without observing any live queries: When Q/K vectors are concentrated, the attention score between any query and key reduces to a function that depends only on their positional distance — encoded as a trigonometric series. TriAttention uses this to score every cached key offline using calibration data alone. TriAttention matches Full Attention reasoning accuracy at a fraction of the memory and compute cost: On AIME25 with 32K-token generation, it achieves 2.5× higher throughput or 10.7× KV memory reduction while matching Full Attention accuracy — nearly doubling R-KV’s accuracy at the same memory budget across both AIME24 and AIME25. The method generalizes beyond math and works on consumer hardware. TriAttention outperforms all baselines on LongBench across 16 general NLP subtasks and on the RULER retrieval benchmark, and enables a 32B reasoning model to run on a single 24GB RTX 4090 via OpenClaw — a task that causes out-of-memory errors under Full Attention. Check out the Paper, Repo and Project Page. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Researchers from MIT, NVIDIA, and Zhejiang University Propose TriAttention: A KV Cache Compression Method That Matches Full Attention at 2.5× Higher Throughput appeared first o […]

admin wrote a new post

Sun, 12 Apr 2026 00:00:40 +0000

How to Build a Secure Local-First Agent Runtime with OpenClaw Gateway, Skills, and Controlled Tool ExecutionIn this tutorial, we build and operate […]

admin wrote a new post

Sat, 11 Apr 2026 21:04:23 +0000

HealthcareExplore how clinicians use ChatGPT to support diagnosis, documentation, and patient care with secure, HIPAA-compliant AI tools.

admin wrote a new post

Sat, 11 Apr 2026 18:08:24 +0000

Responsible and safe use of AILearn how to use AI responsibly with best practices for safety, accuracy, and transparency when using tools like ChatGPT.

admin wrote a new post

Sat, 11 Apr 2026 12:06:51 +0000

Understanding BERTopic: From Raw Text to Interpretable Topics Topic modeling uncovers hidden themes in large document collections. Traditional […]

admin wrote a new post

Sat, 11 Apr 2026 12:02:04 +0000

How Knowledge Distillation Compresses Ensemble Intelligence into a Single Deployable AI Model Complex prediction problems often lead to ensembles […]

admin wrote a new post

Sat, 11 Apr 2026 03:09:22 +0000

Using custom GPTsLearn how to build and use custom GPTs to automate workflows, maintain consistent outputs, and create purpose-built AI assistants.

admin wrote a new post

Sat, 11 Apr 2026 00:22:41 +0000

Research with ChatGPTLearn how to research with ChatGPT using search and deep research to find up-to-date information, analyze sources, and generate structured insights.

admin wrote a new post

Sat, 11 Apr 2026 00:22:40 +0000

From Karpathy’s LLM Wiki to Graphify: AI Memory Layers are Here Most AI workflows follow the same loop: you upload files, ask a question, get an […]

admin wrote a new post

Sat, 11 Apr 2026 00:18:44 +0000

Alibaba’s Tongyi Lab Releases VimRAG: a Multimodal RAG Framework that Uses a Memory Graph to Navigate Massive Visual Contexts Retrieval-Augmented […]

admin wrote a new post

Sat, 11 Apr 2026 00:10:41 +0000

A Coding Guide to Markerless 3D Human Kinematics with Pose2Sim, RTMPose, and OpenSim In this tutorial, we build and run a complete Pose2Sim pipeline […]

admin wrote a new post

Sat, 11 Apr 2026 00:10:35 +0000

New Manychat Features You’ll Actually Want To Use: PDFs in DMs, Inbox Updates, and MoreThere are product updates, and then there are product updates that make you want to sit up, crack your knuckles, and start doing slightly unwell things…

admin wrote a new post

Sat, 11 Apr 2026 00:10:35 +0000

The Influencer Marketing Trends Changing How Brands and Creators CollaborateInstagram marketing is moving fast. What are the latest tactics and how can you apply them to your marketing efforts? Learn the top four trends in this guide.

admin wrote a new post

Sat, 11 Apr 2026 00:10:33 +0000

Best AI Chatbot Platforms for E-commerce in 2026In 2026, the competitive landscape of e-commerce demands more than just a compelling product; […]

admin wrote a new post

Sat, 11 Apr 2026 00:10:33 +0000

Best AI Chatbot Platforms for E-commerce in 2026In 2026, the competitive landscape of e-commerce demands more than just a compelling product; it […]

admin wrote a new post

Sat, 11 Apr 2026 00:10:26 +0000

What’s in a name? Moderna’s “vaccine” vs. “therapy” dilemmaIs it the Department of Defense or the Department of War? The Gulf of Mexico or the Gulf of […]

admin wrote a new post

Sat, 11 Apr 2026 00:00:38 +0000

The Download: an exclusive Jeff VanderMeer story and AI models too scary to release This is today’s edition of The Download, our weekday n […]

admin wrote a new post

Fri, 10 Apr 2026 21:03:14 +0000

Using projects in ChatGPTLearn how to use orojects in ChatGPT to organize chats, files, and instructions, manage ongoing work, and collaborate more effectively.

admin wrote a new post

Fri, 10 Apr 2026 18:05:33 +0000

Getting started with ChatGPTLearn how to use ChatGPT, start your first conversation, and discover simple ways to write, brainstorm, and solve problems with AI.

admin wrote a new post

Fri, 10 Apr 2026 12:09:42 +0000

10 Most Important AI Concepts Explained SimplyAI can feel like a maze sometimes. Everywhere you look, people on social media and in meetings are […]

admin wrote a new post

Fri, 10 Apr 2026 12:00:49 +0000

Five AI Compute Architectures Every Engineer Should Know: CPUs, GPUs, TPUs, NPUs, and LPUs Compared Modern AI is no longer powered by a single type of […]

admin wrote a new post

Fri, 10 Apr 2026 12:00:48 +0000

An End-to-End Coding Guide to NVIDIA KVPress for Long-Context LLM Inference, KV Cache Compression, and Memory-Efficient GenerationIn this tutorial, […]

admin wrote a new post

Fri, 10 Apr 2026 12:00:26 +0000

ConstellationsI. We had crash-landed on the planet. We were far from home. The spaceship could not be repaired, and the rescue beacon had […]

admin wrote a new post

Fri, 10 Apr 2026 00:21:21 +0000

CyberAgent moves faster with ChatGPT Enterprise and CodexCyberAgent uses ChatGPT Enterprise and Codex to securely scale AI adoption, improve quality, and accelerate decisions across advertising, media, and gaming.

admin wrote a new post

Fri, 10 Apr 2026 00:21:21 +0000

Project Glasswing is World’s Most Powerful AI in ActionWe already had a hint. AI would surpass most human capabilities someday. In the field of […]

admin wrote a new post

Fri, 10 Apr 2026 00:21:16 +0000

ConvApparel: Measuring and bridging the realism gap in user simulatorsGenerative AI

admin wrote a new post

Fri, 10 Apr 2026 00:11:55 +0000

Meta Superintelligence Lab Releases Muse Spark: A Multimodal Reasoning Model With Thought Compression and Parallel Agents Meta Superintelligence […]

admin wrote a new post

Fri, 10 Apr 2026 00:11:37 +0000

The Download: AstroTurf wars and exponential AI growthThis is today’s edition of The Download, our weekday newsletter that provides a daily dose o […]

admin wrote a new post

Thu, 09 Apr 2026 15:05:22 +0000

OpenAI Full Fan Mode Contest: Terms & ConditionsExplore the official terms and conditions for the OpenAI Full Fan Mode Contest, including […]

admin wrote a new post

Thu, 09 Apr 2026 12:09:00 +0000

Sigmoid vs ReLU Activation Functions: The Inference Cost of Losing Geometric Context A deep neural network can be understood as a geometric system, […]

admin wrote a new post

Thu, 09 Apr 2026 12:08:59 +0000

A Coding Guide to Build Advanced Document Intelligence Pipelines with Google LangExtract, OpenAI Models, Structured Extraction, and Interactive VisualizationIn this tutorial, we explore how to use Google’s LangExtract library to transform unstructured text into structured, machine-readable information. We begin by installing the required dependencies and securely configuring our OpenAI API key to leverage powerful language models for extraction tasks. Also, we will build a reusable extraction pipeline that enables us to process a range of document types, including contracts, meeting notes, product announcements, and operational logs. Through carefully designed prompts and example annotations, we demonstrate how LangExtract can identify entities, actions, deadlines, risks, and other structured attributes while grounding them to their exact source spans. We also visualize the extracted information and organize it into tabular datasets, enabling downstream analytics, automation workflows, and decision-making systems. Copy CodeCopiedUse a different Browser!pip -q install -U “langextract[openai]” pandas IPython import os import json import textwrap import getpass import pandas as pd OPENAI_API_KEY = getpass.getpass(“Enter OPENAI_API_KEY: “) os.environ[“OPENAI_API_KEY”] = OPENAI_API_KEY import langextract as lx from IPython.display import display, HTML We install the required libraries, including LangExtract, Pandas, and IPython, so that our Colab environment is ready for structured extraction tasks. We securely request the OpenAI API key from the user and store it as an environment variable for safe access during runtime. We then import the core libraries needed to run LangExtract, display results, and handle structured outputs. Copy CodeCopiedUse a different BrowserMODEL_ID = “gpt-4o-mini” def run_extraction( text_or_documents, prompt_description, examples, output_stem, model_id=MODEL_ID, extraction_passes=1, max_workers=4, max_char_buffer=1800, ): result = lx.extract( text_or_documents=text_or_documents, prompt_description=prompt_description, examples=examples, model_id=model_id, api_key=os.environ[“OPENAI_API_KEY”], fence_output=True, use_schema_constraints=False, extraction_passes=extraction_passes, max_workers=max_workers, max_char_buffer=max_char_buffer, ) jsonl_name = f”{output_stem}.jsonl” html_name = f”{output_stem}.html” lx.io.save_annotated_documents([result], output_name=jsonl_name, output_dir=”.”) html_content = lx.visualize(jsonl_name) with open(html_name, “w”, encoding=”utf-8″) as f: if hasattr(html_content, “data”): f.write(html_content.data) else: f.write(html_content) return result, jsonl_name, html_name def extraction_rows(result): rows = [] for ex in result.extractions: start_pos = None end_pos = None if getattr(ex, “char_interval”, None): start_pos = ex.char_interval.start_pos end_pos = ex.char_interval.end_pos rows.append({ “class”: ex.extraction_class, “text”: ex.extraction_text, “attributes”: json.dumps(ex.attributes or {}, ensure_ascii=False), “start”: start_pos, “end”: end_pos, }) return pd.DataFrame(rows) def preview_result(title, result, html_name, max_rows=50): print(“=” * 80) print(title) print(“=” * 80) print(f”Total extractions: {len(result.extractions)}”) df = extraction_rows(result) display(df.head(max_rows)) display(HTML(f’Open interactive visualization: {html_name}’)) We define the core utility functions that power the entire extraction pipeline. We create a reusable run_extraction function that sends text to the LangExtract engine and generates both JSONL and HTML outputs. We also define helper functions to convert the extraction results into tabular rows and preview them interactively in the notebook. Copy CodeCopiedUse a different Browsercontract_prompt = textwrap.dedent(“”” Extract contract-risk information in order of appearance. Rules: 1. Use exact text spans from the source. Do not paraphrase extraction_text. 2. Extract the following classes when present: – party – obligation – deadline – payment_term – penalty – termination_clause – governing_law 3. Add useful attributes: – party_name for obligations or payment terms when relevant – risk_level as low, medium, or high – category for the business meaning 4. Keep output grounded to the exact wording in the source. 5. Do not merge non-contiguous spans into one extraction. “””) contract_examples = [ lx.data.ExampleData( text=( “Acme Corp shall deliver the equipment by March 15, 2026. ” “The Client must pay within 10 days of invoice receipt. ” “Late payment incurs a 2% monthly penalty. ” “This agreement is governed by the laws of Ontario.” ), extractions=[ lx.data.Extraction( extraction_class=”party”, extraction_text=”Acme Corp”, attributes={“category”: “supplier”, “risk_level”: “low”} ), lx.data.Extraction( extraction_class=”obligation”, extraction_text=”shall deliver the equipment”, attributes={“party_name”: “Acme Corp”, “category”: “delivery”, “risk_level”: “medium”} ), lx.data.Extraction( extraction_class=”deadline”, extraction_text=”by March 15, 2026″, attributes={“category”: “delivery_deadline”, “risk_level”: “medium”} ), lx.data.Extraction( extraction_class=”party”, extraction_text=”The Client”, attributes={“category”: “customer”, “risk_level”: “low”} ), lx.data.Extraction( extraction_class=”payment_term”, extraction_text=”must pay within 10 days of invoice receipt”, attributes={“party_name”: “The Client”, “category”: “payment”, “risk_level”: “medium”} ), lx.data.Extraction( extraction_class=”penalty”, extraction_text=”2% monthly penalty”, attributes={“category”: “late_payment”, “risk_level”: “high”} ), lx.data.Extraction( extraction_class=”governing_law”, extraction_text=”laws of Ontario”, attributes={“category”: “legal_jurisdiction”, “risk_level”: “low”} ), ] ) ] contract_text = “”” BluePeak Analytics shall provide a production-ready dashboard and underlying ETL pipeline no later than April 30, 2026. North Ridge Manufacturing will remit payment within 7 calendar days after final acceptance. If payment is delayed beyond 15 days, BluePeak Analytics may suspend support services and charge interest at 1.5% per month. This Agreement shall be governed by the laws of British Columbia. “”” contract_result, contract_jsonl, contract_html = run_extraction( text_or_documents=contract_text, prompt_description=contract_prompt, examples=contract_examples, output_stem=”contract_risk_extraction”, extraction_passes=2, max_workers=4, max_char_buffer=1400, ) preview_result(“USE CASE 1 — Contract risk extraction”, contract_result, contract_html) We build a contract intelligence extraction workflow by defining a detailed prompt and structured examples. We provide LangExtract with annotated training-style examples so that it understands how to identify entities such as obligations, deadlines, penalties, and governing laws. We then run the extraction pipeline on a contract text and preview the structured risk-related outputs. Copy CodeCopiedUse a different Browsermeeting_prompt = textwrap.dedent(“”” Extract action items from meeting notes in order of appearance. Rules: 1. Use exact text spans from the source. No paraphrasing in extraction_text. 2. Extract these classes when present: – assignee – action_item – due_date – blocker – decision 3. Add attributes: – priority as low, medium, or high – workstream when inferable from local context – owner for action_item when tied to a named assignee 4. Keep all spans grounded to the source text. 5. Preserve order of appearance. “””) meeting_examples = [ lx.data.ExampleData( text=( “Sarah will finalize the launch email by Friday. ” “The team decided to postpone the webinar. ” “Blocked by missing legal approval.” ), extractions=[ lx.data.Extraction( extraction_class=”assignee”, extraction_text=”Sarah”, attributes={“priority”: “medium”, “workstream”: “marketing”} ), lx.data.Extraction( extraction_class=”action_item”, extraction_text=”will finalize the launch email”, attributes={“owner”: “Sarah”, “priority”: “high”, “workstream”: “marketing”} ), lx.data.Extraction( extraction_class=”due_date”, extraction_text=”by Friday”, attributes={“priority”: “medium”, “workstream”: “marketing”} ), lx.data.Extraction( extraction_class=”decision”, extraction_text=”decided to postpone the webinar”, attributes={“priority”: “medium”, “workstream”: “events”} ), lx.data.Extraction( extraction_class=”blocker”, extraction_text=”missing legal approval”, attributes={“priority”: “high”, “workstream”: “compliance”} ), ] ) ] meeting_text = “”” Arjun will prepare the revised pricing sheet by Tuesday evening. Mina to confirm the enterprise customer’s data residency requirements this week. The group agreed to ship the pilot only for the Oman region first. Blocked by pending security review from the client’s IT team. Ravi will draft the rollback plan before the production cutover. “”” meeting_result, meeting_jsonl, meeting_html = run_extraction( text_or_documents=meeting_text, prompt_description=meeting_prompt, examples=meeting_examples, output_stem=”meeting_action_extraction”, extraction_passes=2, max_workers=4, max_char_buffer=1400, ) preview_result(“USE CASE 2 — Meeting notes to action tracker”, meeting_result, meeting_html) We design a meeting intelligence extractor that focuses on action items, decisions, assignees, and blockers. We again provide example annotations to help the model structure meet information consistently. We execute the extraction on meeting notes and display the resulting structured task tracker. Copy CodeCopiedUse a different Browserlongdoc_prompt = textwrap.dedent(“”” Extract product launch intelligence in order of appearance. Rules: 1. Use exact text spans from the source. 2. Extract: – company – product – launch_date – region – metric – partnership 3. Add attributes: – category – significance as low, medium, or high 4. Keep the extraction grounded in the original text. 5. Do not paraphrase the extracted span. “””) longdoc_examples = [ lx.data.ExampleData( text=( “Nova Robotics launched Atlas Mini in Europe on 12 January 2026. ” “The company reported 18% faster picking speed and partnered with Helix Warehousing.” ), extractions=[ lx.data.Extraction( extraction_class=”company”, extraction_text=”Nova Robotics”, attributes={“category”: “vendor”, “significance”: “medium”} ), lx.data.Extraction( extraction_class=”product”, extraction_text=”Atlas Mini”, attributes={“category”: “product_name”, “significance”: “high”} ), lx.data.Extraction( extraction_class=”region”, extraction_text=”Europe”, attributes={“category”: “market”, “significance”: “medium”} ), lx.data.Extraction( extraction_class=”launch_date”, extraction_text=”12 January 2026″, attributes={“category”: “timeline”, “significance”: “medium”} ), lx.data.Extraction( extraction_class=”metric”, extraction_text=”18% faster picking speed”, attributes={“category”: “performance_claim”, “significance”: “high”} ), lx.data.Extraction( extraction_class=”partnership”, extraction_text=”partnered with Helix Warehousing”, attributes={“category”: “go_to_market”, “significance”: “medium”} ), ] ) ] long_text = “”” Vertex Dynamics introduced FleetSense 3.0 for industrial logistics teams across the GCC on 5 February 2026. The company said the release improves the accuracy of route deviation detection by 22% and reduces manual review time by 31%. In the first rollout phase, the platform will support Oman and the United Arab Emirates. Vertex Dynamics also partnered with Falcon Telematics to integrate live driver behavior events into the dashboard. A week later, FleetSense 3.0 added a risk-scoring module for safety managers. The update gives supervisors a daily ranked list of high-risk trips and exception events. The company described the module as especially valuable for oilfield transport operations and contractor fleet audits. By late February 2026, the team announced a pilot with Desert Haul Services. The pilot covers 240 heavy vehicles and focuses on speeding up incident triage, compliance review, and evidence retrieval. Internal testing showed analysts could assemble review packets in under 8 minutes instead of the previous 20 minutes. “”” longdoc_result, longdoc_jsonl, longdoc_html = run_extraction( text_or_documents=long_text, prompt_description=longdoc_prompt, examples=longdoc_examples, output_stem=”long_document_extraction”, extraction_passes=3, max_workers=8, max_char_buffer=1000, ) preview_result(“USE CASE 3 — Long-document extraction”, longdoc_result, longdoc_html) batch_docs = [ “”” The supplier must replace defective batteries within 14 days of written notice. Any unresolved safety issue may trigger immediate suspension of shipments. “””, “”” Priya will circulate the revised onboarding checklist tomorrow morning. The team approved the API deprecation plan for the legacy endpoint. “””, “”” Orbit Health launched a remote triage assistant in Singapore on 14 March 2026. The company claims the assistant reduces nurse intake time by 17%. “”” ] batch_prompt = textwrap.dedent(“”” Extract operationally useful spans in order of appearance. Allowed classes: – obligation – deadline – penalty – assignee – action_item – decision – company – product – launch_date – metric Use exact text only and attach a simple attribute: – source_type “””) batch_examples = [ lx.data.ExampleData( text=”Jordan will submit the report by Monday. Late delivery incurs a service credit.”, extractions=[ lx.data.Extraction( extraction_class=”assignee”, extraction_text=”Jordan”, attributes={“source_type”: “meeting”} ), lx.data.Extraction( extraction_class=”action_item”, extraction_text=”will submit the report”, attributes={“source_type”: “meeting”} ), lx.data.Extraction( extraction_class=”deadline”, extraction_text=”by Monday”, attributes={“source_type”: “meeting”} ), lx.data.Extraction( extraction_class=”penalty”, extraction_text=”service credit”, attributes={“source_type”: “contract”} ), ] ) ] batch_results = [] for idx, doc in enumerate(batch_docs, start=1): res, jsonl_name, html_name = run_extraction( text_or_documents=doc, prompt_description=batch_prompt, examples=batch_examples, output_stem=f”batch_doc_{idx}”, extraction_passes=2, max_workers=4, max_char_buffer=1200, ) df = extraction_rows(res) df.insert(0, “document_id”, idx) batch_results.append(df) print(f”Finished document {idx} -> {html_name}”) batch_df = pd.concat(batch_results, ignore_index=True) print(“nCombined batch output”) display(batch_df) print(“nContract extraction counts by class”) display( extraction_rows(contract_result) .groupby(“class”, as_index=False) .size() .sort_values(“size”, ascending=False) ) print(“nMeeting action items only”) meeting_df = extraction_rows(meeting_result) display(meeting_df[meeting_df[“class”] == “action_item”]) print(“nLong-document metrics only”) longdoc_df = extraction_rows(longdoc_result) display(longdoc_df[longdoc_df[“class”] == “metric”]) final_df = pd.concat([ extraction_rows(contract_result).assign(use_case=”contract_risk”), extraction_rows(meeting_result).assign(use_case=”meeting_actions”), extraction_rows(longdoc_result).assign(use_case=”long_document”), ], ignore_index=True) final_df.to_csv(“langextract_tutorial_outputs.csv”, index=False) print(“nSaved CSV: langextract_tutorial_outputs.csv”) print(“nGenerated files:”) for name in [ contract_jsonl, contract_html, meeting_jsonl, meeting_html, longdoc_jsonl, longdoc_html, “langextract_tutorial_outputs.csv” ]: print(” -“, name) We implement a long-document intelligence pipeline capable of extracting structured insights from large narrative text. We run the extraction across product launch reports and operational documents, and also demonstrate batch processing across multiple documents. We also analyze the extracted results, filter key classes, and export the structured dataset to a CSV file for downstream analysis. In conclusion, we constructed an advanced LangExtract workflow that converts complex text documents into structured datasets with traceable source grounding. We ran multiple extraction scenarios, including contract risk analysis, meeting action tracking, long-document intelligence extraction, and batch processing across multiple documents. We also visualized the extractions and exported the final structured results into a CSV file for further analysis. Through this process, we saw how prompt design, example-based extraction, and scalable processing techniques allow us to build robust information extraction systems with minimal code. Check out the Full Codes here. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post A Coding Guide to Build Advanced Document Intelligence Pipelines with Google LangExtract, OpenAI Models, Structured Extraction, and Interactive Visualization appeared first on Ma […]

admin wrote a new post

Thu, 09 Apr 2026 12:01:59 +0000

Conversational AI for Customer Service Customer expectations have never been higher than they are today. People want fast answers, […]

admin wrote a new post

Thu, 09 Apr 2026 12:01:44 +0000

Is fake grass a bad idea? The AstroTurf wars are far from over. A rare warm spell in January melted enough snow to uncover Cornell University’s n […]

admin wrote a new post

Thu, 09 Apr 2026 12:01:43 +0000

Desalination technology, by the numbers When I started digging into desalination technology for a new story, I couldn’t help but obsess over the n […]

admin wrote a new post

Thu, 09 Apr 2026 00:18:06 +0000

The next phase of enterprise AIOpenAI outlines the next phase of enterprise AI, as adoption accelerates across industries with Frontier, ChatGPT Enterprise, Codex, and company-wide AI agents.

admin wrote a new post

Thu, 09 Apr 2026 00:18:05 +0000

How to Run Gemma 4 on Your Phone Without Internet: A Hands-On Guide Most AI tools rely on the internet, sending your prompts to remote servers for […]

admin wrote a new post

Thu, 09 Apr 2026 00:18:04 +0000

Running Gemma 4 Locally with Ollama on Your PCOpen-weight models are driving the latest excitement in the AI landscape. Running powerful models […]

admin wrote a new post

Thu, 09 Apr 2026 00:17:59 +0000

Improving the academic workflow: Introducing two AI agents for better figures and peer reviewGenerative AI

admin wrote a new post

Thu, 09 Apr 2026 00:13:43 +0000

A Comprehensive Implementation Guide to ModelScope for Model Search, Inference, Fine-Tuning, Evaluation, and Export In this tutorial, we explore […]

admin wrote a new post

Thu, 09 Apr 2026 00:06:58 +0000

Meet OSGym: A New OS Infrastructure Framework That Manages 1,000+ Replicas at $0.23/Day for Computer Use Agent Research Training AI agents that can […]

admin wrote a new post

Thu, 09 Apr 2026 00:06:40 +0000

Mustafa Suleyman: AI development won’t hit a wall anytime soon—here’s whyWe evolved for a linear world. If you walk for an hour, you cover a certain […]

admin wrote a new post

Thu, 09 Apr 2026 00:06:40 +0000

The Download: water threats in Iran and AI’s impact on what entrepreneurs makeThis is today’s edition of The Download, our weekday newsletter t […]

admin wrote a new post

Wed, 08 Apr 2026 12:08:58 +0000

Introducing the Child Safety BlueprintDiscover OpenAI’s Child Safety Blueprint—a roadmap for building AI responsibly with safeguards, age-appropriate design, and collaboration to protect and empower young people online.

admin wrote a new post

Wed, 08 Apr 2026 12:08:54 +0000

Z.AI Introduces GLM-5.1: An Open-Weight 754B Agentic Model That Achieves SOTA on SWE-Bench Pro and Sustains 8-Hour Autonomous ExecutionZ.AI, the […]

admin wrote a new post

Wed, 08 Apr 2026 12:01:30 +0000

How to Combine Google Search, Google Maps, and Custom Functions in a Single Gemini API Call With Context Circulation, Parallel Tool IDs, and Multi-Step Agentic Chains In this tutorial, we explore the latest Gemini API tooling updates Google announced in March 2026, specifically the ability to combine built-in tools like Google Search and Google Maps with custom function calls in a single API request. We walk through five hands-on demos that progressively build on each other, starting with the core tool combination feature and ending with a full multi-tool agentic chain. Along the way, we demonstrate how context circulation preserves every tool call and response across turns, enabling the model to reason over prior outputs; how unique tool response IDs let us map parallel function calls to their exact results; and how Grounding with Google Maps brings real-time location data into our applications. We use gemini-3-flash-preview for tool combination features and gemini-2.5-flash for Maps grounding, so everything we build here runs without any billing setup. Copy CodeCopiedUse a different Browserimport subprocess, sys subprocess.check_call( [sys.executable, “-m”, “pip”, “install”, “-qU”, “google-genai”], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, ) import getpass, json, textwrap, os, time from google import genai from google.genai import types if “GOOGLE_API_KEY” not in os.environ: os.environ[“GOOGLE_API_KEY”] = getpass.getpass(“Enter your Gemini API key: “) client = genai.Client(api_key=os.environ[“GOOGLE_API_KEY”]) TOOL_COMBO_MODEL = “gemini-3-flash-preview” MAPS_MODEL = “gemini-2.5-flash” DIVIDER = “=” * 72 def heading(title: str): print(f”n{DIVIDER}”) print(f” {title}”) print(DIVIDER) def wrap(text: str, width: int = 80): for line in text.splitlines(): print(textwrap.fill(line, width=width) if line.strip() else “”) def describe_parts(response): parts = response.candidates[0].content.parts fc_ids = {} for i, part in enumerate(parts): prefix = f” Part {i:2d}:” if hasattr(part, “tool_call”) and part.tool_call: tc = part.tool_call print(f”{prefix} [toolCall] type={tc.tool_type} id={tc.id}”) if hasattr(part, “tool_response”) and part.tool_response: tr = part.tool_response print(f”{prefix} [toolResponse] type={tr.tool_type} id={tr.id}”) if hasattr(part, “executable_code”) and part.executable_code: code = part.executable_code.code[:90].replace(“n”, ” ↵ “) print(f”{prefix} [executableCode] {code}…”) if hasattr(part, “code_execution_result”) and part.code_execution_result: out = (part.code_execution_result.output or “”)[:90] print(f”{prefix} [codeExecResult] {out}”) if hasattr(part, “function_call”) and part.function_call: fc = part.function_call fc_ids[fc.name] = fc.id print(f”{prefix} [functionCall] name={fc.name} id={fc.id}”) print(f” └─ args: {dict(fc.args)}”) if hasattr(part, “text”) and part.text: snippet = part.text[:110].replace(“n”, ” “) print(f”{prefix} [text] {snippet}…”) if hasattr(part, “thought_signature”) and part.thought_signature: print(f” └─ thought_signature present ✓”) return fc_ids heading(“DEMO 1: Combine Google Search + Custom Function in One Request”) print(“”” This demo shows the flagship new feature: passing BOTH a built-in tool (Google Search) and a custom function declaration in a single API call. Gemini will: Turn 1 → Search the web for real-time info, then request our custom function to get weather data. Turn 2 → We supply the function response; Gemini synthesizes everything. Key points: • google_search and function_declarations go in the SAME Tool object • include_server_side_tool_invocations must be True (on ToolConfig) • Return ALL parts (incl. thought_signatures) in subsequent turns “””) get_weather_func = types.FunctionDeclaration( name=”getWeather”, description=”Gets the current weather for a requested city.”, parameters=types.Schema( type=”OBJECT”, properties={ “city”: types.Schema( type=”STRING”, description=”The city and state, e.g. Utqiagvik, Alaska”, ), }, required=[“city”], ), ) print(” Turn 1: Sending prompt with Google Search + getWeather tools…n”) response_1 = client.models.generate_content( model=TOOL_COMBO_MODEL, contents=( “What is the northernmost city in the United States? ” “What’s the weather like there today?” ), config=types.GenerateContentConfig( tools=[ types.Tool( google_search=types.GoogleSearch(), function_declarations=[get_weather_func], ), ], tool_config=types.ToolConfig( include_server_side_tool_invocations=True, ), ), ) print(” Parts returned by the model:n”) fc_ids = describe_parts(response_1) function_call_id = fc_ids.get(“getWeather”) print(f”n Captured function_call id for getWeather: {function_call_id}”) print(“n Turn 2: Returning function result & requesting final synthesis…n”) history = [ types.Content( role=”user”, parts=[ types.Part( text=( “What is the northernmost city in the United States? ” “What’s the weather like there today?” ) ) ], ), response_1.candidates[0].content, types.Content( role=”user”, parts=[ types.Part( function_response=types.FunctionResponse( name=”getWeather”, response={“response”: “Very cold. 22°F / -5.5°C with strong Arctic winds.”}, id=function_call_id, ) ) ], ), ] response_2 = client.models.generate_content( model=TOOL_COMBO_MODEL, contents=history, config=types.GenerateContentConfig( tools=[ types.Tool( google_search=types.GoogleSearch(), function_declarations=[get_weather_func], ), ], tool_config=types.ToolConfig( include_server_side_tool_invocations=True, ), ), ) print(” Final synthesized response:n”) for part in response_2.candidates[0].content.parts: if hasattr(part, “text”) and part.text: wrap(part.text) we install the Google GenAI SDK, securely capture our API key, and define the helper functions that power the rest of the tutorial. We then demonstrate the flagship tool combination feature by sending a single request that pairs Google Search with a custom getWeather function, letting Gemini search the web for real-time geographic data and simultaneously request weather information from our custom tool. We complete the two-turn flow by returning our simulated weather response with the matching function call ID and watching Gemini synthesize both data sources into one coherent answer. Copy CodeCopiedUse a different Browserheading(“DEMO 2: Tool Response IDs for Parallel Function Calls”) print(“”” When Gemini makes multiple function calls in one turn, each gets a unique `id` field. You MUST return each function_response with its matching id so the model maps results correctly. This is critical for parallel calls. “””) time.sleep(2) lookup_inventory = types.FunctionDeclaration( name=”lookupInventory”, description=”Check product inventory by SKU.”, parameters=types.Schema( type=”OBJECT”, properties={ “sku”: types.Schema(type=”STRING”, description=”Product SKU code”), }, required=[“sku”], ), ) get_shipping_estimate = types.FunctionDeclaration( name=”getShippingEstimate”, description=”Get shipping time estimate for a destination zip code.”, parameters=types.Schema( type=”OBJECT”, properties={ “zip_code”: types.Schema(type=”STRING”, description=”Destination ZIP code”), “sku”: types.Schema(type=”STRING”, description=”Product SKU”), }, required=[“zip_code”, “sku”], ), ) print(” Turn 1: Asking about product availability + shipping…n”) resp_parallel = client.models.generate_content( model=TOOL_COMBO_MODEL, contents=( “I want to buy SKU-A100 (wireless headphones). ” “Is it in stock, and how fast can it ship to ZIP 90210?” ), config=types.GenerateContentConfig( tools=[ types.Tool( function_declarations=[lookup_inventory, get_shipping_estimate], ), ], ), ) fc_parts = [] for part in resp_parallel.candidates[0].content.parts: if hasattr(part, “function_call”) and part.function_call: fc = part.function_call fc_parts.append(fc) print(f” [functionCall] name={fc.name} id={fc.id} args={dict(fc.args)}”) print(“n Turn 2: Returning results with matching IDs…n”) simulated_results = { “lookupInventory”: {“in_stock”: True, “quantity”: 342, “warehouse”: “Los Angeles”}, “getShippingEstimate”: {“days”: 2, “carrier”: “FedEx”, “cost”: “$5.99”}, } fn_response_parts = [] for fc in fc_parts: result = simulated_results.get(fc.name, {“error”: “unknown function”}) fn_response_parts.append( types.Part( function_response=types.FunctionResponse( name=fc.name, response=result, id=fc.id, ) ) ) print(f” Responding to {fc.name} (id={fc.id}) → {result}”) history_parallel = [ types.Content( role=”user”, parts=[ types.Part( text=( “I want to buy SKU-A100 (wireless headphones). ” “Is it in stock, and how fast can it ship to ZIP 90210?” ) ) ], ), resp_parallel.candidates[0].content, types.Content(role=”user”, parts=fn_response_parts), ] resp_parallel_2 = client.models.generate_content( model=TOOL_COMBO_MODEL, contents=history_parallel, config=types.GenerateContentConfig( tools=[ types.Tool( function_declarations=[lookup_inventory, get_shipping_estimate], ), ], ), ) print(“n Final answer:n”) for part in resp_parallel_2.candidates[0].content.parts: if hasattr(part, “text”) and part.text: wrap(part.text) We declare two custom functions, lookupInventory and getShippingEstimate, and send a prompt that naturally triggers both in a single turn. We observe that Gemini assigns each function call a unique ID, which we carefully match when constructing our simulated responses for inventory availability and shipping speed. We then pass the complete history back to the model and receive a final answer that seamlessly combines both results into a customer-ready response. Copy CodeCopiedUse a different Browserheading(“DEMO 3: Grounding with Google Maps — Location-Aware Responses”) print(“”” Grounding with Google Maps connects Gemini to real-time Maps data: places, ratings, hours, reviews, and directions. Pass lat/lng for hyper-local results. Available on Gemini 2.5 Flash / 2.0 Flash (free). “””) time.sleep(2) print(” 3a: Finding restaurants near a specific location…n”) maps_response = client.models.generate_content( model=MAPS_MODEL, contents=”What are the best Italian restaurants within a 15-minute walk from here?”, config=types.GenerateContentConfig( tools=[types.Tool(google_maps=types.GoogleMaps())], tool_config=types.ToolConfig( retrieval_config=types.RetrievalConfig( lat_lng=types.LatLng(latitude=34.050481, longitude=-118.248526), ) ), ), ) print(” Generated Response:n”) wrap(maps_response.text) if grounding := maps_response.candidates[0].grounding_metadata: if grounding.grounding_chunks: print(f”n {‘─’ * 50}”) print(” Google Maps Sources:n”) for chunk in grounding.grounding_chunks: if hasattr(chunk, “maps”) and chunk.maps: print(f” • {chunk.maps.title}”) print(f” {chunk.maps.uri}n”) time.sleep(2) print(f”n{‘─’ * 72}”) print(” 3b: Asking detailed questions about a specific place…n”) place_response = client.models.generate_content( model=MAPS_MODEL, contents=”Is there a cafe near the corner of 1st and Main that has outdoor seating?”, config=types.GenerateContentConfig( tools=[types.Tool(google_maps=types.GoogleMaps())], tool_config=types.ToolConfig( retrieval_config=types.RetrievalConfig( lat_lng=types.LatLng(latitude=34.050481, longitude=-118.248526), ) ), ), ) print(” Generated Response:n”) wrap(place_response.text) if grounding := place_response.candidates[0].grounding_metadata: if grounding.grounding_chunks: print(f”n Sources:”) for chunk in grounding.grounding_chunks: if hasattr(chunk, “maps”) and chunk.maps: print(f” • {chunk.maps.title} → {chunk.maps.uri}”) time.sleep(2) print(f”n{‘─’ * 72}”) print(” 3c: Trip planning with the Maps widget token…n”) trip_response = client.models.generate_content( model=MAPS_MODEL, contents=( “Plan a day in San Francisco for me. I want to see the ” “Golden Gate Bridge, visit a museum, and have a nice dinner.” ), config=types.GenerateContentConfig( tools=[types.Tool(google_maps=types.GoogleMaps(enable_widget=True))], tool_config=types.ToolConfig( retrieval_config=types.RetrievalConfig( lat_lng=types.LatLng(latitude=37.78193, longitude=-122.40476), ) ), ), ) print(” Generated Itinerary:n”) wrap(trip_response.text) if grounding := trip_response.candidates[0].grounding_metadata: if grounding.grounding_chunks: print(f”n Sources:”) for chunk in grounding.grounding_chunks: if hasattr(chunk, “maps”) and chunk.maps: print(f” • {chunk.maps.title} → {chunk.maps.uri}”) widget_token = getattr(grounding, “google_maps_widget_context_token”, None) if widget_token: print(f”n Widget context token received ({len(widget_token)} chars)”) print(f” Embed in your frontend with:”) print(f’ ‘) print(f’ ‘) We switch to gemini-2.5-flash and enable Grounding with Google Maps to run three location-aware sub-demos back-to-back. We query for nearby Italian restaurants using downtown Los Angeles coordinates, ask a detailed question about outdoor seating at a specific intersection, and generate a full-day San Francisco itinerary complete with grounding sources and a widget context token. We print every Maps source URI and title returned in the grounding metadata, showing how easy it is to build citation-rich, location-aware applications. Copy CodeCopiedUse a different Browserheading(“DEMO 4: Full Agentic Workflow — Search + Custom Function”) print(“”” This combines Google Search grounding with a custom booking function, all in one request. Context circulation lets the model use Search results to inform which function to call and with what arguments. Scenario: “Find a trending restaurant in Austin and book a table.” “””) time.sleep(2) book_restaurant = types.FunctionDeclaration( name=”bookRestaurant”, description=”Book a table at a restaurant.”, parameters=types.Schema( type=”OBJECT”, properties={ “restaurant_name”: types.Schema( type=”STRING”, description=”Name of the restaurant” ), “party_size”: types.Schema( type=”INTEGER”, description=”Number of guests” ), “date”: types.Schema( type=”STRING”, description=”Reservation date (YYYY-MM-DD)” ), “time”: types.Schema( type=”STRING”, description=”Reservation time (HH:MM)” ), }, required=[“restaurant_name”, “party_size”, “date”, “time”], ), ) print(” Turn 1: Complex multi-tool prompt…n”) agent_response_1 = client.models.generate_content( model=TOOL_COMBO_MODEL, contents=( “I’m staying at the Driskill Hotel in Austin, TX. ” “Find me a highly-rated BBQ restaurant nearby that’s open tonight, ” “and book a table for 4 people at 7:30 PM today.” ), config=types.GenerateContentConfig( tools=[ types.Tool( google_search=types.GoogleSearch(), function_declarations=[book_restaurant], ), ], tool_config=types.ToolConfig( include_server_side_tool_invocations=True, ), ), ) print(” Returned parts:n”) fc_ids = describe_parts(agent_response_1) booking_call_id = fc_ids.get(“bookRestaurant”) if booking_call_id: print(f”n Turn 2: Simulating booking confirmation…n”) history_agent = [ types.Content( role=”user”, parts=[ types.Part( text=( “I’m staying at the Driskill Hotel in Austin, TX. ” “Find me a highly-rated BBQ restaurant nearby that’s ” “open tonight, and book a table for 4 people at 7:30 PM today.” ) ) ], ), agent_response_1.candidates[0].content, types.Content( role=”user”, parts=[ types.Part( function_response=types.FunctionResponse( name=”bookRestaurant”, response={ “status”: “confirmed”, “confirmation_number”: “BBQ-2026-4821”, “message”: “Table for 4 confirmed at 7:30 PM tonight.”, }, id=booking_call_id, ) ) ], ), ] agent_response_2 = client.models.generate_content( model=TOOL_COMBO_MODEL, contents=history_agent, config=types.GenerateContentConfig( tools=[ types.Tool( google_search=types.GoogleSearch(), function_declarations=[book_restaurant], ), ], tool_config=types.ToolConfig( include_server_side_tool_invocations=True, ), ), ) print(” Final agent response:n”) for part in agent_response_2.candidates[0].content.parts: if hasattr(part, “text”) and part.text: wrap(part.text) else: print(“n Model did not request bookRestaurant — showing text response:n”) for part in agent_response_1.candidates[0].content.parts: if hasattr(part, “text”) and part.text: wrap(part.text) We combine Google Search with a custom bookRestaurant function to simulate a realistic end-to-end agent scenario set in Austin, Texas. We send a single prompt to Gemini, asking it to find a highly rated BBQ restaurant near the Driskill Hotel and book a table for four. We inspect the returned parts to see how the model first searches the web and then calls our booking function with the details it discovers. We close the loop by supplying a simulated confirmation response and letting Gemini deliver the final reservation summary to the user. Copy CodeCopiedUse a different Browserheading(“DEMO 5: Context Circulation — Code Execution + Search + Function”) print(“”” Context circulation preserves EVERY tool call and response in the model’s context, so later steps can reference earlier results. Here we combine: • Google Search (look up data) • Code Execution (compute something with it) • Custom function (save the result) The model chains these tools autonomously using context from each step. “””) time.sleep(2) save_result = types.FunctionDeclaration( name=”saveAnalysisResult”, description=”Save a computed analysis result to the database.”, parameters=types.Schema( type=”OBJECT”, properties={ “title”: types.Schema(type=”STRING”, description=”Title of the analysis”), “summary”: types.Schema(type=”STRING”, description=”Summary of findings”), “value”: types.Schema(type=”NUMBER”, description=”Key numeric result”), }, required=[“title”, “summary”, “value”], ), ) print(” Turn 1: Research + compute + save (3-tool chain)…n”) circ_response = client.models.generate_content( model=TOOL_COMBO_MODEL, contents=( “Search for the current US national debt figure, then use code execution ” “to calculate the per-capita debt assuming a population of 335 million. ” “Finally, save the result using the saveAnalysisResult function.” ), config=types.GenerateContentConfig( tools=[ types.Tool( google_search=types.GoogleSearch(), code_execution=types.ToolCodeExecution(), function_declarations=[save_result], ), ], tool_config=types.ToolConfig( include_server_side_tool_invocations=True, ), ), ) print(” Parts returned (full context circulation chain):n”) fc_ids = describe_parts(circ_response) save_call_id = fc_ids.get(“saveAnalysisResult”) if save_call_id: print(f”n Turn 2: Confirming the save operation…n”) history_circ = [ types.Content( role=”user”, parts=[ types.Part( text=( “Search for the current US national debt figure, then use code ” “execution to calculate the per-capita debt assuming a population ” “of 335 million. Finally, save the result using the ” “saveAnalysisResult function.” ) ) ], ), circ_response.candidates[0].content, types.Content( role=”user”, parts=[ types.Part( function_response=types.FunctionResponse( name=”saveAnalysisResult”, response={“status”: “saved”, “record_id”: “analysis-001″}, id=save_call_id, ) ) ], ), ] circ_response_2 = client.models.generate_content( model=TOOL_COMBO_MODEL, contents=history_circ, config=types.GenerateContentConfig( tools=[ types.Tool( google_search=types.GoogleSearch(), code_execution=types.ToolCodeExecution(), function_declarations=[save_result], ), ], tool_config=types.ToolConfig( include_server_side_tool_invocations=True, ), ), ) print(” Final response:n”) for part in circ_response_2.candidates[0].content.parts: if hasattr(part, “text”) and part.text: wrap(part.text) else: print(“n Model completed without requesting saveAnalysisResult.”) for part in circ_response.candidates[0].content.parts: if hasattr(part, “text”) and part.text: wrap(part.text) heading(” ALL DEMOS COMPLETE”) print(“”” Summary of what you’ve seen: 1. Tool Combination — Google Search + custom functions in one call 2. Tool Response IDs — Unique IDs for parallel function call mapping 3. Maps Grounding — Location-aware queries with real Maps data 4. Agentic Workflow — Search + booking function with context circulation 5. Context Circulation — Search + Code Execution + custom function chain Key API patterns: ┌──────────────────────────────────────────────────────────────────┐ │ tools=[types.Tool( │ │ google_search=types.GoogleSearch(), │ │ code_execution=types.ToolCodeExecution(), │ │ function_declarations=[my_func], │ │ )] │ │ │ │ tool_config=types.ToolConfig( │ │ include_server_side_tool_invocations=True, │ │ ) │ │ │ └──────────────────────────────────────────────────────────────────┘ Models: • Tool combination: gemini-3-flash-preview (Gemini 3 only) • Maps grounding: gemini-2.5-flash / gemini-2.5-pro / gemini-2.0-flash • Both features use the FREE tier with rate limits. Docs: “””) We push context circulation to its fullest by chaining three tools, Google Search, Code Execution, and a custom saveAnalysisResult function, in a single request that researches the US national debt, computes the per-capita figure, and saves the output. We inspect the full chain of returned parts, toolCall, toolResponse, executableCode, codeExecutionResult, and functionCall, to see exactly how context flows from one tool to the next across a single generation. We wrap up by confirming the save operation and printing a summary of every key API pattern we have covered across all five demos. In conclusion, we now have a practical understanding of the key patterns that power agentic workflows in the Gemini API. We see that the include_server_side_tool_invocations flag on ToolConfig is the single switch that unlocks tool combination and context circulation, which returns all parts, including thought_signature fields, verbatim in our conversation history, is non-negotiable for multi-turn flows, and that matching every function_response.id to its corresponding function_call.id is what keeps parallel execution reliable. We also see how Maps grounding opens up an entire class of location-aware applications with just a few lines of configuration. From here, we encourage extending these patterns by combining URL Context or File Search with custom functions, wiring real backend APIs in place of our simulated responses, or building conversational agents that chain dozens of tools across many turns. Check out the Full Codes here. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post How to Combine Google Search, Google Maps, and Custom Functions in a Single Gemini API Call With Context Circulation, Parallel Tool IDs, and Multi-Step Agentic Chains appeare […]

admin wrote a new post

Wed, 08 Apr 2026 00:15:31 +0000

Desalination plants in the Middle East are increasingly vulnerableMIT Technology Review Explains: Let our writers untangle the complex, messy […]

admin wrote a new post

Wed, 08 Apr 2026 00:07:01 +0000

Enabling agent-first process redesign Unlike static, rules-based systems, AI agents can learn, adapt, and optimize processes dynamically. As they […]