Needle 2: 45M-Parameter Tool-Calling Model Released

Achieving Tool Calling with 45M Parameters

On August 16, 2026, Cactus Compute published a repository called “needle” on GitHub Trending. The company’s Needle 2 is a 45M-parameter open-source model specializing in tool calling, device usage, and structured extraction. According to the provided information, the entire model is distributed as a single 14MB binary, and the whole session can run in approximately 28MB of RAM.

While conventional LLMs require several gigabytes to tens of gigabytes of memory, Needle 2 is designed for execution on edge devices and embedded environments. It is provided as a Python package and can be installed with “pip install cactus-needle.” The inference engine is fetched from Hugging Face only once and cached. Documentation also provides setup instructions for offline environments or devices with network connectivity severed.

Simple Attention Network Architecture

The technical foundation of Needle 2 is a method called “Simple Attention Network.” According to the published information, this architecture has the following characteristics: Hadamard MLP replaces the conventional FFN, and GQA Attention, engram key-value memory, and multi-lane hyper-connections are incorporated.

Each block contains an update rule, applying flattening of four RMS-normalized residual streams, a Walsh-Hadamard transform, (k_t, v_t) rows gathered from an n-gram table, and doubly stochastic normalization computed via Sinkhorn iterations. The Attention and MLP residuals are both sandwich-normed and gated, and engram sites fire in two layers.

Compression with CQ2-bit is also an important feature. This method, called Cactus Quants, compresses the model to 2-bit precision. In published benchmarks, it competes on par with other small models such as FunctionGemma 270M, LFM2.5 230M, and Apple FM. Whereas those models use f16 precision, Needle 2 achieves 2-bit precision at a size 5 to 70 times smaller.

Tool Calling Implementation and Constraints

Needle 2’s tool calling functionality is implemented with a simple contract: text goes in, JSON comes out. Specifically, it reads tool descriptions and determines which tool to call and what arguments to input. A byte-level grammar is compiled from the schema and constrains each generated token.

A mechanism called “confidence-gated” is also introduced. Every response is assigned a calibrated confidence score from a learned head. A threshold can be set: if the confidence exceeds the threshold, the action is executed; if it falls below, escalation occurs.

A tool search feature is also included. When a large tool catalog is declared, a built-in search head renders only the top five tools per turn. The grammar is restricted to that subset.

Memory usage is managed with a 256-token sliding window. Tools are pinned as KV sinks, keeping total memory usage at approximately 28MB even as conversations grow longer.

Provided as a Python Package

Developers can define tools in Python code and integrate Needle 2. They apply a decorator to functions, specify argument types in the signature, and describe the tool in the docstring. The run() method completes the loop: the model selects calls, Needle 2 executes the functions, feeds back the results, and returns a final response.

In the provided sample code, tools are defined using the @needle.tool decorator. The model selects tool calls, and the executed tool results are returned as results.

Benchmark Comparison

According to the published information, Needle 2, with its 45M parameters, delivers performance on par with FunctionGemma 270M, LFM2.5 230M, and Apple FM. Those models have 270M and 230M parameters and operate at f16 precision. Needle 2, despite using 2-bit precision, has 5 to 70 times fewer parameters.

This comparison suggests advances in model compression technology. It indicates that 2-bit compression with CQ2-bit can potentially achieve performance comparable to much larger models.

Impact on Edge AI

The arrival of Needle 2 could have an impact on the edge AI field. The 14MB binary and 28MB RAM requirement increase feasibility on embedded systems and IoT devices. Tool calling and structured extraction become possible in resource-constrained environments such as smartphones, embedded systems, and industrial equipment.

The paper published by Cactus Compute (arXiv:2607.18363) describes the design of Simple Attention Network and ablation experiments. Technical details are left to that paper, but the provided information lists Hadamard MLP, GQA Attention, and engram key-value memory as key components.

Editorial Opinion

In the short term, the arrival of Needle 2 brings new options to the edge AI market. The low memory requirement of 28MB makes local AI inference on devices that were previously impossible a reality. Integration into smart home devices, industrial sensors, or low-spec embedded systems may accelerate.

In the long term, the practical application of 2-bit compression technology could change the standards for AI chip design and model distribution. An era may arrive in which we move away from heavy cloud dependency and can perform advanced inference locally while preserving privacy. Whether Cactus Quants is extended to other model architectures will determine the proliferation of this technology.

The reliability and versatility of this technology still require verification. Although benchmarks show it on par with existing models, its tool calling accuracy in real-world environments and its ability to handle edge cases remain unknown. How much information loss from 2-bit compression is acceptable, and which tasks it can be applied to, await validation by the developer community.

Related Articles

• Cat Catch 2.7.1 Strengthens m3u8 Download Functionality
• Microsoft Defender Privilege Escalation Vulnerability “RoguePlanet” Disclosed

References

• “cactus-compute /

  needle", by **cactus-compute** — GitHub Trending, 2026-08-16 (ARR)

• Source URL: https://github.com/cactus-compute/needle