Ternlight: 7MB Embedding Model Runs in Browser via WASM

Ternlight Overview

Ternlight provides a 7 MB embedding model that executes inside browsers using WebAssembly. The package exposes a single embed() call built on ternary BitLinear weights and standard transformer blocks. It appeared on Hacker News with a working demo that indexes React documentation chunks and returns query embeddings on the client CPU. No server round-trips are required after the initial WASM load.

Model Architecture Details

The released @ternlight/mini variant keeps the embedding layer as a linear projection followed by BitLinear ternary weights. Transformer layers use a reduced d_model dimension and a modest number of attention heads to stay inside the 7 MB budget. Maximum sequence length is capped to keep activation memory low during browser execution.

Inference runs entirely on the main thread or a worker via WASM bindings. After the first load the engine stays cached, so subsequent page visits show zero network transfer for the model itself. Throughput numbers reported in the demo reach several hundred embeddings per second on mid-range laptop CPUs once the WASM module is initialized.

The ternary representation replaces 16-bit or 8-bit weights with values constrained to {-1, 0, +1}. This choice removes most multiply-accumulate operations in favor of additions and subtractions, which explains the small binary size and the measured single-embed latency in milliseconds.

Practical Trade-offs for Client-side Search

Developers can drop the package into a Next.js or plain React application and compute embeddings for user-provided text without touching an external embedding service. The approach removes API keys, rate limits, and data egress concerns. State can be shared across components by storing the embedding vectors in a simple context or a lightweight store, avoiding prop drilling for search inputs.

Accuracy is the main concession. Ternary quantization reduces representational capacity compared with full-precision models of similar depth. On the React docs corpus the demo shows usable nearest-neighbor results, yet recall on nuanced queries falls behind larger float models hosted on a server. Latency remains consistent because the entire forward pass stays on-device; the reported embed times do not include network variance.

Bundle impact stays modest. The WASM binary plus JavaScript glue adds roughly the stated 7 MB to the initial download. After that, memory usage stays bounded because the model reuses the same buffers for each embed() call. Teams that already ship several megabytes of client assets will notice the increase, while lighter sites may prefer to keep embeddings on the server.

Integration Patterns

Mount the engine once at application startup and expose the embed function through a small wrapper. Components that need semantic search can call it directly on form submit or on debounced input. Because the model is deterministic after initialization, results can be memoized by query string to skip repeated work.

For larger document collections the same WASM instance can embed both the corpus and the query. Chunking logic stays in JavaScript; only the resulting vectors are passed to whatever client-side index the application already uses. No changes to existing React component trees are required beyond the single initialization step.

FAQs

How does ternary quantization affect embedding quality? It reduces precision relative to 8-bit or 16-bit models, which can lower recall on complex queries while preserving enough signal for many documentation search tasks.

What is the measured single-embed latency? The demo reports single embed() calls in the low tens of milliseconds on typical laptop CPUs once the WASM runtime is warm.

Can the model run offline after the first visit? Yes. The package caches the WASM binary and weights locally, so subsequent loads require no network requests for the model itself.

---

📖 Related articles

• Agentic Coding: Una Trappola per lo Sviluppo Software?
• Claude Code: da prompt a agente con skills e subagent
• File agents.md: utili per gli agenti di coding?