gnata-sqlite
Explanation

Architecture

How gnata-sqlite's layers fit together — from Go engine to SQLite extension to WASM LSP to CodeMirror editor

gnata-sqlite is a stack: Go engine, SQLite extension, language server, editor component, and playground. Each layer exists because the one below it created a need.

Layer diagram

@gnata-sqlite/playground
Vite + React — SQLite mode and gnata expression evaluator
uses
@gnata-sqlite/react
Composable hooks and components
sql.js + gnata extension
WASM SQLite with loadable extension
wraps
@gnata-sqlite/codemirror
Syntax highlighting + async WASM LSP
sqlite/ extension
CGo, c-shared, query planner
compiles to
gnata-lsp.wasm
TinyGo — 380 KB (145 KB gzipped)
gnata.wasm
Standard Go WASM — eval engine
powered by
gnata core engine
Pure Go JSONata 2.x — GJSON fast path + full AST evaluator
schema JSON flows from backend into editor autocomplete

Core engine

Pure Go JSONata 2.x implementation. Single external dependency: tidwall/gjson. Minimal dependencies keep CGo and TinyGo builds feasible.

Two-tier evaluation

TierHandlesHow
GJSON fast pathDotted paths (a.b.c), path-vs-literal comparisons, built-in functions on pure pathsZero-copy scan, 10M+ ops/sec
Full AST interpreterLambdas, higher-order functions, sorting, grouping, conditionals, everything elseStandard tree-walking evaluator

Compile() runs AnalyzeFastPath on the AST. If eligible, evaluation bypasses the interpreter entirely. Both tiers share the same Expression type and Eval method — callers never choose.

StreamEvaluator

For batch workloads. Manages compiled expressions, evaluates all against each incoming event.

  • Schema-aware caching — builds a GroupPlan per distinct JSON schema (set of fields present), caches which expressions qualify for GJSON fast path
  • Lock-free reads — copy-on-write atomic pointer for EvalMany
  • Bounded LRU — evicts least-recently-used schemas when capacity is exceeded

SQLite extension

Built with Go's -buildmode=c-shared, speaks SQLite's loadable extension API through a C bridge.

Registered functions

FunctionTypePurpose
jsonata(expr, json)scalarEvaluate expression per row
jsonata_query(expr, json)aggregateCollect rows, evaluate over array
jsonata_each(json, expr)table-valuedExpand array/object result into rows
jsonata_set(json, path, value)mutationSet value at dotted path
jsonata_delete(json, path)mutationRemove key at dotted path

All five share an expression cache (sync.Map keyed by expression string) — SQLite typically calls the same expression across thousands of rows.

See the SQLite functions reference for full signatures.

TinyGo WASM LSP

Why not rewrite in JS? Two parsers means divergent behavior. The Go parser has 1,778 test cases — rewriting risks losing that coverage.

One codebase, two targets

FileTargetTransport
main_wasm.goTinyGo WASM (browser)syscall/js global functions
main_lsp.goNative binary (VS Code, Neovim)JSON-RPC 2.0 over stdin/stdout
diagnostics.go, completions.go, hover.go, schema.go, funcinfo.go, marshal.goBothShared logic

Why TinyGo? Standard Go WASM: ~5.3 MB. TinyGo: 380 KB (14x smaller). For a module that loads on every page visit, that is the difference between instant and a loading spinner.

Trade-off: No reflect means no encoding/json. marshal.go is a hand-written, reflect-free JSON serializer. More code to maintain, but avoids runtime bloat.

See TinyGo WASM for the full build pipeline.

CodeMirror 6 editor

@gnata-sqlite/codemirror provides JSONata language support with a two-tier architecture:

TierWhatDependency
Sync highlightingLezer grammar, instant on mountNone
Async WASM intelligenceDiagnostics, autocomplete, hover docsgnata-lsp.wasm

Degrades gracefully: if WASM fails to load, highlighting and bracket matching still work.

Context-aware autocomplete: Lexes from position zero to cursor, classifies context (function name, field path, or top-level identifier), then suggests built-in functions or walks the schema tree to suggest nested keys. Type Account. and see the available fields immediately.

See the editor API reference.

The playground

@gnata-sqlite/playground — Vite + React app, two modes:

  • SQLite mode — sql.js + gnata WASM extension, full SQL environment with jsonata() in queries
  • gnata mode — pure expression evaluator via @gnata-sqlite/react, same hooks and components available for embedding

Backend-driven autocomplete via schemas

In production, data might be millions of rows — shipping it to the browser for autocomplete is not an option.

Solution: a schema protocol. Lightweight JSON describing document structure:

{"fields":{"id":{"type":"number"},"data":{"type":"object","fields":{"name":{"type":"string"}}}}}
  • Backend generates the schema (sampling, inference, etc.)
  • Frontend passes it to initWasm() or CodeMirror completion config
  • WASM LSP walks the schema tree to suggest fields at cursor position
  • The editor never sees actual data

Why this shape

Each layer emerged from a concrete need:

  1. Fast hot-path evaluation in SQLite → GJSON fast path
  2. JSONata in SQL → CGo extension with expression cache
  3. Extension hard to use without feedback → LSP
  4. LSP in browsers → TinyGo WASM (380 KB, 145 KB gzipped)
  5. WASM needs an editor → CodeMirror package
  6. Production editors need data awareness without data → schema protocol

Each decision constrains the next. Single-dependency engine makes TinyGo feasible. TinyGo makes small WASM possible. Small WASM makes per-page-load acceptable.

On this page