Papers, code and results
Everything we report comes from our own runs on public data, with the conditions stated. Where a result goes against us, we say so.
Publications
CNet: A Complex-Valued Deep Learning Framework with Wirtinger Autodifferentiation and FFT–Hadamard Convolution
The framework: complex layers, exact Wirtinger gradients, GPU kernels checked against CPU references.
Read on arXiv →Born Attention is Linear: Exact Sub-Quadratic Attention and Constant-State Inference from Quantum Measurement
The squared complex inner product has an exact finite feature map, so Born attention runs in linear time with a constant-size state and keeps rotary positions exactly.
Read the PDF →Streaming Is Not Memory: Constant-State Born-Attention Byte Models on PG-19
Separates streaming, quality preservation and long-range use, and measures each. Includes a matched Transformer comparison and an ablation of where the gap comes from.
Read the PDF → Summary on the blog →Measured so far
| Result | Number | What it does not show yet |
|---|---|---|
| Radio modulation recognition, RML22, 11 classes, all signal-to-noise levels, complex model on raw IQ | 0.597 accuracy | Not yet a matched comparison with real-valued models. On RML2016.10a: 0.515. |
| PG-19 test set read in 5,242,880-byte windows, one pass each, 12M complex parameters | 1.312 bits/byte, 12.8 MB state | Useful memory is currently a few thousand bytes. |
| Gap to a standard Transformer of the same size on the same text, before and after one initialisation fix | 0.079 → 0.056 bits/byte | Architecture not yet tuned. One run each, one size. |
| Decoding memory at a one-million-token context, one attention layer, against a Transformer's key/value cache | 6.4 MB vs 3.2 GB | Memory, not speed: our kernels are young. |
Code and models
CNet
C++/CUDA framework for complex-valued networks, MIT licence. Born attention, gated linear attention, streaming decoders, and a one-file API to train your own model.
GitHub →Born-attention language model
A small pretrained fully complex model, ready to sample from with CNet.
Hugging Face →Surprise map
Per-byte surprise of a streaming model over a whole book, with the passages and errors it flags.
Read the post →