---
title: "I put a Fly Brain in an RC car to sniff WiFi"
description: "I wired a simulated fly brain to an RC car, turning WiFi into smell and neural spikes into motion. What worked, what broke, and what still needed my code."
author: "Kevin Pham"
published: "2026-10-11"
updated: "2026-10-20"
category: "Systems"
tags: ["Robotics"]
url: "https://deoxy.net/blog/a-fly-brain-drove-an-rc-car-toward-my-wi-fi/"
---

# I put a Fly Brain in an RC car to sniff WiFi

comma's seventh hackathon ran for two days in September. I spent them getting a fruit fly's brain to drive a toy car toward a Wi-Fi access point.

It worked, in the honest sense of the word. The car crawled around a room, veered off walls before touching them, slammed into reverse when it hit one anyway, drifted up the signal gradient toward an access point named _Flash_, and buzzed through the comma's speaker the whole time. It was also derpy: it meandered like something with better things to do, and the single most important control signal — the one that turns the wheel toward the AP — was the least neural part of the stack. I'll get to that, because it's the interesting part.

The premise is simple. Take a published connectome, simulate every neuron in it, then wire real sensors to real sensory neurons and real actuators to real motor neurons. Train nothing. Put no policy in the middle. Inject at the periphery, read at the periphery, and let the wiring do what the wiring does.

## The hardware, which is the strange part

The compute is a **comma 4** — a Qualcomm SDM845 dashcam-shaped Linux box with 3.5 GB of RAM — paired with an external AMD GPU that comma calls a **chestnut**. The chestnut is the part worth pausing on: a desktop-class AMD card connected over **USB3 → PCIe** through an ASM2464 bridge, driven _entirely from userspace by tinygrad_. There is no kernel PCI device, no DRM node, no amdgpu; `/sys/bus/pci/devices` is empty. tinygrad pokes the card's registers over USB and schedules kernels on it directly.

It works about as well as that description suggests, which is to say it works: PCIe link up at LTSSM `0x78`, USB negotiated at 5 Gbps, \~417 GFLOP/s FP32 steady-state on a 2048³ matmul. It also has opinions. Uploads have to be staged in 16 MB slices off a \~312 MB host-visible BAR window; a raw 4 MB PCIe zeroing pass took 1.164 s and blew through the transport's 1-second timeout, which looked exactly like VRAM exhaustion until I measured it; and tinygrad's BEAM autotuner is forbidden here, because its compile-time probing wedges the GPU.

The car is a Hiwonder MentorPi — an Ackermann-steered toy chassis, 1.2 m/s top speed, 29° of steering lock, 145 mm wheelbase — with a _second_ comma 4 bolted on as the body. It has no eGPU of its own: the chestnut needs a 12 V supply the toy can't carry.

_Image: The RC car on a table, with a comma 4 mounted at the front, a taped-down power bank on top, and the chassis electronics and wiring exposed below\._

The car half of the rig. The fly simulation runs on the separate, wall-powered comma and chestnut; the camera, radio, wheels, and speaker travel with this chassis.

## What "running a fly brain" means here

The model is **MaleCNS v1.0**: 166,700 neurons and 24,469,412 signed synapses covering the brain and ventral nerve cord, released CC-BY by FlyEM/Janelia and collaborators. Each neuron is leaky integrate-and-fire with the Shiu et al. 2024 constants, `dt` = 0.1 ms, an 18-tick axonal delay, and weights that are signed synaptic contact counts scaled by 0.275 mV.

Most of the project went into one number: throughput. The final acceptance run completed 118.008 biological seconds in 59.7895 wall seconds with both cameras, Wi-Fi, IMU, and a browser viewer attached: 1.97× real time.

Faster than real time, on a card talking over a USB cable, with a phone SoC generating the input. Getting there meant replacing dense Poisson input generation (403 ms per 36 ms biological window — a 0.09× ceiling by itself) with sparse event sampling at 3 ms, bit-packing the spike delay ring into `uint32`, and moving viewer JSON formatting into its own process after timing found 97 ms of display preparation in the critical path.

The guardrail was bit-exactness. This model is chaotic at the ulp level: one reference neuron crosses threshold by **7.6 × 10⁻⁶ mV** after a hundred ticks of pure leak, and a single flipped spike diverges the whole network by 21% of total spike count within ten thousand ticks. So every optimization had to reproduce the reference second exactly — all 35,113 spikes over 9,990 ticks, every timestamp. It caught real bugs, including one where strict-float mode silently disabled tinygrad's parallel _compilation_, so the spawned workers lost the no-FMA renderer patch.

What's missing: the upstream constants were fitted on the FlyWire _female_ brain and reused unchanged on MaleCNS, and the pack keeps only acetylcholine, GABA, and glutamate edges. That drops **histamine** — 7,891 neurons, including every photoreceptor — which is fast ionotropic chloride with a known sign and should have been kept. It's also why the rig injects vision one synapse downstream, at the lamina, rather than at R1–R8.

## Hardware in, neurons out

This is the part I came for. Every input drives specific MaleCNS neurons by body ID, not an abstraction of them.

| Population | Cells | Hardware source |
| - | -: | - |
| **L1 / L2 / L3** lamina | 5,327 | camera luma: ON contrast, OFF contrast, sustained brightness |
| **ORN** antennal lobe | 2,635 | Wi-Fi scan — SSID hashed to 1 of 53 glomeruli, RSSI sets the rate |
| **JO** Johnston's organ | 672 | IMU accel/gyro, mic sound pressure, wheelspeed as wind |
| **campaniform sensilla** | 426 | motor strain: throttle commanded while the IMU reports no motion |
| **PFN** goal vector | 456 | Wi-Fi bearing, injected as a goal direction |
| **TRN** thermoreceptors | 25 | `/sys/class/thermal` — the SoC's own board temperature |
| **BM\_Vib** bristles | 22 | IMU jolt, as nociceptive drive |
| **OCG** ocelli | 22 | mean luma of the top camera rows, i.e. sky |
| **sweet GRN** | 17 | charging state — docking is eating |

**Diagram: Where the hardware touches the fly**

Car hardware writes the fly's sensory periphery and is read back at its motor periphery. Camera luma pooled onto the ommatidial lattice drives the lamina's ON, OFF, and luminance channels; a Wi-Fi scan selects an antennal-lobe glomerulus and its signal strength sets the firing rate; the IMU, microphone, motor strain, battery, and board temperature drive Johnston's organ, sensilla, bristles, and insulin cells. Vision continues through the medulla and lobula loom detectors into the central brain, which is untouched connectome. Descending-neuron spike rates are read out as throttle, reverse, and an escape latch that drives the wheels, while ventral-nerve-cord wing motor neurons drive the car's speaker.

_Only the two ends are instrumented. Nothing between the sensory and motor periphery is injected, read, or fed back._

```mermaid
flowchart LR
  camera["Car camera luma"] -->|"pooled onto the ommatidial lattice"| lamina["Lamina<br/>L1 ON / L2 OFF / L3 luminance"]
  wifi["Wi-Fi scan"] -->|"SSID selects a glomerulus, RSSI sets the rate"| antennal["Antennal lobe<br/>2,635 receptor neurons"]
  body["IMU, microphone, strain, battery, board temperature"] -->|"mechanical and interoceptive drive"| afferents["Johnston's organ, sensilla, bristles, insulin cells"]

  lamina -->|"parallel visual channels"| medulla["Medulla"]
  medulla -->|"object and loom detection"| lobula["Lobula<br/>LC4 / LPLC2 / LC15"]
  lobula --> central["Central brain<br/>untouched connectome"]
  antennal --> central
  afferents --> central

  central -->|"command signals"| dns["Descending neurons<br/>DNp09 / MDN / DNp01"]
  central -->|"motor commands"| vnc["Ventral nerve cord motor neurons<br/>wing power and steering muscles"]

  dns -->|"spike rates read out"| drive["Throttle, reverse, escape latch"]
  drive --> motors["Wheels and steering"]
  vnc -->|"wingbeat becomes sound"| speaker["Car speaker"]
```

The retina is the mapping I'm happiest with. Camera luma is pooled onto the eye's _real_ ommatidial hex lattice, parsed out of the MaleCNS annotations, so each of \~530 columns per eye drives its own L1, L2, and L3 cell. L1 and L2 are transients that decay toward zero — the lamina's actual band-pass behavior. Hold the camera still and the scene fades to its L3 floor; wave a hand and it blooms a bright trail that then vanishes.

A few of these do what the tissue does; most don't, and the distinction is the credibility of the whole project, so I sorted every mapping into three labeled classes. **Mechanism**: the hardware performs the same transduction as the tissue. **Homolog**: the sensor measures the same physical quantity the organ measures — a camera and an eye both measure photons. **Substitution**: a different modality standing in for one the car doesn't have.

**Chart: What each eye sees, and what the lamina does with it**

Both hemifield eyes on the rig's real ommatidial sampling lattice, each shown on the three lamina channels. Drag the synthetic dark disc across the combined field: the sustained channel tracks brightness, the ON and OFF channels light only at the disc's moving edges, and one eye loads far harder than the other whenever the disc sits off centre.

_The lattice and its per-cell weights are measured; the response is the documented ON/OFF/sustained rule applied to a synthetic stimulus. No camera frame and no spike train is plotted._

The three lamina channels, what drives each in the rig, and what the modelled panels show. Marks are sized by how many L1 neurons read that sampling cell.

| Channel | Driven by | In the figure |
| - | - | - |
| L3 · sustained luminance | absolute camera brightness, held while the scene is still | the disc as a standing dark patch; the only channel a motionless scene leaves lit |
| L1 · ON transient | positive frame-to-frame change, decaying toward zero | a bright leading edge ahead of the disc, and nothing at all once it stops |
| L2 · OFF transient | negative frame-to-frame change, decaying toward zero | a trailing edge behind the disc, mirroring L1 |
| Disc speed at zero | the two compared frames become identical, so the change term is exactly zero | both transient panels go silent while the sustained panels hold — the band-pass behaviour that makes a parked car nearly blind |
| Left eye lattice | 884 L1 neurons across 531 sampling cells, pooled 2x2 for display | 145 marks of measured sampling coverage; the sheared band is the hex lattice’s own shape, not an eye silhouette |
| Right eye lattice | 892 L1 neurons across 528 sampling cells, pooled 2x2 for display | 143 marks; one wide camera split left-half and right-half gives the two hemifields |

Two things in that figure matter to the rest of the story. The first is why a
still scene goes quiet: hold the disc anywhere and the ON and OFF panels fade
while only the sustained channel survives, which is the lamina's real band-pass
behaviour and the reason a parked car sees almost nothing. The second is the
asymmetry. One wide fisheye is split left-half and right-half into two eyes, so
anything off centre loads one side harder than the other — and that difference,
a few hundredths of a unit wide, is the entire signal a steering readout
downstream ever gets.

Wi-Fi-as-smell is a substitution, and the fun one. The comma has no chemical nose, but it has a radio one. Each SSID hashes to one of the 53 real antennal-lobe glomeruli and its RSSI sets that glomerulus's drive, so driving between access points shifts the _smell of the room_ across the antennal lobe. A 2×2 directional antenna makes it a gradient: the two chains are the fly's two antennae, and the delta between them is a bearing.

_Image: Wi-Fly viewer with sensory inputs on the left, measured MaleCNS cell-body positions in the centre, and motor readouts and an illustrative fly avatar on the right. The screenshot is labeled synthetic inputs._

The viewer's inputs → brain → outputs layout, captured by the project's UI test with **synthetic inputs**. This shows the interface, not a recorded drive or measured neural response. The dots are measured MaleCNS cell-body positions; the fly avatar is illustrative. The modified viewer is built with [fly-connectome-template](https://github.com/cobanov/fly-connectome-template) by [Mert Cobanov](https://github.com/cobanov), with MaleCNS data from FlyEM / HHMI Janelia and collaborators (CC BY 4.0) and the Flybody mesh (Apache-2.0).

## Reading the motor neurons back out

Nothing is fed back into the simulation — every output is a readout of descending-neuron spike rates, which keeps bit-exactness intact. Throttle is `DNp09` (forward walking) minus `MDN` (the moonwalker, backward walking). Loom detectors `LC4` and `LPLC2` feed the `DNp01` giant fibre, which latches a full-reverse escape. The bump reflex turned out to be nearly a dedicated line: driving the 22 `BM_Vib` vibration bristles at 200 Hz produces bilateral `MDN` bursts at 11.5 Hz with every other descending neuron silent, versus 2 Hz through the whole 672-neuron Johnston's organ. A sharp accelerometer transient becomes a hard reverse \~300 ms later, and it fires reliably on the real car.

Plotting those populations on their real cell-body positions makes the shape of the thing obvious: everything the car does arrives through a few hundred cells at the bottom of a 166,700-neuron brain, and the wheel commands come through twelve of them.

**Chart: Where the fly brain reaches the car**

Measured soma positions for the four populations the rig reads: the lobula detectors that report looming walls, the central complex steering cells, the twelve descending neurons carrying throttle and reverse and escape, and the wing motor neurons wired to the car speaker. Scroll moves attention down the funnel; every other soma is the untouched connectome.

_Positions are measured somaLocation values from MaleCNS v1.0 (CC BY 4.0). The sensory side is deliberately absent: those afferents' cell bodies sit in the eye and antenna, outside the imaged volume, so placing them would be invention._

The four read-out populations, what each contributes to the car, and how many of its cells the atlas can place. A body ID with no measured soma location is omitted rather than positioned.

| Population | Reads as | Plotted | In the rig |
| - | - | - | - |
| Lobula detectors — LC4, LPLC2, LC15 | objects and looming walls; feeds the swerve term and the escape latch | 437 | 437 |
| PFL3 — central complex steering output | heading against the Wi-Fi goal direction; the late, opt-in steering path | 24 | 24 |
| Descending neurons — DNa01, DNa02, DNp09, MDN, DNp01 | throttle from forward walking, reverse from the moonwalker, escape from the giant fibre | 12 | 12 |
| Wing motor neurons — DLM/DVM, b1–tp, TTMn, dMS2 | wingbeat and song; their missing partner is muscle, and the speaker is that partner | 78 | 80 |
| The rest of the measured atlas | untouched connectome, shown as a decimated background sample of every 23rd body ID | 6,057 | 139,473 |
| Sensory afferents — lamina and olfactory receptor neurons | not drawn: camera luma reaches 5,327 lamina cells and the Wi-Fi scan reaches 2,635 receptor neurons, but their somata sit outside the imaged volume | 0 | 7,962 |

**Diagram: From spike counts to wheels**

Descending-neuron spike counts pass through a per-group exponential average into a control law producing throttle, steering, and an escape latch. That result is published at 10 Hz on an event stream, read over localhost by the control bridge, and sent as drive frames at 20 Hz to the car control server, which issues serial commands to the motor board and the wheels. Two independent stops protect the car: the bridge watchdog commands zero speed on a stalled stream, and the server halts the car when no frame arrives within 500 milliseconds.

_The readout is one direction only. Nothing on this path is fed back into the simulation, which is what keeps it bit-exact._

```mermaid
flowchart LR
  counts["Descending neuron spike counts"] -->|"per-group exponential average"| law["Control law<br/>throttle, steer, escape"]
  law -->|"published at 10 Hz"| events["Event stream"]
  events -->|"read over localhost"| bridge["Control bridge"]
  bridge -->|"drive frame at 20 Hz"| server["Car control server"]
  server -->|"serial command"| board["Motor board"]
  board --> wheels["Wheels and steering"]

  bridge -->|"stalled stream commands zero speed"| watchdog["Bridge watchdog"]
  server -->|"no frame within 500 ms"| halt["Server stop"]
  watchdog --> wheels
  halt --> wheels
```

The mapping I'd defend hardest is the speaker. `motor_map` carries the ventral-nerve-cord motor neurons themselves — the DLM/DVM wingbeat oscillators and the b1–tp steering muscles — and they drive the comma's speaker directly. Fly song and fly buzz _are_ wing vibration, so this one is mechanism rather than metaphor. A speaker converting wing motor neuron activity into sound performs the same transduction the wing does.

It measures as a real signal: 0.5-second renders go from 0.231 flight at rest to 0.698 under full-field flicker, about three semitones and +31% level. The probe also told me which circuit it came from — `Mi1`, `T4`, and `T5` all read zero, so it doesn't arrive via motion detection. It's loom → `LC4` → giant fibre → jump muscle plus wing power: the escape-takeoff program, reached by the correct route.

## The split-brain rig

The fly needs the eGPU, the eGPU needs wall power, and the toy car can't carry wall power. So the two roles live on two commas over Wi-Fi.

**Diagram: Two commas, one lossy link**

The car node carries a comma 4 on the toy chassis with the front fisheye camera, an eye streamer, a sensor streamer, a Wi-Fi scanner, the car control server, the motor board, the wheels, and the speaker. A separate antenna node polls a directional bearing. Both reach the compute node through a lossy access point. The compute node is a second comma 4 with the chestnut eGPU and holds the fly simulation, the GPU itself, the event stream, the control bridge, and the voice conductor, with both stream readers reading over localhost. Drive frames and sound commands travel back across the same access point to the car's control server and sound player.

_Containment is the point: every pixel and every GPU kernel stays inside its own node, and only small one-way frames cross the radio._

```d2
direction: right

car: Car node, comma 4 on the toy chassis {
  camera: Front fisheye camera
  eyes: Eye streamer
  sensors: Sensor streamer
  scan: Wi-Fi scanner
  server: Car control server
  board: Motor board
  wheels: Wheels and steering
  speaker: Speaker
  voice: Wing sound player

  camera -> eyes: pool to luma grid
  server -> board: serial command
  board -> wheels: drive
  voice -> speaker: play
}

antenna: Antenna node {
  poller: Directional bearing poller
}

air: Access point {
  wifi: Lossy Wi-Fi
}

compute: Compute node, comma 4 with the chestnut eGPU {
  sim: Fly simulation
  gpu: chestnut AMD GPU
  events: Event stream
  bridge: Control bridge
  conductor: Voice conductor

  sim -> gpu: run the connectome
  sim -> events: publish state
  events -> bridge: read over localhost
  events -> conductor: read over localhost
}

car.eyes -> air.wifi: luma grid
car.sensors -> air.wifi: body state
car.scan -> air.wifi: network list
antenna.poller -> air.wifi: bearing

air.wifi -> compute.sim: sensory input
compute.bridge -> air.wifi: drive frame
compute.conductor -> air.wifi: sound command

air.wifi -> car.server: drive frame
air.wifi -> car.voice: sound command
```

Both event-stream readers live on the compute node rather than the car, because the AP loses enough packets that a long-lived car→compute read stalls while localhost never does. Every UDP path is latest-state only and every stream has a timeout, so a sense the fly has lost reads as **silence**, never as a frozen value.

## Pain, and why the fly has a save file

The bit I didn't plan and ended up liking most: the noxious channel has sensitization. Impact jolt, sustained wall-pushing strain, and board temperature past the TrpA1-equivalent threshold feed a phasic sensitivity that decays over 30 seconds, plus a **chronic** floor that ratchets up with injury history and never decays. A car that has been crashed repeatedly flinches harder and sooner than a fresh one, and because the sim divides pain by the neuropeptide-F tone, the same knock plays softer when the fly is doing well.

The chronic floor, the learned odor valences, and the biography — first boot, accumulated _running_ seconds, lifetime injury count — persist to disk, and redeploys don't reset them. There is deliberately no reset flag: killing this particular fly takes `ssh` and `rm`.

## What was the brain, and what was me

Here's the part a demo skips. Of 166,700 neurons, 11,753 receive hardware input or are read out to hardware — 7.05% of the brain.

The other 93% is untouched connectome doing whatever the wiring makes it do. That's the pitch, and I still like it. We inject at the sensory periphery and read at the motor periphery, and everything in between is the fly. It is also only true for throttle, reverse, and escape. Steering is where it falls down.

`DNa02` is the real steering descending neuron, and it's a _pair_: one cell per side. At the 0.2-second windows the control loop runs on, a "5 Hz" single-neuron readout is one spike, and a reproducible signed asymmetry per turn direction never materialized. Meanwhile `LC4`, the lateralized object detector that shows clean asymmetry under a synthetic bar, pegs at full deflection on a rich fisheye view of a real room and swamped everything else.

So the rig that drove around the office ran with visual fixation disabled and steered from a composite I wrote by hand: a two-sine wander term to explore, a graded turn-away-from-wall term gated on loom rate, and a direct RSSI-gradient bias. The descending neurons were live, published, and visible in the viewer — and not steering. That's the derp: a control-loop problem dressed as a neuroscience result.

Late on the second day I got a real version working. The Wi-Fi bearing injects into the `PFN` goal-vector neurons, the central complex does its thing — `EPG`, `PEN`, `hΔ`, `Δ7` — and steering comes out of `PFL3`, its actual steering output, with a slow baseline removed for the resting asymmetry. Signs were correct both ways: AP on the right gave +0.66, AP on the left −0.38. It shipped opt-in behind a flag — tested, but not the configuration that had been driving all afternoon.

Two more for the record. The reward system is **instrumented but mute**: the dopaminergic, octopaminergic, insulin, and neuropeptide-F populations collectively receive 212,836 synapses and have out-degree zero, because the pack only signs fast ionotropic transmitters. For a GPCR acting over hundreds of milliseconds that's arguably correct, but it means the pleasure and pain visible in the viewer are carried by scalar laws beside the connectome, not by it. And there's an open contradiction I never chased: in the last longer-window probe, loom measured 0.00 at _every_ station including the giant fibre, against an earlier run that found it at 15–40 Hz. Loom is the stimulus that should drive that pathway hardest.

## The one synapse a fly can change

There was a second, separate track, and it is the one place anything gets trained. The mushroom body is where a real fly's plasticity lives: 4,064 Kenyon cells fan odors out into a sparse random code, 97 MBONs read that code as approach-or-avoid, and dopamine from `PAM` and `PPL1` depresses exactly the KC→MBON synapses that were just active. So I trained gains on 1,797 of the 61,210 measured KC→MBON edges and nothing else — no edge added, no sign flipped, no zero made non-zero, nothing outside the mushroom body touched. On held-out arenas with a late, sparse radio signal, the starting policy managed 47 of 64 arrivals and the best trained run 62, 59 of them contact-free; against randomized radio lag it held 56 of 64, all contact-free. That policy remained in shadow mode on the live rig, with its motor output gated off. It is a population-rate surrogate rather than the spiking connectome, so these arena results are evidence about a separate model, not evidence that the connectome learned to drive.

## What two days bought

A real connectome, simulated whole and bit-exact, faster than real time, on an AMD GPU driven over USB from a dashcam, its sensory periphery wired to cameras and radios and its motor periphery wired to wheels and a speaker. It crawls, it flinches, it remembers which networks hurt it, and it does not shut up.

And on the connectome rig, the wheel that turns toward the Wi-Fi is turned by four lines of arithmetic I wrote, with 155,000 neurons of _Drosophila melanogaster_ buzzing along beside them, having no say in the matter. Which is, if nothing else, an accurate model of a fly.
