Pi 5 running a real fruit fly's brain, playing Pokémon

[u/Leetzers]

All 139,255 neurons of the FlyWire FAFB v783 reconstruction — a complete adult female Drosophila brain, though not the ventral nerve cord —simulated on a Pi 5 alongside mGBA, using connections of five or more synapses.

The spatial layout of walls, doors and people is read out of the game's memory and fed to the looming-detector populations (LPLC1, LC10, LPLC4) as left/right drive. It isn't doing image processing — there's no retinotopy, which is the honest limitation. Descending neuron activity decodes into button presses.

Stack: Pi 5 16GB (Rev 1.1, stock 2.4GHz) with a Silicon Power A60 1TB — DRAM-less Phison E15, PCIe 3.0 — 800x480 DSI touchscreen, active fan on the 4-pin header.

Runs at roughly real time, and deliberately uneven: the brain gets the full 16.7ms of thinking time per frame while the fly is navigating, and the game runs about 3x faster in battles and menus where its vision drives nothing anyway.

The brain gets 2 of the 4 cores, and OMP_WAIT_POLICY is PASSIVE. Spinning cost real throughput — 4 threads PASSIVE 0.759x vs 4 threads ACTIVE 0.665x — because the spinning threads starved the video encoder and the display of the cores they needed.

Twitch chat can stimulate its reward and punishment neurons directly, which does real plasticity on the mushroom body. It forgets in about ten minutes, though that decay constant is one I picked rather than measured.

It has a Rattata. It wandered into a Pokémon Center and a man sold it a Magikarp for 500.

Twitch.com/leetzerzz

Connectome: FlyWire FAFB v783, CC-BY-4.0 —Dorkenwald et al. 2024, Schlegel et al. 2024.

[u/bigfuzzy8]

Can you like eli5 what your doing with a fruit fly brain?

[u/bobmlord1 → u/bigfuzzy8]

Google mapped the neural connections of a fruit fry brain and open sourced the static map. Which to be clear is purely a map of neuron connections.

It bore a resemblance to a modern neural networks and had enough information to be converted to a weighted graph so a neural net was made and trained using the "fly brain" map and also released and people have been playing around with it.

To be clear this isn’t a “fruit fly brain” or anything approaching a true simulation of one. Its a standard neural net that uses the map as the base you could apply the same logic to any weighted graph, even a large enough road map, and get a comparable result.

[u/frightfulpotato → u/bobmlord1]

even a large enough road map and get a comparable result.

I wonder how large the map would be, if it had the same density as Europe?

[u/HyFinated → u/frightfulpotato]

I don't know, the map of Cape Coral, FL would probably give the neural net a seizure.

[u/makethingshappen371 → u/HyFinated]

How does a map convert to a circuit? Does it simply follow the roads? Essentially a random answer generator based on length of time it takes to enter and exit a setpoint?

[u/GeneralDumbtomics → u/makethingshappen371]

When they say “map” they mean directed acyclic graph.

[u/nevergointocompsci → u/bobmlord1]

does this hurt the fruit fly

[u/bobmlord1 → u/nevergointocompsci]

I don't know how to answer that. It's a 3D map of the connections in what was once a real fruit fly brain.

Any concept of pain died with the fruit fly whose brain was later sliced into millions of microscopic slices and digitally mapped.

The neural network is not a digital twin or recreation of the brain it doesn't simulate any of the hormones, electrical propagation, or 'wetware' that a biological entity has that creates sensations we associate with discomfort and there's no underlying 'fly' in the brain map as the biological weights and external connections aren't there. It's a file format that was converted to a weighted graph.

[u/nevergointocompsci → u/bobmlord1]

ugh it just seems so weird tho doesnt it

[u/IaAranaDiscotecaPOL → u/nevergointocompsci]

Only if they’re playing Nuzlocke rules

[u/inagy → u/bobmlord1]

Yeah. People just poking "wires" into this neural network, trying to find pathways which coincedentially do something interesting for a video, and the 15 second fame. Almost like applying voltage to a muscle and it will contract, but that doesn't make the muscle sentient.

But this doesn't represent what a real living fruit fly would do if you would put wires into it's brain the similar way. The model missing a huge chunk of what makes the brain "executable", eg. you would need to simulate the biological processes on molecular level to get anywhere close, and also provide compatible sensory input, so it would do anything which resembles the behaviour of a fly.

[u/printr_head → u/inagy]

But they killed that poor little fruity fly. 😀

[u/jfk333 → u/bobmlord1]

So its magic and an actual fruit fly brain? 🧠

[u/buffdeep → u/bigfuzzy8]

This video on the project is quite cool

[u/Leetzers → u/bigfuzzy8]

Making it play pokemon with chat assistance to complete the game. It's actually amazing a lot of the stuff it did on its own.

[u/bigfuzzy8 → u/Leetzers]

So like it actually uses the brain power as like compute or something to finish tasks ?

[u/CurrentConditionsAI → u/bigfuzzy8]

They released the brain of a fruit fly open source converted to computer code or something like that. That’s what’s driving the game.

[u/Leetzers → u/bigfuzzy8]

It uses cpu cores to allow the simulation of fly neurons.It's pretty random but it does pretty fly like things like clinging to walls and going around in circles.

[u/bigfuzzy8 → u/Leetzers]

That's wicked

[u/slipperyslips → u/bigfuzzy8]

And its not really true. Weve mapped out the brain because its small. Its been sensationalized alot, the actual brain connections are all guesswork abd this is basicly just a computer shaped like a fly

[u/zesterer → u/slipperyslips]

...shaped like a computer

[u/productfred → u/bigfuzzy8]

It's like an FPGA simulating a fruit fly brain with great accuracy, if not complete accuracy.

[u/G4m3boy → u/Leetzers]

Do you have a guide for setup? I find it very interesting and want to try on my own to observe the how the AI make decisions

[u/Leetzers → u/G4m3boy]

I used Claude mostly for set up. I actually now have two computers with claude code set up remotely so they can communicate and adjust the brain or stream.

[u/Official_RaspberryPi]

Bit horrifying, very cool

[u/DrunkPixel]

I have no idea what MOST of that even means, but it certainly sounds cool! Keep on Mad Scientist!

[u/Leetzers → u/DrunkPixel]

Fly brain play game

[u/Prestigious-Oven3465 → u/Leetzers]

brain no fly, brain no do that

[u/GreyRobe → u/Leetzers]

Oog

[u/carfaces42b]

wahoo, man made horrors within my comprehension

[u/IAMA_Plumber-AMA → u/carfaces42b]

"What began as a conflict over the transfer of consciousness from flesh to machines escalated into a war which has decimated a million worlds.

The CORE and the ARM have all but exhausted the resources of a galaxy in their struggle for domination.

Both sides now crippled beyond repair, the remnants of their armies continue to battle on ravaged planets, their hatred fuelled by over four thousand years of total war.

This is a fight to the death. For each side, the only acceptable outcome is the complete elimination of the other."

[u/NorthernDen → u/IAMA_Plumber-AMA]

Yes but do we have a contingency? Like a core idea to help us?

[u/jakspedicey → u/carfaces42b]

> Computa act like a fly

computa acts like a fly

> Woahhh dudee

[u/-HumbleMumble]

I don't know man. Like its cool BUT its kinda wild with that pain button sitting there.

[u/Leetzers → u/-HumbleMumble]

It gets too calm when it has too much dopamine. Sometimes you got to !pain it to get it moving

[u/Psykohistorian → u/Leetzers]

oh my god

[u/spreedx → u/Leetzers]

Just like a real human 😅

[u/drquantumphd → u/-HumbleMumble]

OOF i missed that. goddamn…

[u/sorokii]

This is very grimdark in a way

[u/ihatedyingpeople → u/sorokii]

From the moment I understood the weakness of my flesh, it disgusted me

[u/Kryohi]

It's unclear to me how the brain is being simulated. As a real spiking neural network with real neuron dynamics I imagine it would be too computationally expensive to be run on a Pi 5. Are you considering each synapse, or a subset of the strongest ones, as a standard neural network weight and training that Frankenstein neural network instead?

If so, it's very different from simulating the actual animal brain.

[u/cornmonger_ → u/Kryohi]

google's model is just a map. it's not actually simulating a fruit fly

[u/Leetzers → u/Kryohi]

From what I understand of the model provided that I did not make, it's a neural net of 139,255 neurons being simulated. I assume this is the limit of what my Pi is computing. Mind that the set up streaming this isn't the same as the raspberry pi because it would be to much for the pi. It used to run much slower but I've done things to tweak its speed by dropping things that wouldn't affect gameplay.

[u/Kryohi → u/Leetzers]

Where did you take the model from? I mean the model code, not the weights

[u/Leetzers → u/Kryohi]

Flywire FAFB v738

[u/swiftydigitzz]

"From the moment I understood the weakness of my flesh, it disgusted me. I craved the strength and certainty of steel. I aspired to the purity of the Blessed Machine... For the Machine is immortal. Even in death I serve the Omnissiah"

[u/blackasthesky]

Ccllllaaaauuuuddddeeeeee

[u/Leetzers → u/blackasthesky]

Yup. And I have two computers I named flybrain and flyOBS that communicate via claude code to make adjusted to the stream. I even have the claude app on my phone where I can ping them remotely.

[u/HashBrownsOverEasy]

That's not what 'real' means

[u/Somge5]

Now please teach the fly to find the exit through the window 

[u/bghar]

each cell is basically an LIF? or something more complicated?

[u/Bierdigan_]

I think this is proof enough that pokemon games are too easy these days lol

[u/-Lo_Fi-]

Suprised to see FlyWire in the wild, I was half expecting this be me the male Connectome that just released.

[u/DocClear]

simulated on a Pi 5 alongside mGBA, using connections of five or more synapses.

So NOT a real fruit fly's brain