Quickstart

Get DOOM Neuron training running in minutes with real biological neurons

Quickstart Guide

Get a DOOM Neuron training session running in under 10 minutes.

This guide assumes you have CL1 hardware available. For SDK-based testing, use the convenience scripts in scripts/run_sdk_*.sh.

Prerequisites

• Python 3.8+
• CUDA-capable GPU (recommended) or CPU
• CL1 hardware or CL SDK for testing
• Network access between training server and CL1 device

Installation

1. Clone and Setup Environment

   python3 -m venv .venv
   source .venv/bin/activate
   pip install -r requirements.txt
   

   Torch 2.10 with CUDA 13.0 was used in testing, but any version compatible with your hardware should work.

2. Verify Installation

   Check that VizDoom and the CL SDK are properly installed:

   python3 -c "import vizdoom; import cl; print('All dependencies installed!')"
   

Training Architecture

DOOM Neuron uses a distributed architecture with two components:

1. CL1 Neural Interface - Runs on the CL1 device, handles stimulation and spike recording
2. Training Server - Runs the PPO training loop, manages the game environment

Quick Start: Local Training

For local development on the same machine:

CL1 Interface (Terminal 1)

python cl1_neural_interface.py --training-host localhost

Start the CL1 interface first, then the training server. Both should launch around the same time.

Quick Start: Remote Training

For training with CL1 on a separate device:

CL1 Interface (on CL1 device at 192.168.1.50)

python cl1_neural_interface.py --training-host 192.168.1.100

Network Ports

Ensure these UDP ports are open:

• 12345 - Stimulation commands (training → CL1)
• 12346 - Spike data (CL1 → training)
• 12347 - Event metadata (training → CL1)
• 12348 - Feedback commands (training → CL1)

Convenience Scripts

The project includes convenience scripts for common workflows:

Hardware CL1

# Run this on CL1 device
./scripts/run_cl1.sh

Check the IP addresses and tick frequency in the scripts before running!

Monitoring Training

1. Watch TensorBoard

   Training metrics are logged to TensorBoard:

   tensorboard --logdir checkpoints/l5_2048_rand/logs --port 6006
   

   Open http://localhost:6006 in your browser to see:

   • Episode rewards
   • Loss curves (policy, value, encoder entropy)
   • Action distributions
   • Decoder weight/bias ratios
2. Visual Feedback (Optional)

   For real-time game visualization, open visualisation.html in a browser and update the image source:

   <img id="img" width="640" src="http://127.0.0.1:12349/doom.mjpeg">
   

Understanding the Output

Training Server Output

Episode 1 | Reward: 150.0 | Length: 245 | Kills: 2
Episode 2 | Reward: -50.0 | Length: 89 | Kills: 0
Episode 3 | Reward: 320.5 | Length: 412 | Kills: 3

• Reward: Total episode reward (game rewards + custom shaping)
• Length: Number of timesteps before episode ended
• Kills: Enemy kills in the episode

Checkpoints

Model checkpoints are saved every 100 episodes to:

checkpoints/l5_2048_rand/episode_*.pt

First Training Session

Let’s run a simple training session:

1. Start CL1 Interface

   python cl1_neural_interface.py \
       --training-host localhost \
       --tick-frequency 10
   

   You should see:

   [CL1] Listening for stimulation on port 12345
   [CL1] Sending spikes to port 12346
   [CL1] Neural loop running at 10 Hz
   
2. Start Training Server

   python training_server.py \
       --mode train \
       --device cuda \
       --cl1-host localhost \
       --max-episodes 1000
   

   Training will begin immediately:

   [Training] Connected to CL1 at localhost
   [Training] Scenario: progressive_deathmatch.cfg
   [Training] Episode 1 starting...
   
3. Monitor Progress

   In another terminal, launch TensorBoard:

   tensorboard --logdir checkpoints/l5_2048_rand/logs
   

   Watch the reward curve increase as the biological neurons learn!

Common Training Patterns

Resume from Checkpoint

python training_server.py \
    --mode train \
    --device cuda \
    --cl1-host 192.168.1.50 \
    --checkpoint checkpoints/episode_500.pt

Watch Trained Policy

python training_server.py \
    --mode watch \
    --device cuda \
    --checkpoint checkpoints/final_model.pt

Watch mode uses direct hardware access and has not been ported to UDP yet.

Show Game Window (Debug)

python training_server.py \
    --mode train \
    --device cuda \
    --cl1-host localhost \
    --show_window

Custom Recording Path

python training_server.py \
    --mode train \
    --device cuda \
    --cl1-host 192.168.1.50 \
    --recording_path /data/doom_recordings

Stopping Training

Press Ctrl+C in either terminal to gracefully shutdown:

• Training server sends completion signal
• CL1 interface saves recording and exits
• Both processes cleanup sockets

Output Files

Training Server

• checkpoints/episode_*.pt - Model checkpoints
• checkpoints/l5_2048_rand/logs/ - TensorBoard logs
• training_log.jsonl - Episode statistics

CL1 Interface

• <recording_path>/*.cl1 - Neural recordings with metadata

Next Steps

Configuration

Tune PPO hyperparameters and feedback settings

Scenarios

Explore different DOOM scenarios and curriculum learning

Architecture

Deep dive into the encoder-decoder pipeline

Troubleshooting

Common issues and solutions