Smart Device + Home Server Architecture

AI Wearable with On-Device Intelligence

Not just another "$600 microphone." This device has actual brains - Syntiant NDP120 handles wake detection and speaker ID at 150 microwatts. Your home server does the rest. No cloud. No subscription. Your data never leaves your network.

~$115
Total Device Cost
150uW
Always-On Power
$0/mo
Subscription Cost
100%
Local Processing
The Vision

What Makes BrainGoblin Different?

Commercial devices are fancy microphones. This is a smart AI device.

⚡

On-Device Wake

Syntiant NDP120 neural processor runs at 150 microwatts - always listening for human speech, not music, not TV, not silence. Wakes the main MCU only when needed.

👥

On-Device Speaker ID

The NDP120 knows WHO is talking - not just that someone is talking. Train it on your voice and your family. All on the device itself. No cloud processing needed.

🏠

Home Server Processing

Whisper transcription and LLM extraction run on YOUR hardware. Raspberry Pi 5 or a mini PC. Your data never touches the internet. Ever.

🔒

100% Private

No cloud accounts. No telemetry. No data harvesting. Everything stays on your local network. Works even without internet.

💳

No Subscription

One-time hardware cost. No monthly fees. No "$24/month forever" like commercial alternatives. Own your device, own your data.

🧠

Built for ADHD

Commitment tracking that knows the difference between what someone TOLD you to do vs what YOU said you'd do. Life logging for jokes, ideas, memories.

💡 The Key Insight

Commercial devices like Limitless ($300) and Humane AI Pin ($699) are literally just microphones. All the AI runs in their cloud. You pay for the device, then pay $24/month forever to use it.

BrainGoblin puts real AI on the device (wake + speaker ID) and runs the heavy processing on hardware you control. $115 total, $0/month ongoing.

System Design

Complete System Architecture

Three components working together - device, server, and viewer

+-----------------------------------------------------------------------------+
|                         BRAINGOBLIN DEVICE                                  |
|                                                                             |
|   +-------------------+    +-------------------+    +------------------+   |
|   | SYNTIANT NDP120   |    | ESP32-S3          |    | Storage          |   |
|   | (Always-on AI)    |    | (Main MCU)        |    |                  |   |
|   |                   |    |                   |    | SD Card 32GB     |   |
|   | * Voice wake      |--->| * Record audio    |--->| Weeks of         |   |
|   | * Speaker ID      |    | * Compress        |    | recordings       |   |
|   | * 150 microwatt   |    | * WiFi/BLE        |    | Offline queue    |   |
|   +-------------------+    +-------------------+    +------------------+   |
|                                                                             |
|   ON-DEVICE: Wake detection, Speaker ID, Recording, Storage                |
+-----------------------------------------------------------------------------+
                                    |
                                    | WiFi (home network)
                                    v
+-----------------------------------------------------------------------------+
|                           HOME SERVER                                       |
|                    (Raspberry Pi 5 / Mini PC)                               |
|                                                                             |
|   +-------------------+    +-------------------+    +------------------+   |
|   | Whisper.cpp       |    | Local LLM         |    | Database         |   |
|   |                   |    | (Llama/Mistral)   |    |                  |   |
|   | * Transcription   |--->| * Commitments     |--->| * All your data  |   |
|   | * 100% local      |    | * Life logging    |    | * Searchable     |   |
|   | * No cloud        |    | * People          |    | * Encrypted      |   |
|   +-------------------+    +-------------------+    +------------------+   |
|                                                                             |
|   YOUR DATA NEVER LEAVES YOUR NETWORK                                      |
+-----------------------------------------------------------------------------+
                                    |
                                    v
+-----------------------------------------------------------------------------+
|                           PHONE APP                                         |
|                                                                             |
|   * Just a VIEWER - no heavy processing                                    |
|   * Shows transcripts, commitments, reminders                              |
|   * Configure device via BLE                                               |
|   * Minimal battery usage                                                  |
|                                                                             |
+-----------------------------------------------------------------------------+
                

📡 Why This Architecture?

  • Phone battery unaffected - Phone is just a viewer, not doing AI processing
  • Works without phone - Device stores locally, syncs when connected
  • Privacy by design - Data stays on your network
  • No subscription lock-in - You own the hardware, you run the software
  • Upgradeable - Want a better LLM? Just update your server

🎯 What Runs Where

Task Location Power
Voice wake detection NDP120 on device 150uW
Speaker identification NDP120 on device 150uW
Audio recording ESP32-S3 on device 50mA
Local storage SD card on device -
Transcription Home server -
AI extraction Home server -
Viewing results Phone app Minimal
Hardware Design

BrainGoblin Device

Dual-board design with on-device AI and weeks of local storage

Syntiant NDP120 - The Secret Weapon

This is what makes BrainGoblin different from "dumb microphones." The NDP120 is a neural decision processor that runs tiny ML models at ultra-low power. It's the same technology Google uses for "Hey Google" always-on listening.

⚡ 150 Microwatts

That's 0.00015 watts. A standard LED draws 20mW - 130x more power. This chip can listen 24/7 for MONTHS on a coin cell battery.

What It Does On-Device:

  • Voice Activity Detection - Wakes ESP32 only when humans speak
  • Speaker Identification - Knows WHO is talking (5-10 enrolled voices)
  • Wake Word Detection - Optional "Hey BrainGoblin" style wake
  • Audio Classification - Distinguishes speech from TV/music

Development Board: Arduino Nicla Voice

  • NDP120 built-in with on-board microphone
  • Edge Impulse compatible for model training
  • 22.86mm x 22.86mm form factor
  • GPIO pins for ESP32 communication

NDP120 Specifications

Power Consumption ~150 uW
Neural Network 1M parameters (FW) or 4M (HW)
Audio Inputs PDM, I2S
Sample Rates 8kHz - 48kHz
Host Interface SPI, I2C, GPIO
Training Platform Edge Impulse
Package 3.5mm x 3.5mm BGA

Speaker Enrollment Workflow

# Train via Edge Impulse
1. Record 3-10 voice samples per person
2. Upload to Edge Impulse project
3. Train speaker classification model
4. Deploy to NDP120

# On-device identification
NDP120 outputs speaker ID with each
audio segment (0-N speaker index
or "unknown")

XIAO ESP32-S3 Sense - Main MCU

The ESP32-S3 handles everything the NDP120 doesn't - recording audio, compressing it, storing on SD card, and uploading to your home server via WiFi. The XIAO Sense version includes an SD card slot built in.

Key Features:

  • WiFi + BLE - Connect to home server, configure via phone
  • SD Card Slot - Built into XIAO Sense board
  • USB-C - Programming and charging
  • Deep Sleep - Wake on NDP120 interrupt
  • Plenty of GPIO - I2S mic, SPI to NDP120, etc.

Why ESP32-S3 (not nRF52840)?

The nRF52840 is great for BLE-only wearables, but we need WiFi to upload to the home server. The NDP120 handles the ultra-low-power always-on listening, so the ESP32-S3's higher power consumption only matters when actively recording/uploading.

💡 Sleep Mode Operation

ESP32-S3 stays in deep sleep (~14uA) until NDP120 sends a wake interrupt. Then it records, compresses, stores, and goes back to sleep. WiFi only activates for batch uploads to save power.

XIAO ESP32-S3 Sense Specs

Processor ESP32-S3 Dual-core 240MHz
RAM 8MB PSRAM
Flash 8MB
WiFi 802.11 b/g/n
Bluetooth BLE 5.0
SD Card MicroSD slot (built-in)
Battery LiPo charging circuit
Dimensions 21mm x 17.5mm
Deep Sleep ~14uA

Wiring to NDP120

# ESP32-S3 to Nicla Voice
ESP32 GPIO4  -> Nicla WAKE_INT (interrupt)
ESP32 GPIO5  -> Nicla SPI_CS
ESP32 GPIO6  -> Nicla SPI_CLK
ESP32 GPIO7  -> Nicla SPI_MOSI
ESP32 GPIO8  -> Nicla SPI_MISO
3.3V         -> Nicla 3.3V
GND          -> Nicla GND

# I2S Microphones
ESP32 GPIO1  -> I2S BCLK
ESP32 GPIO2  -> I2S LRCK
ESP32 GPIO3  -> I2S DOUT

Dual ICS-43434 MEMS Microphones

The latest generation I2S MEMS microphone from TDK InvenSense. Unlike the older (now discontinued) INMP441, the ICS-43434 has native I2S output with standard timing - no hacks needed.

Why Dual Mics?

  • Beamforming - Focus on voice in front, reject background noise
  • 6-10dB SNR improvement - Voice stands out even in noisy environments
  • Redundancy - If one mic fails, you still have audio
  • Stereo for NDP120 - Better speaker ID accuracy

Key Specifications

  • Signal-to-Noise Ratio: 65 dB(A)
  • Acoustic Overload Point: 120 dB SPL
  • Frequency Response: 60Hz - 20kHz
  • Current Consumption: 490uA typical
  • Interface: Native I2S (24-bit, up to 51.2kHz)
  • Package: 3.5mm x 2.65mm x 0.98mm
🔔 Bottom Port Design

The ICS-43434 is a bottom-port microphone. The sound hole faces DOWN toward the PCB. Your enclosure needs an acoustic port aligned with this hole!

Microphone Comparison

Mic Interface SNR Status
ICS-43434 Native I2S 65dB Chosen
INMP441 PDM->I2S 61dB EOL
SPH0645 Weird I2S 65dB Problematic
Nicla PDM PDM ~60dB Backup

Stereo Configuration

# Both mics share same I2S bus
# L/R pin determines which clock edge

Mic 1 (Left):   L/R -> GND
Mic 2 (Right):  L/R -> 3.3V

# Same BCLK, LRCK, DOUT lines
# MCU receives stereo interleaved data

Where to Buy

Power System

The magic of BrainGoblin is that the NDP120 does the always-on listening at microwatts, while the ESP32-S3 only wakes when there's actual speech to record.

Recommended Battery: 1000-1500mAh LiPo

  • 3.7V nominal, 4.2V max
  • USB-C charging via XIAO built-in circuit
  • JST connector compatible
  • Fits in ~50mm puck enclosure

Power States

State Current Battery Life (1000mAh)
Idle (NDP120 listening) ~150uA Months
Active recording ~50mA ~20 hours
WiFi sync ~100mA ~10 hours
🔋 Real-World Estimate

Assuming 4-6 hours of actual speech per day (realistic for most people), a 1000mAh battery should last 2-3 days between charges. With WiFi duty cycling (sync every 15-30 minutes), even longer.

Power Budget Breakdown

ALWAYS ON (NDP120 listening):
- NDP120:     ~150 uA
- Total:      ~150 uA

RECORDING (ESP32 active):
- NDP120:     ~150 uA
- ESP32-S3:   ~40 mA (with I2S)
- ICS-43434:  ~1 mA (2 mics)
- SD Card:    ~5 mA (write)
- Total:      ~46 mA

WIFI UPLOAD:
- NDP120:     ~150 uA
- ESP32-S3:   ~100 mA (WiFi TX)
- Total:      ~100 mA

DEEP SLEEP:
- NDP120:     ~150 uA
- ESP32-S3:   ~14 uA
- Total:      ~164 uA

Charging

The XIAO ESP32-S3 Sense has a built-in LiPo charging circuit. Just plug in USB-C and it charges the battery automatically. No extra components needed.

  • USB-C input (5V)
  • Built-in charge controller
  • Battery protection circuit
  • Can run while charging
Local Processing

Home Server Setup

Your data never leaves your network - run everything locally

💻 Hardware Options

Any computer that can run Docker. Two main options:

Option 1: Raspberry Pi 5 (~$80)

  • 8GB RAM model recommended
  • Whisper.cpp runs at ~2x real-time
  • Can run smaller LLMs (Phi-3, Gemma 2B)
  • Low power consumption (~5W)
  • Fits in a shoebox, runs 24/7

Option 2: Mini PC with N100 (~$150)

  • Intel N100 processor (much faster than Pi)
  • 16GB RAM typical
  • Whisper.cpp runs at ~5x real-time
  • Can run larger LLMs (Llama 3.2 8B, Mistral 7B)
  • Still low power (~15W)

📦 Software Stack (Docker)

Everything runs in Docker containers for easy setup and isolation.

Services:

  • whisper.cpp - Local transcription (no cloud)
  • Ollama - Local LLM runner
  • PostgreSQL/SQLite - Database
  • BrainGoblin API - REST API for devices/apps
# docker-compose.yml (simplified)
services:
  whisper:
    image: ghcr.io/ggerganov/whisper.cpp
    volumes:
      - ./audio:/data

  ollama:
    image: ollama/ollama
    volumes:
      - ./models:/models

  api:
    image: braingoblin/server
    ports:
      - "8080:8080"
    depends_on:
      - whisper
      - ollama

🔐 LLM Model Recommendations

For Raspberry Pi 5:

  • Phi-3 Mini (3.8B) - Microsoft's efficient model
  • Gemma 2B - Google's small model
  • TinyLlama (1.1B) - Very fast, okay quality

For N100 Mini PC:

  • Llama 3.2 8B - Best overall quality
  • Mistral 7B - Great for extraction tasks
  • Gemma 7B - Good balance
Bill of Materials

Complete Parts List

Everything you need to build a BrainGoblin device (~$115 total)

Core Components

Arduino Nicla Voice
Syntiant NDP120 development board with on-board mic and Edge Impulse support
x1
~$65
XIAO ESP32-S3 Sense
Main MCU with WiFi, BLE, SD card slot, and battery charging
x1
~$14
ICS-43434 Breakout
I2S MEMS microphone breakouts - need 2 for stereo beamforming
x2
~$12

Power & Storage

1000mAh LiPo Battery
3.7V lithium polymer with JST connector
x1
~$8
MicroSD Card 32GB
For local audio storage - weeks of recordings
x1
~$8

Enclosure & Misc

3D Printed Case
~50mm puck-style enclosure (STL files provided)
x1
~$5
Wires, Switch, Misc
Hookup wire, power switch, JST connectors
-
~$5

💰 Total Device Cost: ~$115

Compare to Limitless ($300 + $24/mo) or Humane AI Pin ($699 + $24/mo). And this has ACTUAL on-device AI, not just a microphone streaming to the cloud!

Optional: Home server adds ~$80-150 depending on whether you use a Raspberry Pi 5 or a Mini PC. But you may already have suitable hardware lying around.

Why BrainGoblin?

Comparison

How BrainGoblin stacks up against commercial alternatives

Limitless Pendant
$300 + $24/mo
  • ✘ Wake detection in cloud
  • ✘ Speaker ID in cloud
  • ✘ Transcription in cloud
  • ✘ All data on their servers
  • ✘ Requires subscription
  • ✘ Closed source
  • ✔ Polished product
Humane AI Pin
$699 + $24/mo
  • ✘ All processing in cloud
  • ✘ Requires cellular data
  • ✘ All data on their servers
  • ✘ Requires subscription
  • ✘ Closed source
  • ✘ Poor battery life
  • ✔ Has a projector?
Feature Limitless Humane AI Pin BrainGoblin
Wake Detection Cloud Cloud On-Device (NDP120)
Speaker ID Cloud Cloud On-Device (NDP120)
Transcription Their cloud Their cloud Your server
AI Processing Their cloud Their cloud Your server
Monthly Cost $24/mo $24/mo $0
Data Location Their servers Their servers Your network
Works Offline No No Yes (stores locally)
Open Source No No Yes
ADHD-Focused No No Yes