From 97da7f2d0f4aeafbb5460f68402c4095cc92acdb Mon Sep 17 00:00:00 2001 From: imonlinux <39863321+imonlinux@users.noreply.github.com> Date: Fri, 5 Dec 2025 19:50:01 -0800 Subject: [PATCH] Initial draft xvf3800 tutorial --- docs/linux-voice-assistant-xvf3800.md | 403 ++++++++++++++++++++++++++ 1 file changed, 403 insertions(+) create mode 100644 docs/linux-voice-assistant-xvf3800.md diff --git a/docs/linux-voice-assistant-xvf3800.md b/docs/linux-voice-assistant-xvf3800.md new file mode 100644 index 0000000..38d4154 --- /dev/null +++ b/docs/linux-voice-assistant-xvf3800.md @@ -0,0 +1,403 @@ +# Linux Voice Assistant with ReSpeaker XVF3800 4‑Mic Array — Installation & Configuration Guide + +> Drafted with ChatGPT based on existing `linux-voice-assistant-2mic-install.md` and +> community test results from the XVF3800 integration Discussion. + +This guide shows how to use the **Seeed ReSpeaker XVF3800 USB 4‑Mic Array** with the +**linux-voice-assistant** (LVA) project on Raspberry Pi or other Linux hosts. + +Unlike the ReSpeaker 2‑Mic HAT, the XVF3800 is a **USB Audio Class 2.0** device: +it acts as a “smart microphone + speaker” front end with built‑in **AEC, beamforming, +noise suppression, AGC, and echo‑canceling speakerphone** processing. + +LVA treats the XVF3800 as a **standard USB mic/speaker**: +- Audio in: 16 kHz, 16‑bit, 2‑channel PCM (downmixed to mono by the audio stack) +- Audio out: 16 kHz, 16‑bit, 2‑channel PCM via the XVF3800 headphone jack / speaker + +No custom kernel drivers are required. Plug it in, select it as your audio device in +LVA’s `config.json`, and it will “just work” as a high‑quality mic for wake word and +ESPHome streaming. + +--- + +## Prerequisites + +- A Linux host (e.g., Raspberry Pi 4/5) running one of: + - Raspberry Pi OS (Bookworm or Trixie) + - Debian 13 (trixie) or similar + - Other modern Linux with PipeWire or PulseAudio +- Python 3.11+ recommended +- **ReSpeaker XVF3800 4‑Mic Array**, connected via its **XMOS USB‑C port** +- Network access to your Home Assistant instance +- The `linux-voice-assistant` repository checked out (optionally on the `XVF3800` branch) + +--- + +## 1. Get the code (XVF3800 branch) + +If you don’t already have the project cloned, you can pull the dedicated XVF3800 branch: + +```bash +cd ~ +git clone --branch XVF3800 --single-branch https://github.com/imonlinux/linux-voice-assistant.git +cd linux-voice-assistant +``` + +If you already have the repository, you can switch branches: + +```bash +cd ~/linux-voice-assistant +git fetch origin +git checkout XVF3800 +``` + +Set up the project (creates a virtualenv, installs dependencies, etc.): + +```bash +cd ~/linux-voice-assistant +script/setup +``` + +> Tip: If you use a different username than `pi`, adjust paths accordingly. + +--- + +## 2. Connect and verify the XVF3800 + +### 2.1 Plug in the board + +1. Connect the XVF3800 to your Pi/host using the **XMOS USB‑C port** (the one next to + the 3.5 mm audio jack). +2. If you want local audio output, connect the XVF3800’s headphone jack to an external + powered speaker. + +### 2.2 Confirm ALSA sees the device + +```bash +arecord -l +aplay -l +``` + +You should see entries similar to: + +```text +**** List of CAPTURE Hardware Devices **** +card 3: Array [reSpeaker XVF3800 4-Mic Array], device 0: USB Audio [USB Audio] + Subdevices: 0/1 + Subdevice #0: subdevice #0 + +**** List of PLAYBACK Hardware Devices **** +card 3: Array [reSpeaker XVF3800 4-Mic Array], device 0: USB Audio [USB Audio] + Subdevices: 1/1 + Subdevice #0: subdevice #0 +``` + +The exact card index (`3` above) may differ on your system. + +### 2.3 Quick `arecord` test (optional) + +Replace `3` with your card number: + +```bash +arecord -D plughw:3,0 -c 2 -r 16000 -f S16_LE -d 5 -vv /tmp/xvf3800-test.wav +aplay /tmp/xvf3800-test.wav +``` + +You should hear your voice through your default playback device, confirming that: +- The XVF3800 is streaming at **16 kHz**, **16‑bit**, **stereo**, and +- ALSA can capture from it reliably. + +--- + +## 3. PipeWire / PulseAudio device mapping + +On a PipeWire or PulseAudio system, check how the XVF3800 shows up at the graph level: + +```bash +pactl list short sources +pactl list short sinks +``` + +Typical output: + +```text +# Sources +309 alsa_input.usb-Seeed_Studio_reSpeaker_XVF3800_4-Mic_Array_...-00.analog-stereo PipeWire s16le 2ch 16000Hz RUNNING + +# Sinks +308 alsa_output.usb-Seeed_Studio_reSpeaker_XVF3800_4-Mic_Array_...-00.analog-stereo PipeWire s16le 2ch 16000Hz SUSPENDED +``` + +Key points: + +- The XVF3800 appears as an **analog stereo** source/sink (`2ch 16000 Hz`). +- LVA will use these devices through the `soundcard` library. + +--- + +## 4. LVA’s view of the XVF3800 (input/output devices) + +From the project root, use LVA’s helper flags to inspect devices: + +```bash +cd ~/linux-voice-assistant +script/run -- --list-input-devices +script/run -- --list-output-devices +``` + +Example output: + +```text +Input devices + +[0] reSpeaker XVF3800 4-Mic Array Analog Stereo +``` + +```text +Output devices + +auto: Autoselect device +pipewire: Default (pipewire) +pipewire/alsa_output.usb-Seeed_Studio_reSpeaker_XVF3800_4-Mic_Array_...analog-stereo: reSpeaker XVF3800 4-Mic Array Analog Stereo +... +``` + +Take note of: + +- The **input device name** for the XVF3800 (e.g. `reSpeaker XVF3800 4-Mic Array Analog Stereo`) +- The output device you want to use (often `pipewire` or the specific XVF “analog‑stereo” sink) + +--- + +## 5. Configure LVA audio for XVF3800 + +Edit the main config: + +```bash +nano ~/linux-voice-assistant/linux_voice_assistant/config.json +``` + +Add or update the `audio` section: + +```json +"audio": { + "input_device": "reSpeaker XVF3800 4-Mic Array Analog Stereo", + "output_device": "pipewire", + "input_block_size": 1024 +} +``` + +Examples: + +- If you want both input and output on the XVF3800: + + ```json + "audio": { + "input_device": "reSpeaker XVF3800 4-Mic Array Analog Stereo", + "output_device": "pipewire/alsa_output.usb-Seeed_Studio_reSpeaker_XVF3800_4-Mic_Array_101991441253700076-00.analog-stereo", + "input_block_size": 1024 + } + ``` + +- If you only want to use the XVF3800 as a **mic** (and another device for playback), set + `output_device` to your preferred sink from `--list-output-devices`. + +### 5.1 How channels are handled (important detail) + +The XVF3800 exposes itself as a **2‑channel** source (stereo, 16 kHz). Internally, the +LVA `AudioEngine` uses the `soundcard` library and **always requests 1 channel** from the +selected input device: + +```python +with self.mic.recorder( + samplerate=16000, + channels=1, + blocksize=self.block_size, +) as mic_in: + audio_chunk_array = mic_in.record(self.block_size).reshape(-1) + ... +``` + +This has two important implications: + +1. The audio stack (PipeWire/PulseAudio/ALSA) takes care of converting from the XVF3800’s + **2‑channel** stream to a **mono** stream before LVA sees it. +2. LVA’s wake‑word engines and ESPHome streaming always receive 16 kHz **mono** PCM + (`int16`), regardless of whether the underlying hardware is mono or stereo. + +In practice: + +> You do **not** need any XVF3800‑specific code changes in the audio engine. +> Selecting the XVF3800 as the `audio.input_device` is sufficient; LVA will process it +> as a mono mic just like other devices. + +--- + +## 6. Start LVA as a user-mode service + +The service setup is identical to other configurations (2‑Mic, etc.). Assuming you want +LVA to run as a user‑mode systemd service: + +### 6.1 Enable linger for your user (e.g. `pi`) + +```bash +sudo loginctl enable-linger pi +``` + +### 6.2 Install the systemd user service + +```bash +mkdir -p ~/.config/systemd/user +cp ~/linux-voice-assistant/service/linux-voice-assistant.service ~/.config/systemd/user/linux-voice-assistant.service +``` + +### 6.3 Enable & start + +```bash +systemctl --user daemon-reload +systemctl --user enable --now linux-voice-assistant.service +``` + +Check status: + +```bash +systemctl --user status linux-voice-assistant --no-pager -l +``` + +Look for log lines indicating: + +- LVA connected to Home Assistant +- Wake‑word engine initialized +- Satellite streaming started on wake word + +--- + +## 7. Acoustic Echo Cancellation (AEC) considerations + +The XVF3800 already implements **hardware AEC / echo‑canceling speakerphone** processing. +In many cases, you can rely **only** on the XVF3800’s built‑in AEC and **skip** the +PipeWire/PulseAudio `module-echo-cancel` setup. + +You have three practical options: + +### 7.1 Rely on XVF3800 AEC only (recommended starting point) + +- Do **not** load any user‑mode echo‑cancel module. +- Set `input_device` to the XVF3800 source (or an echo‑cancel source that wraps it, if your + system has one by default). +- Play media (music, games, etc.) through the XVF3800 or another output and test: + - Wake word recognition reliability + - STT quality in noisy playback conditions + +If it works well, this is the simplest and most CPU‑efficient setup. + +### 7.2 Combine XVF3800 with software AEC + +If you already have a PipeWire/PulseAudio echo‑cancel device configured, you can route +XVF3800 through it: + +1. Create an echo‑cancel source/sink pair (see `install_pipewire.md` or + `install_pulseaudio.md`). +2. Use `script/run --list-input-devices` / `--list-output-devices` to find names like + `Echo-Cancel Source` and `pipewire/echo-cancel-sink`. +3. Point LVA at those devices in `config.json`: + + ```json + "audio": { + "input_device": "Echo-Cancel Source", + "input_block_size": 1024, + "output_device": "pipewire/echo-cancel-sink" + } + ``` + +This stacks XVF3800’s hardware processing with additional software AEC. It may help in +particularly difficult acoustic environments, at the cost of some complexity and CPU. + +### 7.3 No AEC at all (not recommended for open‑air speakers) + +You can also run without any AEC (hardware or software) if: + +- You use **headphones** instead of speakers, or +- The device isn’t playing audio while listening (e.g. strictly microphone‑only role). + +In that case, you can treat XVF3800 as “just another USB mic” and ignore its speakerphone +capabilities. + +--- + +## 8. Troubleshooting & debug tips + +### 8.1 Check that LVA sees the XVF3800 + +Re‑run: + +```bash +cd ~/linux-voice-assistant +script/run -- --list-input-devices +script/run -- --list-output-devices +``` + +If `reSpeaker XVF3800 4-Mic Array Analog Stereo` doesn’t appear: + +- Check `arecord -l` / `aplay -l` again. +- Ensure the XVF3800 is connected to the XMOS USB‑C port and powered. +- Check `lsusb` for a device with `2886:001a`: + + ```bash + lsusb | grep -i 2886 + ``` + +### 8.2 Tail the LVA logs + +```bash +journalctl --user -u linux-voice-assistant.service -f +``` + +Look for: + +- “Audio engine started.” +- “Opening audio input device: reSpeaker XVF3800 4-Mic Array Analog Stereo” +- Wake‑word detection lines (MicroWakeWord / OpenWakeWord) + +### 8.3 Optional: XVF3800 probe script + +On the `XVF3800` branch, there is a small probe script in `tests/xvf3800_probe.py` that +can be used to inspect how the audio backend opens the device. + +For example: + +```bash +cd ~/linux-voice-assistant +python tests/xvf3800_probe.py --device "reSpeaker XVF3800 4-Mic Array Analog Stereo" +``` + +This will print: + +- The device list from the `soundcard` backend +- The samplerate and channel count used when opening the stream +- The shape and dtype of the recorded buffer + +It’s mainly a debug tool for development and Discussion threads; you don’t need it for +normal operation. + +--- + +## 9. Next steps: LEDs, button, and DOA integration + +This document covers **audio‑only** integration for the XVF3800. The board also has: + +- A 12‑LED WS2812 ring +- A hardware mute button and mute LED +- Direction‑of‑arrival (DOA) and voice activity indicators + +Future work in the LVA project may add an optional `Xvf3800Controller` that: + +- Talks to the XVF3800 over USB using a Python SDK (pyusb or HID) +- Publishes DOA, VAD, and mute button events to LVA’s internal event bus +- Synchronizes mute state and LED behavior between XVF3800, LVA, and Home Assistant + +For now, the important takeaway is: + +> As soon as the XVF3800 is selected as the input device in `config.json`, the Linux +> Voice Assistant can use it as a high‑quality microphone front end with no additional +> code changes.