Initial draft xvf3800 tutorial
This commit is contained in:
imonlinux
@@ -0,0 +1,403 @@
|
|||||||
|
# Linux Voice Assistant with ReSpeaker XVF3800 4‑Mic Array — Installation & Configuration Guide
|
||||||
|
|
||||||
|
> Drafted with ChatGPT based on existing `linux-voice-assistant-2mic-install.md` and
|
||||||
|
> community test results from the XVF3800 integration Discussion.
|
||||||
|
|
||||||
|
This guide shows how to use the **Seeed ReSpeaker XVF3800 USB 4‑Mic Array** with the
|
||||||
|
**linux-voice-assistant** (LVA) project on Raspberry Pi or other Linux hosts.
|
||||||
|
|
||||||
|
Unlike the ReSpeaker 2‑Mic HAT, the XVF3800 is a **USB Audio Class 2.0** device:
|
||||||
|
it acts as a “smart microphone + speaker” front end with built‑in **AEC, beamforming,
|
||||||
|
noise suppression, AGC, and echo‑canceling speakerphone** processing.
|
||||||
|
|
||||||
|
LVA treats the XVF3800 as a **standard USB mic/speaker**:
|
||||||
|
- Audio in: 16 kHz, 16‑bit, 2‑channel PCM (downmixed to mono by the audio stack)
|
||||||
|
- Audio out: 16 kHz, 16‑bit, 2‑channel PCM via the XVF3800 headphone jack / speaker
|
||||||
|
|
||||||
|
No custom kernel drivers are required. Plug it in, select it as your audio device in
|
||||||
|
LVA’s `config.json`, and it will “just work” as a high‑quality mic for wake word and
|
||||||
|
ESPHome streaming.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- A Linux host (e.g., Raspberry Pi 4/5) running one of:
|
||||||
|
- Raspberry Pi OS (Bookworm or Trixie)
|
||||||
|
- Debian 13 (trixie) or similar
|
||||||
|
- Other modern Linux with PipeWire or PulseAudio
|
||||||
|
- Python 3.11+ recommended
|
||||||
|
- **ReSpeaker XVF3800 4‑Mic Array**, connected via its **XMOS USB‑C port**
|
||||||
|
- Network access to your Home Assistant instance
|
||||||
|
- The `linux-voice-assistant` repository checked out (optionally on the `XVF3800` branch)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Get the code (XVF3800 branch)
|
||||||
|
|
||||||
|
If you don’t already have the project cloned, you can pull the dedicated XVF3800 branch:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~
|
||||||
|
git clone --branch XVF3800 --single-branch https://github.com/imonlinux/linux-voice-assistant.git
|
||||||
|
cd linux-voice-assistant
|
||||||
|
```
|
||||||
|
|
||||||
|
If you already have the repository, you can switch branches:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/linux-voice-assistant
|
||||||
|
git fetch origin
|
||||||
|
git checkout XVF3800
|
||||||
|
```
|
||||||
|
|
||||||
|
Set up the project (creates a virtualenv, installs dependencies, etc.):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/linux-voice-assistant
|
||||||
|
script/setup
|
||||||
|
```
|
||||||
|
|
||||||
|
> Tip: If you use a different username than `pi`, adjust paths accordingly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Connect and verify the XVF3800
|
||||||
|
|
||||||
|
### 2.1 Plug in the board
|
||||||
|
|
||||||
|
1. Connect the XVF3800 to your Pi/host using the **XMOS USB‑C port** (the one next to
|
||||||
|
the 3.5 mm audio jack).
|
||||||
|
2. If you want local audio output, connect the XVF3800’s headphone jack to an external
|
||||||
|
powered speaker.
|
||||||
|
|
||||||
|
### 2.2 Confirm ALSA sees the device
|
||||||
|
|
||||||
|
```bash
|
||||||
|
arecord -l
|
||||||
|
aplay -l
|
||||||
|
```
|
||||||
|
|
||||||
|
You should see entries similar to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
**** List of CAPTURE Hardware Devices ****
|
||||||
|
card 3: Array [reSpeaker XVF3800 4-Mic Array], device 0: USB Audio [USB Audio]
|
||||||
|
Subdevices: 0/1
|
||||||
|
Subdevice #0: subdevice #0
|
||||||
|
|
||||||
|
**** List of PLAYBACK Hardware Devices ****
|
||||||
|
card 3: Array [reSpeaker XVF3800 4-Mic Array], device 0: USB Audio [USB Audio]
|
||||||
|
Subdevices: 1/1
|
||||||
|
Subdevice #0: subdevice #0
|
||||||
|
```
|
||||||
|
|
||||||
|
The exact card index (`3` above) may differ on your system.
|
||||||
|
|
||||||
|
### 2.3 Quick `arecord` test (optional)
|
||||||
|
|
||||||
|
Replace `3` with your card number:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
arecord -D plughw:3,0 -c 2 -r 16000 -f S16_LE -d 5 -vv /tmp/xvf3800-test.wav
|
||||||
|
aplay /tmp/xvf3800-test.wav
|
||||||
|
```
|
||||||
|
|
||||||
|
You should hear your voice through your default playback device, confirming that:
|
||||||
|
- The XVF3800 is streaming at **16 kHz**, **16‑bit**, **stereo**, and
|
||||||
|
- ALSA can capture from it reliably.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. PipeWire / PulseAudio device mapping
|
||||||
|
|
||||||
|
On a PipeWire or PulseAudio system, check how the XVF3800 shows up at the graph level:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pactl list short sources
|
||||||
|
pactl list short sinks
|
||||||
|
```
|
||||||
|
|
||||||
|
Typical output:
|
||||||
|
|
||||||
|
```text
|
||||||
|
# Sources
|
||||||
|
309 alsa_input.usb-Seeed_Studio_reSpeaker_XVF3800_4-Mic_Array_...-00.analog-stereo PipeWire s16le 2ch 16000Hz RUNNING
|
||||||
|
|
||||||
|
# Sinks
|
||||||
|
308 alsa_output.usb-Seeed_Studio_reSpeaker_XVF3800_4-Mic_Array_...-00.analog-stereo PipeWire s16le 2ch 16000Hz SUSPENDED
|
||||||
|
```
|
||||||
|
|
||||||
|
Key points:
|
||||||
|
|
||||||
|
- The XVF3800 appears as an **analog stereo** source/sink (`2ch 16000 Hz`).
|
||||||
|
- LVA will use these devices through the `soundcard` library.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. LVA’s view of the XVF3800 (input/output devices)
|
||||||
|
|
||||||
|
From the project root, use LVA’s helper flags to inspect devices:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/linux-voice-assistant
|
||||||
|
script/run -- --list-input-devices
|
||||||
|
script/run -- --list-output-devices
|
||||||
|
```
|
||||||
|
|
||||||
|
Example output:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Input devices
|
||||||
|
|
||||||
|
[0] reSpeaker XVF3800 4-Mic Array Analog Stereo
|
||||||
|
```
|
||||||
|
|
||||||
|
```text
|
||||||
|
Output devices
|
||||||
|
|
||||||
|
auto: Autoselect device
|
||||||
|
pipewire: Default (pipewire)
|
||||||
|
pipewire/alsa_output.usb-Seeed_Studio_reSpeaker_XVF3800_4-Mic_Array_...analog-stereo: reSpeaker XVF3800 4-Mic Array Analog Stereo
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
Take note of:
|
||||||
|
|
||||||
|
- The **input device name** for the XVF3800 (e.g. `reSpeaker XVF3800 4-Mic Array Analog Stereo`)
|
||||||
|
- The output device you want to use (often `pipewire` or the specific XVF “analog‑stereo” sink)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Configure LVA audio for XVF3800
|
||||||
|
|
||||||
|
Edit the main config:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nano ~/linux-voice-assistant/linux_voice_assistant/config.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Add or update the `audio` section:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"audio": {
|
||||||
|
"input_device": "reSpeaker XVF3800 4-Mic Array Analog Stereo",
|
||||||
|
"output_device": "pipewire",
|
||||||
|
"input_block_size": 1024
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
- If you want both input and output on the XVF3800:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"audio": {
|
||||||
|
"input_device": "reSpeaker XVF3800 4-Mic Array Analog Stereo",
|
||||||
|
"output_device": "pipewire/alsa_output.usb-Seeed_Studio_reSpeaker_XVF3800_4-Mic_Array_101991441253700076-00.analog-stereo",
|
||||||
|
"input_block_size": 1024
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- If you only want to use the XVF3800 as a **mic** (and another device for playback), set
|
||||||
|
`output_device` to your preferred sink from `--list-output-devices`.
|
||||||
|
|
||||||
|
### 5.1 How channels are handled (important detail)
|
||||||
|
|
||||||
|
The XVF3800 exposes itself as a **2‑channel** source (stereo, 16 kHz). Internally, the
|
||||||
|
LVA `AudioEngine` uses the `soundcard` library and **always requests 1 channel** from the
|
||||||
|
selected input device:
|
||||||
|
|
||||||
|
```python
|
||||||
|
with self.mic.recorder(
|
||||||
|
samplerate=16000,
|
||||||
|
channels=1,
|
||||||
|
blocksize=self.block_size,
|
||||||
|
) as mic_in:
|
||||||
|
audio_chunk_array = mic_in.record(self.block_size).reshape(-1)
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
This has two important implications:
|
||||||
|
|
||||||
|
1. The audio stack (PipeWire/PulseAudio/ALSA) takes care of converting from the XVF3800’s
|
||||||
|
**2‑channel** stream to a **mono** stream before LVA sees it.
|
||||||
|
2. LVA’s wake‑word engines and ESPHome streaming always receive 16 kHz **mono** PCM
|
||||||
|
(`int16`), regardless of whether the underlying hardware is mono or stereo.
|
||||||
|
|
||||||
|
In practice:
|
||||||
|
|
||||||
|
> You do **not** need any XVF3800‑specific code changes in the audio engine.
|
||||||
|
> Selecting the XVF3800 as the `audio.input_device` is sufficient; LVA will process it
|
||||||
|
> as a mono mic just like other devices.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Start LVA as a user-mode service
|
||||||
|
|
||||||
|
The service setup is identical to other configurations (2‑Mic, etc.). Assuming you want
|
||||||
|
LVA to run as a user‑mode systemd service:
|
||||||
|
|
||||||
|
### 6.1 Enable linger for your user (e.g. `pi`)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo loginctl enable-linger pi
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.2 Install the systemd user service
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p ~/.config/systemd/user
|
||||||
|
cp ~/linux-voice-assistant/service/linux-voice-assistant.service ~/.config/systemd/user/linux-voice-assistant.service
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.3 Enable & start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user daemon-reload
|
||||||
|
systemctl --user enable --now linux-voice-assistant.service
|
||||||
|
```
|
||||||
|
|
||||||
|
Check status:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user status linux-voice-assistant --no-pager -l
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for log lines indicating:
|
||||||
|
|
||||||
|
- LVA connected to Home Assistant
|
||||||
|
- Wake‑word engine initialized
|
||||||
|
- Satellite streaming started on wake word
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Acoustic Echo Cancellation (AEC) considerations
|
||||||
|
|
||||||
|
The XVF3800 already implements **hardware AEC / echo‑canceling speakerphone** processing.
|
||||||
|
In many cases, you can rely **only** on the XVF3800’s built‑in AEC and **skip** the
|
||||||
|
PipeWire/PulseAudio `module-echo-cancel` setup.
|
||||||
|
|
||||||
|
You have three practical options:
|
||||||
|
|
||||||
|
### 7.1 Rely on XVF3800 AEC only (recommended starting point)
|
||||||
|
|
||||||
|
- Do **not** load any user‑mode echo‑cancel module.
|
||||||
|
- Set `input_device` to the XVF3800 source (or an echo‑cancel source that wraps it, if your
|
||||||
|
system has one by default).
|
||||||
|
- Play media (music, games, etc.) through the XVF3800 or another output and test:
|
||||||
|
- Wake word recognition reliability
|
||||||
|
- STT quality in noisy playback conditions
|
||||||
|
|
||||||
|
If it works well, this is the simplest and most CPU‑efficient setup.
|
||||||
|
|
||||||
|
### 7.2 Combine XVF3800 with software AEC
|
||||||
|
|
||||||
|
If you already have a PipeWire/PulseAudio echo‑cancel device configured, you can route
|
||||||
|
XVF3800 through it:
|
||||||
|
|
||||||
|
1. Create an echo‑cancel source/sink pair (see `install_pipewire.md` or
|
||||||
|
`install_pulseaudio.md`).
|
||||||
|
2. Use `script/run --list-input-devices` / `--list-output-devices` to find names like
|
||||||
|
`Echo-Cancel Source` and `pipewire/echo-cancel-sink`.
|
||||||
|
3. Point LVA at those devices in `config.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"audio": {
|
||||||
|
"input_device": "Echo-Cancel Source",
|
||||||
|
"input_block_size": 1024,
|
||||||
|
"output_device": "pipewire/echo-cancel-sink"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This stacks XVF3800’s hardware processing with additional software AEC. It may help in
|
||||||
|
particularly difficult acoustic environments, at the cost of some complexity and CPU.
|
||||||
|
|
||||||
|
### 7.3 No AEC at all (not recommended for open‑air speakers)
|
||||||
|
|
||||||
|
You can also run without any AEC (hardware or software) if:
|
||||||
|
|
||||||
|
- You use **headphones** instead of speakers, or
|
||||||
|
- The device isn’t playing audio while listening (e.g. strictly microphone‑only role).
|
||||||
|
|
||||||
|
In that case, you can treat XVF3800 as “just another USB mic” and ignore its speakerphone
|
||||||
|
capabilities.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Troubleshooting & debug tips
|
||||||
|
|
||||||
|
### 8.1 Check that LVA sees the XVF3800
|
||||||
|
|
||||||
|
Re‑run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/linux-voice-assistant
|
||||||
|
script/run -- --list-input-devices
|
||||||
|
script/run -- --list-output-devices
|
||||||
|
```
|
||||||
|
|
||||||
|
If `reSpeaker XVF3800 4-Mic Array Analog Stereo` doesn’t appear:
|
||||||
|
|
||||||
|
- Check `arecord -l` / `aplay -l` again.
|
||||||
|
- Ensure the XVF3800 is connected to the XMOS USB‑C port and powered.
|
||||||
|
- Check `lsusb` for a device with `2886:001a`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lsusb | grep -i 2886
|
||||||
|
```
|
||||||
|
|
||||||
|
### 8.2 Tail the LVA logs
|
||||||
|
|
||||||
|
```bash
|
||||||
|
journalctl --user -u linux-voice-assistant.service -f
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for:
|
||||||
|
|
||||||
|
- “Audio engine started.”
|
||||||
|
- “Opening audio input device: reSpeaker XVF3800 4-Mic Array Analog Stereo”
|
||||||
|
- Wake‑word detection lines (MicroWakeWord / OpenWakeWord)
|
||||||
|
|
||||||
|
### 8.3 Optional: XVF3800 probe script
|
||||||
|
|
||||||
|
On the `XVF3800` branch, there is a small probe script in `tests/xvf3800_probe.py` that
|
||||||
|
can be used to inspect how the audio backend opens the device.
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/linux-voice-assistant
|
||||||
|
python tests/xvf3800_probe.py --device "reSpeaker XVF3800 4-Mic Array Analog Stereo"
|
||||||
|
```
|
||||||
|
|
||||||
|
This will print:
|
||||||
|
|
||||||
|
- The device list from the `soundcard` backend
|
||||||
|
- The samplerate and channel count used when opening the stream
|
||||||
|
- The shape and dtype of the recorded buffer
|
||||||
|
|
||||||
|
It’s mainly a debug tool for development and Discussion threads; you don’t need it for
|
||||||
|
normal operation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Next steps: LEDs, button, and DOA integration
|
||||||
|
|
||||||
|
This document covers **audio‑only** integration for the XVF3800. The board also has:
|
||||||
|
|
||||||
|
- A 12‑LED WS2812 ring
|
||||||
|
- A hardware mute button and mute LED
|
||||||
|
- Direction‑of‑arrival (DOA) and voice activity indicators
|
||||||
|
|
||||||
|
Future work in the LVA project may add an optional `Xvf3800Controller` that:
|
||||||
|
|
||||||
|
- Talks to the XVF3800 over USB using a Python SDK (pyusb or HID)
|
||||||
|
- Publishes DOA, VAD, and mute button events to LVA’s internal event bus
|
||||||
|
- Synchronizes mute state and LED behavior between XVF3800, LVA, and Home Assistant
|
||||||
|
|
||||||
|
For now, the important takeaway is:
|
||||||
|
|
||||||
|
> As soon as the XVF3800 is selected as the input device in `config.json`, the Linux
|
||||||
|
> Voice Assistant can use it as a high‑quality microphone front end with no additional
|
||||||
|
> code changes.
|
||||||
Reference in New Issue
Block a user