Port HA Voice PE to Onju

[TheStigh]

Hi,

Anybody tried and succeeded porting the PE firmware to Onju (for updated mic, media_player, mww) ?

I have started, and it does detect my wakeword, but do not pick up my command/question.
Also for speaker, I currently get an error when trying to send anything to the media_player.

[Hedda]

While not the same hardware you will probably find a lot more info here in this forum thread about "reSpeaker Lite", though note that it too has an XMOS xCORE chip so obviously you need to remove that. I for example read a recent comment there that if use Music Assistant then it needs an update for each specific player:

https://community.home-assistant.io/t/respeaker-lite-new-seeed-studio-voice-assistant-development-kit-hardware-combine-esp32-with-xmos-xu316-dsp-chip-for-advanced-audio-processing-as-a-esphome-based-home-assistant-assist-satellite-voice-devkit/756944/

Recommend check out the ESPHome config for reSpeaker Lite from formatBCE as well his Koala-Satellite as well as forks of that:

https://github.com/formatBCE/Respeaker-Lite-ESPHome-integration

https://github.com/formatBCE/Koala-Satellite

https://github.com/rananna/Koala-Satellite/tree/main?tab=readme-ov-file

PS: They also said they back-port/update esphome mainline voice-assistant code for ESP32-S3-BOX-3, etc. "Ready-Made Projects" which has a more similar hardware to that of onju-voice:

• https://esphome.io/projects/

  • https://github.com/esphome/firmware/tree/main/voice-assistant/

[Hedda]

As well as check out tetele's ESPHome config for onju-voice-satellite:

• https://github.com/tetele/onju-voice-satellite/

MarkWattTech also has an updated config:

• https://gist.github.com/MarkWattTech/4c248cee42b32d1c5f84091256cc3279

[TheStigh]

Hi @Hedda ,

I've made several custom code versions for the Onju, both with continuous conversation, random greeting messages etc and got full control of the code itself :)

What I wanted to do is implement the updated components for voice_assistant, nabu_microphone and nabu_media_player. I ended up with two different issues yesterday;
1 - The microphone clearly pick up my wakeword, but it does not hear my command (VAD)
2 - Whatever I try to play to the media_player just causes an error (mp3 file etc)

The issue might be the shared i2c bus, but I'm not sure.
I could not import adf_pipeline as prevsiously.

I loaded following in the test configuration:

Full esp32 as for HAVPE:

esp32:
  board: esp32-s3-devkitc-1

Then for external_components:

external_components:
  - source:
      type: git
      url: https://github.com/esphome/home-assistant-voice-pe
      ref: dev
    components:
      - media_player
      - micro_wake_word
      - microphone
      - nabu
      - nabu_microphone
      - voice_assistant
    refresh: 0s

This is the revised settings for audio + media_player + wakeword + voice_assistant:

i2s_audio:
  - id: i2s_shared
    i2s_lrclk_pin: GPIO13
    i2s_bclk_pin: GPIO18
    #access_mode: duplex

microphone:
  - platform: nabu_microphone
    i2s_din_pin: GPIO17
    adc_type: external
    pdm: false
    use_apll: true
    sample_rate: 16000
    bits_per_sample: 32bit
    i2s_mode: primary
    i2s_audio_id: i2s_shared
    channel_0:
      id: asr_mic
      amplify_shift: 0

speaker:
  - platform: i2s_audio
    sample_rate: 48000
    i2s_mode: primary
    i2s_dout_pin: GPIO12
    bits_per_sample: 32bit
    i2s_audio_id: i2s_shared
    dac_type: external
    channel: left
    timeout: 500ms
    buffer_duration: 100ms

media_player:
  - platform: nabu
    id: nabu_media_player
    name: Media Player
    internal: false
    #audio_dac:
    speaker:
    sample_rate: 48000
    volume_increment: 0.05
    volume_min: 0.4
    volume_max: 0.85


micro_wake_word:
  id: mww
  models:
    - model: https://github.com/kahrendt/microWakeWord/releases/download/okay_nabu_20241226.3/okay_nabu.json
      id: okay_nabu
    - model: hey_jarvis
      id: hey_jarvis
    - model: alexa
      id: alexa
    - model: https://github.com/kahrendt/microWakeWord/releases/download/stop/stop.json
      id: stop
      internal: true
  vad:
  microphone: asr_mic
  on_wake_word_detected:
    # If the wake word is detected when the device is muted (Possible with the software mute switch): Do nothing
    - if:
        condition:
          switch.is_on: mute
        then:
          - switch.turn_off: mute
        # Start voice assistant, stop current announcement.
        else:
          - if:
              condition:
                lambda: return id(nabu_media_player)->state == media_player::MediaPlayerState::MEDIA_PLAYER_STATE_ANNOUNCING;
              then:
                lambda: |-
                  id(nabu_media_player)
                    ->make_call()
                    .set_command(media_player::MediaPlayerCommand::MEDIA_PLAYER_COMMAND_STOP)
                    .set_announcement(true)
                    .perform();
              else:
                - voice_assistant.start:
                    wake_word: !lambda return wake_word;

voice_assistant:
  id: va
  microphone: asr_mic
  media_player: nabu_media_player
  micro_wake_word: mww
  use_wake_word: false
  noise_suppression_level: 0
  auto_gain: 0 dbfs
  volume_multiplier: 1

Spot anything wrong?