kpb: add multi-KPB WOV arbiter for 3-keyword DMIC capture with VAD by lgirdwood · Pull Request #11022 · thesofproject/sof

lgirdwood · 2026-07-22T12:06:35Z

EDIT now added test detection in each WoV module to wake on voice frequencies for male, female and children voice frequencies. i.e. each WoV will trigger on a different speaker type.

Adds a new WOV arbiter component and multi-KPB topology to support three simultaneous wake-word detectors sharing a single DMIC, with exclusive host PCM drain arbitration when any keyword fires.

New component: src/audio/wov_arbiter/

3 input pins (one per KPB host-sink), 1 output pin (to host copier)
AMS subscriber for AMS_WOV_DETECT_MSG_UUID: activates the triggering slot and broadcasts WOV_CTRL PAUSE to sibling detectors
On trigger(STOP): deactivates and broadcasts WOV_CTRL RESUME so all detectors return to listening
Idle inputs are silently drained to prevent buffer stalls
UUID 4a5b6c7d-8e9f-4a1b-2c3d-4e5f60718293 registered in uuid-registry

AMS additions (ams_msg.h):

AMS_WOV_DETECT_MSG_UUID: detector -> arbiter on keyword detect
AMS_WOV_CTRL_MSG_UUID: arbiter -> detectors for pause/resume control

detect_test extensions:

IPC4_DETECT_TEST_SET_WOV_SLOT large-config param assigns a slot id (0-2)
On detection: also sends AMS_WOV_DETECT_MSG_UUID with slot_id to arbiter
AMS consumer for WOV_CTRL: sets paused flag; copy() drains buffer without running the detector when paused
The existing energy-threshold detector fires on a hand clap, making detect_test a ready-to-use lab test stub for this topology

Topology: tools/topology/topology2/platform/intel/dmic-wov-multi.conf

Pipeline 100 (Core 0): DMIC Copier -> Mixin (fan-out to 3 Mixouts)
Pipelines 101-102 (Core 0): Mixout -> KPB -> detect_test (slots 0-1)
Pipeline 103 (Core 1): Mixout -> KPB -> detect_test (slot 2)
Pipeline 104 (Core 0): WOV Arbiter -> Host Copier (single PCM stream)
KPB output_pin_binding: pin 0 -> detector (sel_sink), pin 1 -> arbiter (host_sink)

Requires: CONFIG_COMP_WOV_ARBITER, CONFIG_COMP_KPB, CONFIG_COMP_MIXIN_MIXOUT,
CONFIG_AMS, CONFIG_COMP_KWD_DETECT

lgirdwood

Some opens.

Update KPB component to retrieve downstream WOV detector widgets via comp_buffer_get_sink_component(sink) on sel_sink and trigger downstream detector pipelines directly during copy. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

…g copy Ensure all connected mixout components are automatically activated during mixin_copy() so capture stream audio is mixed concurrently to all active WOV detector pipelines. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add VAD gate audio component to pre-filter background silence and gate downstream WOV keyphrase detector pipelines when no voice energy is present. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add AMS message UUIDs, payload structures (wov_detect_payload, wov_ctrl_payload), and command codes for inter-component communication between WOV detectors and the multi-slot WOV arbiter. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add multi-slot WOV arbiter audio component to manage keyword detection events across multiple detector pipelines, dispatch AMS slot activation and control messages, and broadcast pause/resume commands to un-triggered slots. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

…ication - Add multi-slot zero-crossing frequency estimation for Slot 0 (Male 120Hz), Slot 1 (Female 220Hz), and Slot 2 (Child 350Hz). - Require 30ms sustained frequency match (consec_match_count >= 3) and energy > 500 to eliminate transient background room noise triggers. - Map IPC4 pipeline IDs 101/102/103 to slot IDs 0/1/2. - Populate IPC4 phrase detection notifications with WOV detector module ID, instance ID, and slot ID. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

Add topology manifests and configuration files supporting 3-slot WOV capture (Male, Female, Child) with multi-KPB buffer routes, mixin/mixout splitters, wov-arbiter widget bindings, and HDA Jack Mic / DMIC input bindings. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

…tion, and testing Add detailed documentation for multi-slot Wake-On-Voice (WOV) architecture, including Mermaid diagrams for audio dataflow and control messaging, Kconfig requirements, topology syntax, and test procedures on Tiger Lake (Spider/Moth). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

lgirdwood

Needs more inline comments around code blocks and important statements can also do with more context in each commit message. I would put the README.md patch first to give reviewer context first before further review.

lgirdwood · 2026-07-29T15:27:01Z

 		return -ENOMEM;
 	}

+	struct comp_buffer *sink;


lets comment why this is needed, and what we are doing in these 2 new blocks. This does look like it would benefit a utility API.

lgirdwood · 2026-07-29T15:27:56Z

 	}
 #endif /* CONFIG_IPC_MAJOR_4 */

+	if (!kpb->sel_sink || !kpb->host_sink) {


lets comment what this block is doing and why.

lgirdwood · 2026-07-29T15:28:08Z

 		comp_err(dev, "could not find sink: sel_sink %p",
 			 kpb->sel_sink);
 		ret = -EIO;
+	} else {


ditto re comments.

lgirdwood · 2026-07-29T15:29:01Z

-	#UUID: D8218443-5FF3-4A4C-B388-6CFE07B9562E
-	uuid		"43:84:21:d8:f3:5f:4c:4a:b3:88:6c:fe:07:b9:56:2e"
+	#UUID: a8a0cb32-4a77-4db1-85c753d7ee07bce6 (kpb4 static module)
+	uuid		"32:cb:a0:a8:77:4a:b1:4d:85:c7:53:d7:ee:07:bc:e6"


why do we need a new UUID

lgirdwood · 2026-07-29T15:30:04Z

 		if (frames_to_copy + start_frame > mixout_data->mixed_frames)
 			mixout_data->mixed_frames = frames_to_copy + start_frame;
+
+		if (mixout_mod && mixout_mod->dev)


need comment for what we are checking and why.

lgirdwood · 2026-07-29T16:09:59Z

@@ -0,0 +1,455 @@
+// SPDX-License-Identifier: BSD-3-Clause
+//
+// Copyright(c) 2024 Intel Corporation. All rights reserved.


lgirdwood · 2026-07-29T16:10:40Z

+	 */
+	uint8_t active_slot;
+
+#if CONFIG_AMS


we should actually depend on AMS in teh Kconfig and then we can remove all these ifdefs

lgirdwood · 2026-07-29T16:11:30Z

+	if(CONFIG_COMP_VAD_GATE)
+		add_subdirectory(vad_gate)
+	endif()


this should be in the vad gate patch.

lgirdwood · 2026-07-29T16:11:55Z

 e93326d8-0d14-4bf0-bcb9e063d3d80136 twb_sched
 42f8060c-832f-4dbf-b24751e961997b34 up_down_mixer
 6f6b6f4b-6f73-7466-20e1e62b9779f003 userspace_proxy
+8c7d6e5f-4a3b-2c1d-0e9f8a7b6c5d4e3f vad_gate


this should be in vad gate patch

lgirdwood · 2026-07-29T16:13:52Z

+			dai_type "DMIC"
+			copier_type "DMIC"
+			stream_name $DMIC1_NAME


isnt this passed in via build for the target ?

lyakh · 2026-07-30T06:49:10Z


 		if (sink_buf_id == buf_id) {
+			struct comp_dev *sc = comp_buffer_get_sink_component(sink);
+			comp_err(dev, "kpb_bind: buf_id=%d sink_comp=0x%x -> %s",


is this an error condition?

lyakh · 2026-07-30T06:49:57Z

+	comp_dev_for_each_consumer(dev, sink) {
+		struct comp_dev *sc = comp_buffer_get_sink_component(sink);
+		if (sc) {
+			comp_err(dev, "kpb consumer in bsink_list: comp_id=0x%x type=%d sink_buf=%p",


replace comp_err() everywhere in this commit / PR

lyakh · 2026-07-30T06:50:53Z

+		if (sc) {
+			comp_err(dev, "kpb consumer in bsink_list: comp_id=0x%x type=%d sink_buf=%p",
+				 dev_comp_id(sc), sc->drv ? sc->drv->type : -1, sink);
+			if (dev_comp_id(sc) != 0x10)


what is 0x10?

lyakh · 2026-07-30T06:51:58Z

+				kpb->sel_sink = sink;
+			else
+				kpb->host_sink = sink;
+		}


I suppose the loop should run exactly twice to only set each of these pointers once? Should we check or at least add a comment?

lyakh · 2026-07-30T06:52:39Z

+			if (!kpb->sel_sink)
+				kpb->sel_sink = sink;
+			else if (!kpb->host_sink)
+				kpb->host_sink = sink;


same here - shouldn't be overwriting, right?

lyakh · 2026-07-30T07:33:00Z

+}

-LOG_MODULE_REGISTER(kd_test, CONFIG_SOF_LOG_LEVEL);
+LOG_MODULE_REGISTER(kd_test, LOG_LEVEL_INF);


lyakh · 2026-07-30T07:34:00Z

 	struct comp_data *cd = comp_get_drvdata(dev);

-	comp_info(dev, "entry");
+	comp_err(dev, "notify_host: WOV module_id=0x%x instance_id=0x%x slot_id=%u detected",


comp_err() wrong

lyakh · 2026-07-30T07:34:41Z

+	uint8_t slot_id = cd->wov_slot_id;
+	if (slot_id == WOV_SLOT_INVALID) {
+		if (dev->pipeline && dev->pipeline->pipeline_id >= 101 && dev->pipeline->pipeline_id <= 103)
+			slot_id = dev->pipeline->pipeline_id - 101;


what are these magic numbers?

lyakh · 2026-07-30T07:37:01Z

+				freq_match = (freq_hz >= 80 && freq_hz <= 170);
+				voice_type = "MALE";
+				break;
+			case 1: /* Female Voice Range: 175 - 270 Hz */


so 172Hz will be "nobody?" Or are frequencies detected with 5Hz granularity?

lyakh · 2026-07-30T07:38:17Z

+	else if (ppl_id == 102 || ppl_id == 3)
+		cd->wov_slot_id = 1;
+	else if (ppl_id == 103 || ppl_id == 4)
+		cd->wov_slot_id = 2;


these numbers look familiar

lgirdwood requested review from abonislawski, naveen-manohar and udaymb July 22, 2026 12:07

lgirdwood commented Jul 22, 2026

View reviewed changes

Comment thread src/audio/wov_arbiter/wov_arbiter.c

Comment thread src/audio/wov_arbiter/wov_arbiter.c

Comment thread src/audio/CMakeLists.txt Outdated

Comment thread src/audio/Kconfig Outdated

Comment thread src/include/sof/audio/wov_arbiter.h Outdated

Comment thread uuid-registry.txt Outdated

lgirdwood force-pushed the feature/wov-multi-kpb branch from 49f3dc6 to c3c7161 Compare July 22, 2026 13:13

lgirdwood changed the title ~~WIP: audio/topology: add multi-KPB WOV arbiter for 3-keyword DMIC capture~~ WIP: audio/topology: add multi-KPB WOV arbiter for 3-keyword DMIC capture with VAD Jul 23, 2026

TsaiGaggery mentioned this pull request Jul 23, 2026

audio: iadk: support API 4.5.2 loadable detector modules #11032

Closed

lgirdwood force-pushed the feature/wov-multi-kpb branch from 20edbb3 to 0f6f889 Compare July 25, 2026 12:47

lrgirdwo added 8 commits July 29, 2026 16:17

audio: vad_gate: add Voice Activity Detection gate component

54a8f62

Add VAD gate audio component to pre-filter background silence and gate downstream WOV keyphrase detector pipelines when no voice energy is present. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>

lgirdwood force-pushed the feature/wov-multi-kpb branch from 0f6f889 to 3af37c0 Compare July 29, 2026 15:22

lgirdwood marked this pull request as ready for review July 29, 2026 15:23

lgirdwood requested review from dbaluta, fkwasowi, iganakov, jsarha, kv2019i, lbetlej, mmaka1, pblaszko, plbossart, ranj063 and tmleman as code owners July 29, 2026 15:23

lgirdwood changed the title ~~WIP: audio/topology: add multi-KPB WOV arbiter for 3-keyword DMIC capture with VAD~~ kpb: add multi-KPB WOV arbiter for 3-keyword DMIC capture with VAD Jul 29, 2026

lgirdwood commented Jul 29, 2026

View reviewed changes

lyakh requested changes Jul 30, 2026

View reviewed changes

Uh oh!

Conversation

lgirdwood commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lgirdwood commented Jul 22, 2026 •

edited

Loading