Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] generic effect type with fir and iir eq #127

Merged
merged 2 commits into from
Sep 24, 2018

Conversation

juimonen
Copy link

This is a demo/test to try to separate different effect types. Needs a lot of massaging and review. For actually to work, you need corresponding PR's from firmware and tools.

@ranj063
Copy link
Collaborator

ranj063 commented Sep 12, 2018

@juimonen could you please update the commit message for the first commit? It seems to be empty


ret = sof_parse_tokens(scomp, &eq->config, comp_tokens,
ret = sof_parse_tokens(scomp, &fir->config, comp_tokens,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this be moved to load_effect instead of duplicating it for load_iir and load_fir?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes will try to do it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now thinking of this, the ipc struct (including config) is created only in the specialized effect loading. I should create separate config then in effect_load and send it as parameter to special effect loading. IMHO actually does not help too much and makes things little bit more confusing (also need to take care of extra error cases).

if (!iir)
return -ENOMEM;

/* configure mixer IPC message */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo in the comment?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah will fix

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw this also in tonegen...where I copied this part :)

@@ -43,6 +43,8 @@
#define TLV_STEP 1
#define TLV_MUTE 2

#define SOF_EFFECT_DATA_SIZE 156
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the size of the data required for configuring the effect component?
Why should it be limited to 156? Should it be max ipc size - the headers?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is probably a bad name... It is the position of widget priv + comp data + effect type (where the actual eq payload starts). I understood the comp data needs to be there all the time. Now I'm adding the effect type. As the data following is binary blob, I can't rely on the vendor array mechanism to get to the proper position.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see then maybe it should be called PAYLOAD_OFFSET?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also I'm thinking, should this be hardcoded? What if one of the values changes?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean somehow we need to have an "agreement" what is in the priv member before the payload starts and what is the size of it. I thought before the situation was that for pipeline components you need to have the sof_comp_tokens there. if there's something following that (like bytes field) you somehow need to know the offset to that. I guess the vendor token parsing is updating the array pointer, but I'm not sure can that be used (is the position guaranteed to be after tuples). I'm open to suggestions :)

@ranj063
Copy link
Collaborator

ranj063 commented Sep 12, 2018

@juimonen the commit title prefix should follow the standard prefixes we have such as ASOC: SOF: topology

ret = -EINVAL;
break;
}
if (ret < 0)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need an error message here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok will add

/* get any common EFFECT tokens */
ret = sof_parse_tokens(scomp, &config, effect_tokens,
ARRAY_SIZE(effect_tokens), private->array,
SOF_EFFECT_DATA_SIZE);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the last argument could just be le32_to_cpu(private->size) just like in every other case. @juimonen what do you think?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I understood correctly, sof_parse_tokens doesn't like if the size spans the whole priv struct. It assumes it's all vendor token arrays and tries to read the following bytes field as tokens (will fail). So this is limiting the amount and making this actually work.

@juimonen juimonen force-pushed the eq_test branch 2 times, most recently from e2d73b1 to 5d27a12 Compare September 13, 2018 10:57
swidget->private = (void *)eq;
ret = sof_parse_tokens(scomp, &iir->config, comp_tokens,
ARRAY_SIZE(comp_tokens), private->array,
SOF_EFFECT_PAYLOAD_OFFSET);
Copy link
Collaborator

@ranj063 ranj063 Sep 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juimonen I still think this line should be le32_to_cpu(private->size) instead of SOF_EFFECT_PAYLOAD_OFFSET

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my previous comment was lost due to pr update. So sof_parse_tokens, assumes that the private->array consist solely of vendor tokens and it tries to parse them as much as le32_to_cpu(private->size) is. Now if the private has also other stuff like bytes fields the function will try to read that as tokens (and will fail, because array size is random etc.). That's why I have to "limit" the size what is given to token parsing. Not sure then If I have understood the parsing incorrectly...?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juimonen I think sof_parse_tokens only tries to find the tokens in the token_type_array (comp_tokens in the above call) in the private->array and this precisely why I am worried that this wont work if we add things later. The comp_tokens could be anywhere in the private->array and we need to do a full sweep.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm ok. But I guess then we assume that the binary blob mixed with vendor arrays always contains first 4 bytes as size?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lgirdwood any comments how should I parse the bytes field mixed with vendor arrays in the widget private data? Also should I use the abi_hdr for sending the bytes to firmware?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juimonen iirc, this has it's own private data structure and does not use tuples. i.e. you create a private data structure and append the bytes to the end of it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juimonen sorry it took me a while to get back to it. But I understand the sticky situation here. Can we pass this as a private data for the byte control associated with the EQ instead? Then there will not be any mixing of vendor tuples array and byte data.

@juimonen juimonen force-pushed the eq_test branch 3 times, most recently from 5d27a12 to 6daa8b6 Compare September 19, 2018 10:34
eq = kzalloc(sizeof(*eq), GFP_KERNEL);
if (!eq)
/* search possible initial parameters from eq controls */
list_for_each_entry(scontrol, &sdev->kcontrol_list, list) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juimonen what happens if there is more than 1 control. Is that allowed?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean I check that the comp_id of the control matches this components id? Can you set multiple controls from topology, isn't that quite bad already? If there are multiple controls for 1 component then this code will try to take the params from the first control...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juimonen yes you can have as many controls in topology as you want. Thats why we should check for sanity.

if (scontrol->cmd == SOF_CTRL_CMD_BINARY &&
scontrol->comp_id == swidget->comp_id) {
pdata = scontrol->control_data->data;
if (pdata->size > 0 && pdata->magic == SOF_ABI_MAGIC) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there's no private data for the control, will it cause a failure in the component load?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, the data is copied only if the private data exists

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I tried this with "empty" data section and also removing data section completely from control in topology. eq loads fine, but there's no parameters so its passthrough

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK can you please add it in the comments that if private data is empty, it will be passthrough?

int ret;

/* search possible initial parameters from eq controls */
list_for_each_entry(scontrol, &sdev->kcontrol_list, list) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. I think we should check that we have one control or type byte control

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the following line if (scontrol->cmd == SOF_CTRL_CMD_BINARY &&
scontrol->comp_id == swidget->comp_id)

int ret;

/* search possible initial parameters from eq controls */
list_for_each_entry(scontrol, &sdev->kcontrol_list, list) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juimonen also I dont think we need to loop through the list. The widget->kcontrol_news[0] will be the control for this widget. So if we should sure there's only one control for eq and then use that one.

Introduce effect type enumeration and vendor type to
differentiate between effects. We can use these types
to build multieffect pipelines in the topology and parse
those in the sof driver to send the correct ipc messages
to DSP firmware.

Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>
@juimonen
Copy link
Author

@ranj063 updated according to your comments

Parse iir and fir eq from widget data field specifying
the effect type. Parse also optional initial eq params
sent in the eq control's private data.

Signed-off-by: Jaska Uimonen <jaska.uimonen@intel.com>
@lgirdwood lgirdwood merged commit 9d17c65 into thesofproject:topic/sof-dev Sep 24, 2018
/* general purpose EFFECT configuration */
struct sof_ipc_comp_effect {
enum sof_ipc_effect_type type;
} __attribute__((packed));
Copy link
Member

@plbossart plbossart Oct 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lgirdwood @juimonen @singalsu
Are you sure this definition is correct? All soc_ipc_comp_xxx structures start with struct sof_ipc_comp comp;
e.g.

struct sof_ipc_comp_eq_fir {
	struct sof_ipc_comp comp;
	struct sof_ipc_comp_config config;
	uint32_t size;
	unsigned char data[0];
} __attribute__((packed));

here you are missing the comp field so all the address calculations will be off.

struct sof_ipc_comp_effect {
>>> missing comp field?
	enum sof_ipc_effect_type type;
} __attribute__((packed));

I wonder how this might work since there is no way the IPC can read the right fields, this will read uninitialized data.

While I am at it, is there really a need for this IPC? For example, for an EQ, you will have to rely on two IPC messages, one to set the effect type and one to set the actual parameters. Is this a desired feature and can the topology actually support this? I can't recall any other component where we do this.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi,
long time since I checked this...

So as I understand the ipc_comp_effect is never sent to DSP it is mainly helper to used to parse the token from topology, so that we can then parse that particular effects data (which is sent to dsp). Like ipc_frame_type or ipc_dai_type. So actually I don't know why it is called "ipc". I think I copied the structure from the existing types.

And as I understood in the topology the ordering doesn't matter if you are just using tuples and not some byte fields etc?. So you can basically parse the tuples in any order you want...now I might be horribly wrong with this. I think I tested this to work with tuples and I remember that whole thing collapsed if you put bytes field to widget data. The effect type is actually after the general comp data in the topology.

As the dapm "effect" doesn't have any subcategories we need some way to differentiate between effects, like we have now already iir and fir. in the previous demo the dapm widget effect was hardcoded to be iir.

paulstelian97 pushed a commit to paulstelian97/linux that referenced this pull request May 4, 2020
tools: labs: qemu: Makefile: Remove deprecated vlan parameter
aiChaoSONG pushed a commit to aiChaoSONG/linux that referenced this pull request May 6, 2021
aiChaoSONG pushed a commit to aiChaoSONG/linux that referenced this pull request May 6, 2021
plbossart pushed a commit that referenced this pull request Jul 26, 2021
During testing of f263a81 ("bpf: Track subprog poke descriptors correctly
and fix use-after-free") under various failure conditions, for example, when
jit_subprogs() fails and tries to clean up the program to be run under the
interpreter, we ran into the following freeze:

  [...]
  #127/8 tailcall_bpf2bpf_3:FAIL
  [...]
  [   92.041251] BUG: KASAN: slab-out-of-bounds in ___bpf_prog_run+0x1b9d/0x2e20
  [   92.042408] Read of size 8 at addr ffff88800da67f68 by task test_progs/682
  [   92.043707]
  [   92.044030] CPU: 1 PID: 682 Comm: test_progs Tainted: G   O   5.13.0-53301-ge6c08cb33a30-dirty #87
  [   92.045542] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1 04/01/2014
  [   92.046785] Call Trace:
  [   92.047171]  ? __bpf_prog_run_args64+0xc0/0xc0
  [   92.047773]  ? __bpf_prog_run_args32+0x8b/0xb0
  [   92.048389]  ? __bpf_prog_run_args64+0xc0/0xc0
  [   92.049019]  ? ktime_get+0x117/0x130
  [...] // few hundred [similar] lines more
  [   92.659025]  ? ktime_get+0x117/0x130
  [   92.659845]  ? __bpf_prog_run_args64+0xc0/0xc0
  [   92.660738]  ? __bpf_prog_run_args32+0x8b/0xb0
  [   92.661528]  ? __bpf_prog_run_args64+0xc0/0xc0
  [   92.662378]  ? print_usage_bug+0x50/0x50
  [   92.663221]  ? print_usage_bug+0x50/0x50
  [   92.664077]  ? bpf_ksym_find+0x9c/0xe0
  [   92.664887]  ? ktime_get+0x117/0x130
  [   92.665624]  ? kernel_text_address+0xf5/0x100
  [   92.666529]  ? __kernel_text_address+0xe/0x30
  [   92.667725]  ? unwind_get_return_address+0x2f/0x50
  [   92.668854]  ? ___bpf_prog_run+0x15d4/0x2e20
  [   92.670185]  ? ktime_get+0x117/0x130
  [   92.671130]  ? __bpf_prog_run_args64+0xc0/0xc0
  [   92.672020]  ? __bpf_prog_run_args32+0x8b/0xb0
  [   92.672860]  ? __bpf_prog_run_args64+0xc0/0xc0
  [   92.675159]  ? ktime_get+0x117/0x130
  [   92.677074]  ? lock_is_held_type+0xd5/0x130
  [   92.678662]  ? ___bpf_prog_run+0x15d4/0x2e20
  [   92.680046]  ? ktime_get+0x117/0x130
  [   92.681285]  ? __bpf_prog_run32+0x6b/0x90
  [   92.682601]  ? __bpf_prog_run64+0x90/0x90
  [   92.683636]  ? lock_downgrade+0x370/0x370
  [   92.684647]  ? mark_held_locks+0x44/0x90
  [   92.685652]  ? ktime_get+0x117/0x130
  [   92.686752]  ? lockdep_hardirqs_on+0x79/0x100
  [   92.688004]  ? ktime_get+0x117/0x130
  [   92.688573]  ? __cant_migrate+0x2b/0x80
  [   92.689192]  ? bpf_test_run+0x2f4/0x510
  [   92.689869]  ? bpf_test_timer_continue+0x1c0/0x1c0
  [   92.690856]  ? rcu_read_lock_bh_held+0x90/0x90
  [   92.691506]  ? __kasan_slab_alloc+0x61/0x80
  [   92.692128]  ? eth_type_trans+0x128/0x240
  [   92.692737]  ? __build_skb+0x46/0x50
  [   92.693252]  ? bpf_prog_test_run_skb+0x65e/0xc50
  [   92.693954]  ? bpf_prog_test_run_raw_tp+0x2d0/0x2d0
  [   92.694639]  ? __fget_light+0xa1/0x100
  [   92.695162]  ? bpf_prog_inc+0x23/0x30
  [   92.695685]  ? __sys_bpf+0xb40/0x2c80
  [   92.696324]  ? bpf_link_get_from_fd+0x90/0x90
  [   92.697150]  ? mark_held_locks+0x24/0x90
  [   92.698007]  ? lockdep_hardirqs_on_prepare+0x124/0x220
  [   92.699045]  ? finish_task_switch+0xe6/0x370
  [   92.700072]  ? lockdep_hardirqs_on+0x79/0x100
  [   92.701233]  ? finish_task_switch+0x11d/0x370
  [   92.702264]  ? __switch_to+0x2c0/0x740
  [   92.703148]  ? mark_held_locks+0x24/0x90
  [   92.704155]  ? __x64_sys_bpf+0x45/0x50
  [   92.705146]  ? do_syscall_64+0x35/0x80
  [   92.706953]  ? entry_SYSCALL_64_after_hwframe+0x44/0xae
  [...]

Turns out that the program rejection from e411901 ("bpf: allow for tailcalls
in BPF subprograms for x64 JIT") is buggy since env->prog->aux->tail_call_reachable
is never true. Commit ebf7d1f ("bpf, x64: rework pro/epilogue and tailcall
handling in JIT") added a tracker into check_max_stack_depth() which propagates
the tail_call_reachable condition throughout the subprograms. This info is then
assigned to the subprogram's func[i]->aux->tail_call_reachable. However, in the
case of the rejection check upon JIT failure, env->prog->aux->tail_call_reachable
is used. func[0]->aux->tail_call_reachable which represents the main program's
information did not propagate this to the outer env->prog->aux, though. Add this
propagation into check_max_stack_depth() where it needs to belong so that the
check can be done reliably.

Fixes: ebf7d1f ("bpf, x64: rework pro/epilogue and tailcall handling in JIT")
Fixes: e411901 ("bpf: allow for tailcalls in BPF subprograms for x64 JIT")
Co-developed-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Link: https://lore.kernel.org/bpf/618c34e3163ad1a36b1e82377576a6081e182f25.1626123173.git.daniel@iogearbox.net
plbossart pushed a commit that referenced this pull request Oct 4, 2021
Attempt to mount 9p file system as root gives the following kernel panic:

 9pnet_virtio: no channels available for device root
 Kernel panic - not syncing: VFS: Unable to mount root "root" (9p), err=-2
 CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.15.0-rc1+ #127
 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
 Call Trace:
  dump_stack_lvl+0x45/0x59
  panic+0x1e2/0x44b
  ? __warn_printk+0xf3/0xf3
  ? free_unref_page+0x2d4/0x4a0
  ? trace_hardirqs_on+0x32/0x120
  ? free_unref_page+0x2d4/0x4a0
  mount_root+0x189/0x1e0
  prepare_namespace+0x136/0x165
  kernel_init_freeable+0x3b8/0x3cb
  ? rest_init+0x2e0/0x2e0
  kernel_init+0x19/0x130
  ret_from_fork+0x1f/0x30
 Kernel Offset: disabled
 ---[ end Kernel panic - not syncing: VFS: Unable to mount root "root" (9p), err=-2 ]---

QEMU command line:
 "qemu-system-x86_64 -append root=/dev/root rw rootfstype=9p rootflags=trans=virtio ..."

This error is because root_device_name is truncated in prepare_namespace() from
being "/dev/root" to be "root" prior to call to mount_nodev_root().

As a solution, don't treat errors in mount_nodev_root() as errors that
require panics and allow failback to the mount flow that existed before
patch citied in Fixes tag.

Fixes: f9259be ("init: allow mounting arbitrary non-blockdevice filesystems as root")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
vijendarmukunda pushed a commit to vijendarmukunda/linux that referenced this pull request Feb 13, 2024
Like commit 1cf3bfc ("bpf: Support 64-bit pointers to kfuncs")
for s390x, add support for 64-bit pointers to kfuncs for LoongArch.
Since the infrastructure is already implemented in BPF core, the only
thing need to be done is to override bpf_jit_supports_far_kfunc_call().

Before this change, several test_verifier tests failed:

  # ./test_verifier | grep # | grep FAIL
  thesofproject#119/p calls: invalid kfunc call: ptr_to_mem to struct with non-scalar FAIL
  thesofproject#120/p calls: invalid kfunc call: ptr_to_mem to struct with nesting depth > 4 FAIL
  thesofproject#121/p calls: invalid kfunc call: ptr_to_mem to struct with FAM FAIL
  thesofproject#122/p calls: invalid kfunc call: reg->type != PTR_TO_CTX FAIL
  thesofproject#123/p calls: invalid kfunc call: void * not allowed in func proto without mem size arg FAIL
  thesofproject#124/p calls: trigger reg2btf_ids[reg->type] for reg->type > __BPF_REG_TYPE_MAX FAIL
  thesofproject#125/p calls: invalid kfunc call: reg->off must be zero when passed to release kfunc FAIL
  thesofproject#126/p calls: invalid kfunc call: don't match first member type when passed to release kfunc FAIL
  thesofproject#127/p calls: invalid kfunc call: PTR_TO_BTF_ID with negative offset FAIL
  thesofproject#128/p calls: invalid kfunc call: PTR_TO_BTF_ID with variable offset FAIL
  thesofproject#129/p calls: invalid kfunc call: referenced arg needs refcounted PTR_TO_BTF_ID FAIL
  thesofproject#130/p calls: valid kfunc call: referenced arg needs refcounted PTR_TO_BTF_ID FAIL
  thesofproject#486/p map_kptr: ref: reference state created and released on xchg FAIL

This is because the kfuncs in the loaded module are far away from
__bpf_call_base:

  ffff800002009440 t bpf_kfunc_call_test_fail1    [bpf_testmod]
  9000000002e128d8 T __bpf_call_base

The offset relative to __bpf_call_base does NOT fit in s32, which breaks
the assumption in BPF core. Enable bpf_jit_supports_far_kfunc_call() lifts
this limit.

Note that to reproduce the above result, tools/testing/selftests/bpf/config
should be applied, and run the test with JIT enabled, unpriv BPF enabled.

With this change, the test_verifier tests now all passed:

  # ./test_verifier
  ...
  Summary: 777 PASSED, 0 SKIPPED, 0 FAILED

Tested-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants