Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RISCV] fix: Use WRITE_ONCE() when setting page table entries #398

Merged
merged 7 commits into from
Sep 4, 2024
Merged

[RISCV] fix: Use WRITE_ONCE() when setting page table entries #398

merged 7 commits into from
Sep 4, 2024

Conversation

RevySR
Copy link
Contributor

@RevySR RevySR commented Sep 3, 2024

membarrier: riscv: Add full memory barrier in switch_mm()
riscv: Use WRITE_ONCE() when setting page table entries

Andrea Parri and others added 7 commits September 3, 2024 21:06
@RevySR
membarrier: riscv: Add full memory barrier in switch_mm()
be052d1 
The membarrier system call requires a full memory barrier after storing
to rq->curr, before going back to user-space.  The barrier is only
needed when switching between processes: the barrier is implied by
mmdrop() when switching from kernel to userspace, and it's not needed
when switching from userspace to kernel.

Rely on the feature/mechanism ARCH_HAS_MEMBARRIER_CALLBACKS and on the
primitive membarrier_arch_switch_mm(), already adopted by the PowerPC
architecture, to insert the required barrier.

Fixes: fab957c ("RISC-V: Atomic and Locking Code")
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://lore.kernel.org/r/20240131144936.29190-2-parri.andrea@gmail.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
@palmer-dabbelt @RevySR
Merge patch series "membarrier: riscv: Core serializing command"
df1fec1 
RISC-V was lacking a membarrier implementation for the store/fetch
ordering, which is a bit tricky because of the deferred icache flushing
we use in RISC-V.

* b4-shazam-merge:
  membarrier: riscv: Provide core serializing command
  locking: Introduce prepare_sync_core_cmd()
  membarrier: Create Documentation/scheduler/membarrier.rst
  membarrier: riscv: Add full memory barrier in switch_mm()

Link: https://lore.kernel.org/r/20240131144936.29190-1-parri.andrea@gmail.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
@RevySR
riscv: Use WRITE_ONCE() when setting page table entries
e4a8b43 
To avoid any compiler "weirdness" when accessing page table entries which
are concurrently modified by the HW, let's use WRITE_ONCE() macro
(commit 20a004e ("arm64: mm: Use READ_ONCE/WRITE_ONCE when accessing
page tables") gives a great explanation with more details).

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Link: https://lore.kernel.org/r/20231213203001.179237-2-alexghiti@rivosinc.com
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
@RevySR
mm: Introduce pudp/p4dp/pgdp_get() functions
4cac026 
Instead of directly dereferencing page tables entries, which can cause
issues (see commit 20a004e ("arm64: mm: Use READ_ONCE/WRITE_ONCE when
accessing page tables"), let's introduce new functions to get the
pud/p4d/pgd entries (the pte and pmd versions already exist).

Note that arm pgd_t is actually an array so pgdp_get() is defined as a
macro to avoid a build error.

Those new functions will be used in subsequent commits by the riscv
architecture.

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Link: https://lore.kernel.org/r/20231213203001.179237-3-alexghiti@rivosinc.com
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
@RevySR
riscv: mm: Only compile pgtable.c if MMU
da437b5 
All functions defined in there depend on MMU, so no need to compile it
for !MMU configs.

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Link: https://lore.kernel.org/r/20231213203001.179237-4-alexghiti@rivosinc.com
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
@RevySR
riscv: Use accessors to page table entries instead of direct dereference
de63ddb 
As very well explained in commit 20a004e ("arm64: mm: Use
READ_ONCE/WRITE_ONCE when accessing page tables"), an architecture whose
page table walker can modify the PTE in parallel must use
READ_ONCE()/WRITE_ONCE() macro to avoid any compiler transformation.

So apply that to riscv which is such architecture.

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Acked-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20231213203001.179237-5-alexghiti@rivosinc.com
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
@RevySR
symbol: gpl: export pud_offset/p4d_offset symbol
723b1de 
export pud_offset/p4d_offset symbol

Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
Signed-off-by: Han Gao <rabenda.cn@gmail.com>
@deepin-ci-robot
Copy link

deepin pr auto review

RISC-V: Fix memory barrier on rq->curr modification in scheduler.

Found by:

  • Marek target: riscv64-unknown-elf-gcc -c -o /dev/null /home/marcin/riscv-kernel/linux-5.12-um/arch/riscv/include/linux/sched.h
  • Marek target: riscv64-unknown-elf-gcc -c -o /dev/null /home/marcin/riscv-kernel/linux-5.12-um/arch/riscv/include/linux/sync_core.h

Signed-off-by: Marek Kobus 7f5b850f44d874279132030410830424d850f4@gmail.com

@deepin-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign matrix-wsk for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@RevySR RevySR changed the title riscv: Use WRITE_ONCE() when setting page table entries [RISCV] fix: Use WRITE_ONCE() when setting page table entries Sep 3, 2024
@opsiff
Copy link
Member

opsiff commented Sep 4, 2024

/lgtm

@opsiff opsiff merged commit 6d2bace into deepin-community:linux-6.6.y Sep 4, 2024
5 of 6 checks passed
@RevySR RevySR deleted the deepin/linux-6.6.y/riscv-mm branch September 4, 2024 04:51
opsiff pushed a commit that referenced this pull request Sep 4, 2024
@opsiff
membarrier: riscv: Add full memory barrier in switch_mm()
52a0460 
mainline inclusion
from mainline-v6.9-rc1
link:#398

The membarrier system call requires a full memory barrier after storing
to rq->curr, before going back to user-space.  The barrier is only
needed when switching between processes: the barrier is implied by
mmdrop() when switching from kernel to userspace, and it's not needed
when switching from userspace to kernel.

Rely on the feature/mechanism ARCH_HAS_MEMBARRIER_CALLBACKS and on the
primitive membarrier_arch_switch_mm(), already adopted by the PowerPC
architecture, to insert the required barrier.

Fixes: fab957c ("RISC-V: Atomic and Locking Code")
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://lore.kernel.org/r/20240131144936.29190-2-parri.andrea@gmail.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
(cherry picked from commit d6cfd17)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
opsiff pushed a commit that referenced this pull request Sep 4, 2024
@opsiff
membarrier: Create Documentation/scheduler/membarrier.rst
51166f6 
mainline inclusion
from mainline-v6.9-rc1
link:#398

To gather the architecture requirements of the "private/global
expedited" membarrier commands.  The file will be expanded to
integrate further information about the membarrier syscall (as
needed/desired in the future).  While at it, amend some related
inline comments in the membarrier codebase.

Suggested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://lore.kernel.org/r/20240131144936.29190-3-parri.andrea@gmail.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
(cherry picked from commit a14d11a)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
opsiff pushed a commit that referenced this pull request Sep 4, 2024
@opsiff
locking: Introduce prepare_sync_core_cmd()
3b0a39e 
mainline inclusion
from mainline-v6.9-rc1
link:#398

Introduce an architecture function that architectures can use to set
up ("prepare") SYNC_CORE commands.

The function will be used by RISC-V to update its "deferred icache-
flush" data structures (icache_stale_mask).

Architectures defining prepare_sync_core_cmd() static inline need to
select ARCH_HAS_PREPARE_SYNC_CORE_CMD.

Suggested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://lore.kernel.org/r/20240131144936.29190-4-parri.andrea@gmail.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
(cherry picked from commit 4ff4c74)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
opsiff pushed a commit that referenced this pull request Sep 4, 2024
@opsiff
membarrier: riscv: Provide core serializing command
9b78932 
mainline inclusion
from mainline-v6.9-rc1
link:#398

RISC-V uses xRET instructions on return from interrupt and to go back
to user-space; the xRET instruction is not core serializing.

Use FENCE.I for providing core serialization as follows:

 - by calling sync_core_before_usermode() on return from interrupt (cf.
   ipi_sync_core()),

 - via switch_mm() and sync_core_before_usermode() (respectively, for
   uthread->uthread and kthread->uthread transitions) before returning
   to user-space.

On RISC-V, the serialization in switch_mm() is activated by resetting
the icache_stale_mask of the mm at prepare_sync_core_cmd().

Suggested-by: Palmer Dabbelt <palmer@dabbelt.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://lore.kernel.org/r/20240131144936.29190-5-parri.andrea@gmail.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
(cherry picked from commit cd9b290)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
opsiff pushed a commit that referenced this pull request Sep 4, 2024
@opsiff
riscv: Use WRITE_ONCE() when setting page table entries
7cd282b 
mainline inclusion
from mainline-v6.8-rc1
link:#398

To avoid any compiler "weirdness" when accessing page table entries which
are concurrently modified by the HW, let's use WRITE_ONCE() macro
(commit 20a004e ("arm64: mm: Use READ_ONCE/WRITE_ONCE when accessing
page tables") gives a great explanation with more details).

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Link: https://lore.kernel.org/r/20231213203001.179237-2-alexghiti@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
(cherry picked from commit c30fa83)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
opsiff pushed a commit that referenced this pull request Sep 4, 2024
@opsiff
mm: Introduce pudp/p4dp/pgdp_get() functions
117b687 
mainline inclusion
from mainline-v6.8-rc1
link:#398

Instead of directly dereferencing page tables entries, which can cause
issues (see commit 20a004e ("arm64: mm: Use READ_ONCE/WRITE_ONCE when
accessing page tables"), let's introduce new functions to get the
pud/p4d/pgd entries (the pte and pmd versions already exist).

Note that arm pgd_t is actually an array so pgdp_get() is defined as a
macro to avoid a build error.

Those new functions will be used in subsequent commits by the riscv
architecture.

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Link: https://lore.kernel.org/r/20231213203001.179237-3-alexghiti@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
(cherry picked from commit eba2591)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
opsiff pushed a commit that referenced this pull request Sep 4, 2024
@opsiff
riscv: mm: Only compile pgtable.c if MMU
a03b800 
mainline inclusion
from mainline-v6.8-rc1
link:#398

All functions defined in there depend on MMU, so no need to compile it
for !MMU configs.

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Link: https://lore.kernel.org/r/20231213203001.179237-4-alexghiti@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
(cherry picked from commit d650899)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
opsiff pushed a commit that referenced this pull request Sep 4, 2024
@opsiff
riscv: Use accessors to page table entries instead of direct dereference
416066b 
mainline inclusion
from mainline-v6.8-rc1
link:#398

As very well explained in commit 20a004e ("arm64: mm: Use
READ_ONCE/WRITE_ONCE when accessing page tables"), an architecture whose
page table walker can modify the PTE in parallel must use
READ_ONCE()/WRITE_ONCE() macro to avoid any compiler transformation.

So apply that to riscv which is such architecture.

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Acked-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20231213203001.179237-5-alexghiti@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
(cherry picked from commit edf9556)
Signed-off-by: Wentao Guan <guanwentao@uniontech.com>
@MingcongBai MingcongBai mentioned this pull request Nov 12, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@opsiff opsiff

Labels
lgtm
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@RevySR @deepin-ci-robot @opsiff @palmer-dabbelt