[RAMEN9610-21992]mm: thp: make the THP mapcount atomic against __split_huge_pmd_locked()
authorAndrea Arcangeli <aarcange@redhat.com>
Wed, 27 May 2020 23:06:24 +0000 (19:06 -0400)
committerlingsen1 <lingsen1@lenovo.com>
Thu, 1 Apr 2021 13:29:44 +0000 (21:29 +0800)
commita31f82b42759a74c25d41c82dbdc957bc15edbe6
treeda073a8771359b20efb5600e7d80022c42c14ed0
parentef667c7793fc3d5346b96af8aedfaf42b348325b
[RAMEN9610-21992]mm: thp: make the THP mapcount atomic against __split_huge_pmd_locked()

commit c444eb564fb16645c172d550359cb3d75fe8a040 upstream.

Write protect anon page faults require an accurate mapcount to decide
if to break the COW or not. This is implemented in the THP path with
reuse_swap_page() ->
page_trans_huge_map_swapcount()/page_trans_huge_mapcount().

If the COW triggers while the other processes sharing the page are
under a huge pmd split, to do an accurate reading, we must ensure the
mapcount isn't computed while it's being transferred from the head
page to the tail pages.

reuse_swap_cache() already runs serialized by the page lock, so it's
enough to add the page lock around __split_huge_pmd_locked too, in
order to add the missing serialization.

Note: the commit in "Fixes" is just to facilitate the backporting,
because the code before such commit didn't try to do an accurate THP
mapcount calculation and it instead used the page_count() to decide if
to COW or not. Both the page_count and the pin_count are THP-wide
refcounts, so they're inaccurate if used in
reuse_swap_page(). Reverting such commit (besides the unrelated fix to
the local anon_vma assignment) would have also opened the window for
memory corruption side effects to certain workloads as documented in
such commit header.

Change-Id: I702663cf38fd55f76fa2772e4b1cc8b1d0a45a0a
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Suggested-by: Jann Horn <jannh@google.com>
Reported-by: Jann Horn <jannh@google.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Fixes: 6d0a07edd17c ("mm: thp: calculate the mapcount correctly for THP pages during WP faults")
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
mm/huge_memory.c