powerpc/64s: Fix compiler store ordering to SLB shadow area

[ Upstream commit 926bc2f100c24d4842b3064b5af44ae964c1d81c ]

The stores to update the SLB shadow area must be made as they appear
in the C code, so that the hypervisor does not see an entry with
mismatched vsid and esid. Use WRITE_ONCE for this.

GCC has been observed to elide the first store to esid in the update,
which means that if the hypervisor interrupts the guest after storing
to vsid, it could see an entry with old esid and new vsid, which may
possibly result in memory corruption.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Sasha Levin <alexander.levin@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/arch/powerpc/mm/slb.c b/arch/powerpc/mm/slb.c
index 13cfe413b40d48cf0bca57fd757dc63bc01487dc..6d9bf014b3e78173fec41ade3d33d4d8bbc97512 100644 (file)
--- a/arch/powerpc/mm/slb.c
+++ b/arch/powerpc/mm/slb.c
@@ -62,14 +62,14 @@ static inline void slb_shadow_update(unsigned long ea, int ssize,
         * updating it.  No write barriers are needed here, provided
         * we only update the current CPU's SLB shadow buffer.
         */
-       p->save_area[index].esid = 0;
-       p->save_area[index].vsid = cpu_to_be64(mk_vsid_data(ea, ssize, flags));
-       p->save_area[index].esid = cpu_to_be64(mk_esid_data(ea, ssize, index));
+       WRITE_ONCE(p->save_area[index].esid, 0);
+       WRITE_ONCE(p->save_area[index].vsid, cpu_to_be64(mk_vsid_data(ea, ssize, flags)));
+       WRITE_ONCE(p->save_area[index].esid, cpu_to_be64(mk_esid_data(ea, ssize, index)));
 }
 
 static inline void slb_shadow_clear(enum slb_index index)
 {
-       get_slb_shadow()->save_area[index].esid = 0;
+       WRITE_ONCE(get_slb_shadow()->save_area[index].esid, 0);
 }
 
 static inline void create_shadowed_slbe(unsigned long ea, int ssize,