bpf: Fix off-by-one for area size in creating mask to left

commit 10d2bb2e6b1d8c4576c56a748f697dbeb8388899 upstream.

retrieve_ptr_limit() computes the ptr_limit for registers with stack and
map_value type. ptr_limit is the size of the memory area that is still
valid / in-bounds from the point of the current position and direction
of the operation (add / sub). This size will later be used for masking
the operation such that attempting out-of-bounds access in the speculative
domain is redirected to remain within the bounds of the current map value.

When masking to the right the size is correct, however, when masking to
the left, the size is off-by-one which would lead to an incorrect mask
and thus incorrect arithmetic operation in the non-speculative domain.
Piotr found that if the resulting alu_limit value is zero, then the
BPF_MOV32_IMM() from the fixup_bpf_calls() rewrite will end up loading
0xffffffff into AX instead of sign-extending to the full 64 bit range,
and as a result, this allows abuse for executing speculatively out-of-
bounds loads against 4GB window of address space and thus extracting the
contents of kernel memory via side-channel.

Fixes: 979d63d50c0c ("bpf: prevent out of bounds speculation on pointer arithmetic")
Signed-off-by: Piotr Krysiuk <piotras@gmail.com>
Co-developed-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 615a2e44d2a095d85ae39d582a4647927d8eb7f1..393f1209a776ff3d422b4cdf9dfbcbf6660cef49 100644 (file)
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2035,13 +2035,13 @@ static int retrieve_ptr_limit(const struct bpf_reg_state *ptr_reg,
        case PTR_TO_STACK:
                off = ptr_reg->off + ptr_reg->var_off.value;
                if (mask_to_left)
-                       *ptr_limit = MAX_BPF_STACK + off;
+                       *ptr_limit = MAX_BPF_STACK + off + 1;
                else
                        *ptr_limit = -off;
                return 0;
        case PTR_TO_MAP_VALUE:
                if (mask_to_left) {
-                       *ptr_limit = ptr_reg->umax_value + ptr_reg->off;
+                       *ptr_limit = ptr_reg->umax_value + ptr_reg->off + 1;
                } else {
                        off = ptr_reg->smin_value + ptr_reg->off;
                        *ptr_limit = ptr_reg->map_ptr->value_size - off;