mm/page_alloc: speed up the iteration of max_order
authorMuchun Song <songmuchun@bytedance.com>
Tue, 15 Dec 2020 03:11:25 +0000 (19:11 -0800)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Wed, 22 Sep 2021 09:42:57 +0000 (11:42 +0200)
commitc88c1fd8861ea33933917b5d4ed3c1d167cb8118
tree87e51c26c074736df8012fbed2e463673cfc3016
parent43b225cb64c260023bb4b7f511317c468e3259e1
mm/page_alloc: speed up the iteration of max_order

commit 7ad69832f37e3cea8557db6df7c793905f1135e8 upstream.

When we free a page whose order is very close to MAX_ORDER and greater
than pageblock_order, it wastes some CPU cycles to increase max_order to
MAX_ORDER one by one and check the pageblock migratetype of that page
repeatedly especially when MAX_ORDER is much larger than pageblock_order.

We also should not be checking migratetype of buddy when "order ==
MAX_ORDER - 1" as the buddy pfn may be invalid, so adjust the condition.
With the new check, we don't need the max_order check anymore, so we
replace it.

Also adjust max_order initialization so that it's lower by one than
previously, which makes the code hopefully more clear.

Link: https://lkml.kernel.org/r/20201204155109.55451-1-songmuchun@bytedance.com
Fixes: d9dddbf55667 ("mm/page_alloc: prevent merging between isolated and other pageblocks")
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Reviewed-by: Oscar Salvador <osalvador@suse.de>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
mm/page_alloc.c