[COMMON] mm: gup: drain pagevec if FOLL_CMA is set

Some users of get_user_pages() give FOLL_CMA not to pin CMA pages.
However a CMA page may not be migrated even though FOLL_CMA is set
because it is not in LRU but in per-cpu pagevec.
The previous approach to solve this problem is the invocation of
migrate_prep_local() that drains the pagevec of the local CPU. It was
the only choice in if it is called in __need_migrate_cma_page(). It is
because __need_migrate_cma_page() is called under a spinlock of page
table is held. This approach unfortunately does not drain pagevecs of
other CPUs and makes CMA fail to allocate physically contiguous memory
at last.
We need a different approach to get over the problem: clearing pagevec
of all CPUs if FOLL_CMA is set unconditionally. It degrades the
performance of get_user_pages() with FOLL_CMA if no CMA page is found
in the given range. But it is not a problem because there is very few
user that gives FOLL_CMA to get_user_pages().

Change-Id: Idcd651d5e2bc7ec66d17b3296e742b7e714f4b48
Signed-off-by: Cho KyongHo <pullip.cho@samsung.com>

diff --git a/mm/gup.c b/mm/gup.c
index b7183fd5a7770d2e8015f294f2c78a23fa49071b..1f2cc56856cf6ccc15371bbc36a937da2d1e3858 100644 (file)
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -49,10 +49,10 @@ static bool __need_migrate_cma_page(struct page *page,
        if (!(flags & FOLL_CMA))
                return false;
 
-       migrate_prep_local();
-
-       if (!PageLRU(page))
+       if (WARN_ON(!PageLRU(page))) {
+               __dump_page(page, "non-lru cma page");
                return false;
+       }
 
        return true;
 }
@@ -783,6 +783,9 @@ static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
        if (!(gup_flags & FOLL_FORCE))
                gup_flags |= FOLL_NUMA;
 
+       if ((gup_flags & FOLL_CMA) != 0)
+               migrate_prep();
+
        do {
                struct page *page;
                unsigned int foll_flags = gup_flags;