[PATCH] mm: dequeue a huge page near to this node

This was discussed at
http://marc.theaimsgroup.com/?l=linux-kernel&m=113166526217117&w=2

This patch changes the dequeueing to select a huge page near the node
executing instead of always beginning to check for free nodes from node 0.
This will result in a placement of the huge pages near the executing
processor improving performance.

The existing implementation can place the huge pages far away from the
executing processor causing significant degradation of performance.  The
search starting from zero also means that the lower zones quickly run out
of memory.  Selecting a huge page near the process distributed the huge
pages better.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: William Lee Irwin III <wli@holomorphy.com>
Cc: Adam Litke <agl@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index da8a211414c94950014365f745f86c5e3e67dfc9..e93bd63462f01b682903af309f7b73c9efc94bbf 100644 (file)
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -40,14 +40,16 @@ static struct page *dequeue_huge_page(void)
 {
        int nid = numa_node_id();
        struct page *page = NULL;
+       struct zonelist *zonelist = NODE_DATA(nid)->node_zonelists;
+       struct zone **z;
 
-       if (list_empty(&hugepage_freelists[nid])) {
-               for (nid = 0; nid < MAX_NUMNODES; ++nid)
-                       if (!list_empty(&hugepage_freelists[nid]))
-                               break;
+       for (z = zonelist->zones; *z; z++) {
+               nid = (*z)->zone_pgdat->node_id;
+               if (!list_empty(&hugepage_freelists[nid]))
+                       break;
        }
-       if (nid >= 0 && nid < MAX_NUMNODES &&
-           !list_empty(&hugepage_freelists[nid])) {
+
+       if (*z) {
                page = list_entry(hugepage_freelists[nid].next,
                                  struct page, lru);
                list_del(&page->lru);