tmpfs: optimize clearing when writing
authorHugh Dickins <hughd@google.com>
Tue, 29 May 2012 22:06:39 +0000 (15:06 -0700)
committerLinus Torvalds <torvalds@linux-foundation.org>
Tue, 29 May 2012 23:22:22 +0000 (16:22 -0700)
Nick proposed years ago that tmpfs should avoid clearing its pages where
write will overwrite them with new data, as ramfs has long done.  But I
messed it up and just got bad data.  Tried again recently, it works
fine.

Here's time output for writing 4GiB 16 times on this Core i5 laptop:

before: real 0m21.169s user 0m0.028s sys 0m21.057s
        real 0m21.382s user 0m0.016s sys 0m21.289s
        real 0m21.311s user 0m0.020s sys 0m21.217s

after:  real 0m18.273s user 0m0.032s sys 0m18.165s
        real 0m18.354s user 0m0.020s sys 0m18.265s
        real 0m18.440s user 0m0.032s sys 0m18.337s

ramfs:  real 0m16.860s user 0m0.028s sys 0m16.765s
        real 0m17.382s user 0m0.040s sys 0m17.273s
        real 0m17.133s user 0m0.044s sys 0m17.021s

Yes, I have done perf reports, but they need more explanation than they
deserve: in summary, clear_page vanishes, its cache loading shifts into
copy_user_generic_unrolled; shmem_getpage_gfp goes down, and
surprisingly mark_page_accessed goes way up - I think because they are
respectively where the cache gets to be reloaded after being purged by
clear or copy.

Suggested-by: Nick Piggin <npiggin@gmail.com>
Signed-off-by: Hugh Dickins <hughd@google.com>
Cc: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
mm/shmem.c

index fe5ae6962ab347a9d0f79482fae89b25f78659e1..45c26476f0fca9c3a94642d4c07e847379365765 100644 (file)
@@ -1095,9 +1095,14 @@ repeat:
                shmem_recalc_inode(inode);
                spin_unlock(&info->lock);
 
-               clear_highpage(page);
-               flush_dcache_page(page);
-               SetPageUptodate(page);
+               /*
+                * Let SGP_WRITE caller clear ends if write does not fill page
+                */
+               if (sgp != SGP_WRITE) {
+                       clear_highpage(page);
+                       flush_dcache_page(page);
+                       SetPageUptodate(page);
+               }
                if (sgp == SGP_DIRTY)
                        set_page_dirty(page);
        }
@@ -1307,6 +1312,14 @@ shmem_write_end(struct file *file, struct address_space *mapping,
        if (pos + copied > inode->i_size)
                i_size_write(inode, pos + copied);
 
+       if (!PageUptodate(page)) {
+               if (copied < PAGE_CACHE_SIZE) {
+                       unsigned from = pos & (PAGE_CACHE_SIZE - 1);
+                       zero_user_segments(page, 0, from,
+                                       from + copied, PAGE_CACHE_SIZE);
+               }
+               SetPageUptodate(page);
+       }
        set_page_dirty(page);
        unlock_page(page);
        page_cache_release(page);
@@ -1768,6 +1781,7 @@ static int shmem_symlink(struct inode *dir, struct dentry *dentry, const char *s
                kaddr = kmap_atomic(page);
                memcpy(kaddr, symname, len);
                kunmap_atomic(kaddr);
+               SetPageUptodate(page);
                set_page_dirty(page);
                unlock_page(page);
                page_cache_release(page);