From: Ingo Molnar Date: Wed, 25 Feb 2009 07:22:20 +0000 (+0100) Subject: x86: usercopy: check for total size when deciding non-temporal cutoff X-Git-Url: http://pilppa.com/gitweb/?a=commitdiff_plain;h=95108fa34a83ffd97e0af959e4b28d7c62008781;p=linux-2.6-omap-h63xx.git x86: usercopy: check for total size when deciding non-temporal cutoff Impact: make more types of copies non-temporal This change makes the following simple fix: 30d697f: x86: fix performance regression in write() syscall A bit more sophisticated: we check the 'total' number of bytes written to decide whether to copy in a cached or a non-temporal way. This will for example cause the tail (modulo 4096 bytes) chunk of a large write() to be non-temporal too - not just the page-sized chunks. Cc: Salman Qazi Cc: Nick Piggin Cc: Linus Torvalds Signed-off-by: Ingo Molnar --- diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h index a748253db0c..dcaa0404cf7 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -198,7 +198,7 @@ static inline int __copy_from_user_nocache(void *dst, const void __user *src, * non-temporal stores here. Smaller writes get handled * via regular __copy_from_user(): */ - if (likely(size >= PAGE_SIZE)) + if (likely(total >= PAGE_SIZE)) return __copy_user_nocache(dst, src, size, 1); else return __copy_from_user(dst, src, size); @@ -207,7 +207,7 @@ static inline int __copy_from_user_nocache(void *dst, const void __user *src, static inline int __copy_from_user_inatomic_nocache(void *dst, const void __user *src, unsigned size, unsigned total) { - if (likely(size >= PAGE_SIZE)) + if (likely(total >= PAGE_SIZE)) return __copy_user_nocache(dst, src, size, 0); else return __copy_from_user_inatomic(dst, src, size);