From: Arjan van de Ven <arjan@intel.linux.com>
Date: Sat, 25 Mar 2006 15:30:10 +0000 (+0100)
Subject: [PATCH] x86_64: prefetch the mmap_sem in the fault path
X-Git-Tag: v2.6.17-rc1~705
X-Git-Url: http://pilppa.com/gitweb/?a=commitdiff_plain;h=a9ba9a3b3897561d01e04cd21433746df46548c0;p=linux-2.6-omap-h63xx.git

[PATCH] x86_64: prefetch the mmap_sem in the fault path

In a micro-benchmark that stresses the pagefault path, the down_read_trylock
on the mmap_sem showed up quite high on the profile. Turns out this lock is
bouncing between cpus quite a bit and thus is cache-cold a lot. This patch
prefetches the lock (for write) as early as possible (and before some other
somewhat expensive operations). With this patch, the down_read_trylock
basically fell out of the top of profile.

Signed-off-by: Arjan van de Ven <arjan@linux.intel.com>
Signed-off-by: Andi Kleen <ak@suse.de>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
---

diff --git a/arch/x86_64/mm/fault.c b/arch/x86_64/mm/fault.c
index de91e17daf6..316c53de47b 100644
--- a/arch/x86_64/mm/fault.c
+++ b/arch/x86_64/mm/fault.c
@@ -314,11 +314,13 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs,
 	unsigned long flags;
 	siginfo_t info;
 
+	tsk = current;
+	mm = tsk->mm;
+	prefetchw(&mm->mmap_sem);
+
 	/* get the address */
 	__asm__("movq %%cr2,%0":"=r" (address));
 
-	tsk = current;
-	mm = tsk->mm;
 	info.si_code = SEGV_MAPERR;