x86: add smp_mb() before sending INVALIDATE_TLB_VECTOR

Impact: fix rare x2apic hang

On x86, x2apic mode accesses for sending IPI's don't have serializing
semantics. If the IPI receivner refers(in lock-free fashion) to some
memory setup by the sender, the need for smp_mb() before sending the
IPI becomes critical in x2apic mode.

Add the smp_mb() in native_flush_tlb_others() before sending the IPI.

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>

diff --git a/arch/x86/kernel/tlb_32.c b/arch/x86/kernel/tlb_32.c
index e00534b3353483dc67bc8282b839c5257a16ca72..f4049f3513b6b5e01e71523a5490fbedd5db9992 100644 (file)
--- a/arch/x86/kernel/tlb_32.c
+++ b/arch/x86/kernel/tlb_32.c
@@ -154,6 +154,12 @@ void native_flush_tlb_others(const cpumask_t *cpumaskp, struct mm_struct *mm,
        flush_mm = mm;
        flush_va = va;
        cpus_or(flush_cpumask, cpumask, flush_cpumask);
+
+       /*
+        * Make the above memory operations globally visible before
+        * sending the IPI.
+        */
+       smp_mb();
        /*
         * We have to send the IPI only to
         * CPUs affected.

diff --git a/arch/x86/kernel/tlb_64.c b/arch/x86/kernel/tlb_64.c
index dcbf7a1159eaf25713b83f6eaf83b1d9379a1fe7..8f919ca69494d62541453202b27f4ec13903d2f8 100644 (file)
--- a/arch/x86/kernel/tlb_64.c
+++ b/arch/x86/kernel/tlb_64.c
@@ -182,6 +182,11 @@ void native_flush_tlb_others(const cpumask_t *cpumaskp, struct mm_struct *mm,
        f->flush_va = va;
        cpus_or(f->flush_cpumask, cpumask, f->flush_cpumask);
 
+       /*
+        * Make the above memory operations globally visible before
+        * sending the IPI.
+        */
+       smp_mb();
        /*
         * We have to send the IPI only to
         * CPUs affected.