powerpc/e6500: hw tablewalk: clear TID in kernel indirect entries

Previously TID was being cleared before the tlbsx, but not after.  This
can lead to a multiway hit between a TLB entry with TID=0 (previously
inserted when PID=0) and a TLB entry with TID!=0 that matches PID.
This can theoretically result in undefined behavior, though we probably
get lucky due to the details of the overlap.  It also results in the
inability to use multihit detection to detect other conflicting TLB
entries, as well as poorer TLB utilization due to duplicating kernel
TLB entries.

Rather than try to patch up MAS1 after tlbsx, the entire value is
saved/restored as with MAS2.

I observed a slight improvement in TLB miss performance with this patch
applied.

Signed-off-by: Scott Wood <scottwood@freescale.com>
Reported-by: Ed Swarthout <ed.swarthout@freescale.com>

diff --git a/arch/powerpc/mm/tlb_low_64e.S b/arch/powerpc/mm/tlb_low_64e.S
index 356e8b41fb09d8e41e5fa36391238fbfa4353831..3298d1039e80e9fb6cc0a826bae997fd0d80b3a5 100644 (file)
--- a/arch/powerpc/mm/tlb_low_64e.S
+++ b/arch/powerpc/mm/tlb_low_64e.S
@@ -322,19 +322,17 @@ tlb_miss_common_e6500:
        b       1b
        .previous
 
-       mfspr   r15,SPRN_MAS2
+       mfspr   r15,SPRN_MAS1
+       mfspr   r10,SPRN_MAS2
 
        tlbsx   0,r16
+       mtspr   SPRN_MAS2,r10
        mfspr   r10,SPRN_MAS1
+       mtspr   SPRN_MAS1,r15
+
        andis.  r10,r10,MAS1_VALID@h
        bne     tlb_miss_done_e6500
 
-       /* Undo MAS-damage from the tlbsx */
-       mfspr   r10,SPRN_MAS1
-       oris    r10,r10,MAS1_VALID@h
-       mtspr   SPRN_MAS1,r10
-       mtspr   SPRN_MAS2,r15
-
        /* Now, we need to walk the page tables. First check if we are in
         * range.
         */