Processor Cache and TLB Control

Caches are crucial in terms of overall system performance, which is why the kernel tries to exploit them as effectively as possible. It does this primarily by skillfully aligning kernel data in memory. A judicious mix of normal functions, inline definitions, and macros also helps extract greater performance from the processor. The compiler optimizations discussed in Appendix C also make their contribution.

However, the above aspects affect the cache only indirectly. Use of the correct alignment for a data structure does indeed have an effect on the cache but only implicitly — active control of the processor cache is not necessary.

Nevertheless, the kernel features some commands that act directly on the cache and the TLB of the processor. However, they are not intended to boost system efficiency but to maintain the cache contents in a consistent state and to ensure that no entries are incorrect and out-of-date. For example, when a mapping is removed from the address space of a process, the kernel is responsible for removing the corresponding entries from the TLBs. If it failed to do so and new data were added at the position previously occupied by the mapping, a read or write operation to the virtual address would be redirected to the incorrect location in physical memory.

The hardware implementation of caches and TLBs differs significantly from architecture to architecture. The kernel must therefore create a view on TLBs and caches that takes adequate account of the different approaches without neglecting the specific properties of each architecture.

□ The meaning of the translation lookaside buffer is abstracted to refer to a mechanism that translates a virtual address into a physical address.38

□ The kernel regards a cache as a mechanism that provides rapid access to data by reference to a virtual address without the need for a request to RAM memory. There is not always an explicit difference between data and instruction caches. The architecture-specific code is responsible for any differentiation if its caches are split in this manner.

It is not necessary for each processor type to implement every control function defined by the kernel. If a function is not required, its invocation can be replaced with an empty operation (do {} while (0)) that is optimized away by the compiler. This is very frequently the case with cache-related operations because, as above, the kernel assumes that addressing is based on virtual addresses. The resultant problems do not occur in physically organized caches so that it is not usually necessary to implement the cache control functions.

The following functions must be made available (even if only as an empty operation) by each CPU-specific part of the kernel in order to control the TLBs and caches39:

□ flush_tlb_all and flush_cache_all flush the entire TLB/cache. This is only required when the page tables of the kernel (and not of a userspace process) are manipulated because a modification of this kind affects not only all processes but also all processors in the system.

□ flush_tlb_mm(struct mm_struct *mm) and flush_cache_mm flush all TLB/cache entries belonging to the address space mm.

□ flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end) and flush_cache_range(vma, start, end) flush all entries from the TLB/cache between the start and end virtual addresses in the address range vma->vm_mm.

□ flush_tlb_page(struct vm_area_struct *vma, unsigned long page) and flush_cache_page(vma, page) flush all entries from the TLB/cache whose virtual addresses are in an interval that begins at page and consists of page_size bytes.

□ update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_tpte) is invoked after a page fault has been handled. It inserts information in the memory management unit of the processor so that the entry at the virtual address address is described by the page table entry pte.

This function is needed only if there is an external MMU. Typically, the MMU is integrated into the processor, but MIPS processors, for example, have external MMUs.

The kernel makes no distinction between data and instruction caches. If a distinction is required, the processor-specific code can reference the vm_exec flag in vm_area_struct->flags to ascertain whether the cache contains data or instructions.

The flush_cache_ and flush_tlb_ functions very often occur in pairs; for instance, when the address space of a process is duplicated using fork.

kernel/fork.c flush_cache_mm(oldmm);

38Whether TLBs are the only hardware resource for doing this or whether other alternatives (e.g., page tables) are provided is irrele-

39The following description is based on the documentation by David Miller [Mil] in the kernel sources.

/* Manipulate page tables */ flush_tlb_mm(oldmm);

The sequence of operations — cache flushing, memory manipulation, and TLB flushing — is important for two reasons:

□ If the sequence were reversed, another CPU in a multiprocessor system could take the wrong information from the process table after the TLBs have been flushed but before the correct information is supplied.

□ Some architectures require the presence of "virtual-to-physical" transformation rules in the TLB when the cache is flushed (caches with this property are referred to as strict). flush_tlb_mm must execute after flush_cache_mm to guarantee that this is the case.

Some control functions apply specifically to data caches (flush_dcache_ ... ) or instruction caches (flush_icache_... ).

□ flush_dcache_page(struct page *page) helps prevent alias problems that arise if a cache may contain several entries (with different virtual addresses) that point to the same page in memory. It is always invoked when the kernel writes to a page in the page cache or when it wants to read data from a page that is also mapped in userspace. This routine gives each architecture in which alias problems can occur an opportunity to prevent such problems.

□ flush_icache_range(unsigned long start, unsigned long end) is invoked when the kernel writes data to kernel memory (between start and end) for subsequent execution. A standard example of this scenario is when a module is loaded into the kernel. The binary data are first copied to RAM and are then executed. flush_icache_range ensures that data and instruction caches do not interfere with each other if implemented separately.

□ flush_icache_user_range(*vma, *page, addr, len) is a special function for the ptrace mechanism. It is needed to propagate changes to the address space of a traced process.

It is beyond the scope of this book to discuss the implementation details of the cache and TLB control functions. Too much background knowledge on the structure of the underlying processor (and the subtle problems involved) would be required for a full understanding of the implementation details.

Continue reading here: Summary

Was this article helpful?

+1 0