This is a somewhat low-level question. In x86 assembly there are two SSE instructions:
MOVDQA xmmi, m128
and
MOVNTDQA xmmi, m128
The IA-32 Software Developer's Manual says that the NT in MOVNTDQA stands for Non-Temporal, and that otherwise it's the same as MOVDQA.
My question is, what does Non-Temporal mean?
MOVNTDQA xmmi, m128
is an NT load, while all the other NT instructions are stores, except for prefetchnta
. The accepted answer here only seems to be talking about stores. This is what I've been able to turn up about NT loads. TL:DR: hopefully the CPU does something useful with the NT hint to minimize cache pollution, but they don't override the strongly-ordered semantics of "normal" WB memory, so they do have to use the cache.
Non-Temporal SSE instructions (MOVNTI, MOVNTQ, etc.), don't follow the normal cache-coherency rules. Therefore non-temporal stores must be followed by an SFENCE instruction in order for their results to be seen by other processors in a timely fashion.
When data is produced and not (immediately) consumed again, the fact that memory store operations read a full cache line first and then modify the cached data is detrimental to performance. This operation pushes data out of the caches which might be needed again in favor of data which will not be used soon. This is especially true for large data structures, like matrices, which are filled and then used later. Before the last element of the matrix is filled the sheer size evicts the first elements, making caching of the writes ineffective.
For this and similar situations, processors provide support for non-temporal write operations. Non-temporal in this context means the data will not be reused soon, so there is no reason to cache it. These non-temporal write operations do not read a cache line and then modify it; instead, the new content is directly written to memory.
Source: http://lwn.net/Articles/255364/
Espo is pretty much bang on target. Just wanted to add my two cents:
The "non temporal" phrase means lacking temporal locality. Caches exploit two kinds of locality - spatial and temporal, and by using a non-temporal instruction you're signaling to the processor that you don't expect the data item be used in the near future.
I am a little skeptical about the hand-coded assembly that uses the cache control instructions. In my experience these things lead to more evil bugs than any effective performance increases.
According to the Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 1: Basic Architecture, "Programming with Intel Streaming SIMD Extensions (Intel SSE)" chapter:
Caching of Temporal vs. Non-Temporal Data
Data referenced by a program can be temporal (data will be used again) or non-temporal (data will be referenced once and not reused in the immediate future). For example, program code is generally temporal, whereas, multimedia data, such as the display list in a 3-D graphics application, is often non-temporal. To make efficient use of the processor’s caches, it is generally desirable to cache temporal data and not cache non-temporal data. Overloading the processor’s caches with non-temporal data is sometimes referred to as "polluting the caches". The SSE and SSE2 cacheability control instructions enable a program to write non-temporal data to memory in a manner that minimizes pollution of caches.
Description of non-temporal load and store instructions. Source: Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 2: Instruction Set Reference
LOAD (MOVNTDQA—Load Double Quadword Non-Temporal Aligned Hint)
Loads a double quadword from the source operand (second operand) to the destination operand (first operand) using a non-temporal hint if the memory source is WC (write combining) memory type [...] [...] the processor does not read the data into the cache hierarchy, nor does it fetch the corresponding cache line from memory into the cache hierarchy.
Note that, as Peter Cordes comments, it's not useful on normal WB (write-back) memory on current processors because the NT hint is ignored (probably because there are no NT-aware HW prefetchers) and the full strongly-ordered load semantics apply. prefetchnta
can be used as a pollution-reducing load from WB memory
STORE (MOVNTDQ—Store Packed Integers Using Non-Temporal Hint)
Moves the packed integers in the source operand (second operand) to the destination operand (first operand) using a non-temporal hint to prevent caching of the data during the write to memory. [...] the processor does not write the data into the cache hierarchy, nor does it fetch the corresponding cache line from memory into the cache hierarchy.
Using the terminology defined in Cache Write Policies and Performance, they can be considered as write-around (no-write-allocate, no-fetch-on-write-miss).
Finally, it may be interesting to review John McAlpin notes about non-temporal stores.
MOVNTDQA
only does anything special on WC (uncacheable Write-Combining) memory regions, e.g. video RAM. It's not at all useful on normal WB (write-back) memory on current HW, the NT hint is ignored and the full strongly-ordered load semantics apply. prefetchnta
can be useful, though, as a pollution-reducing load from WB memory. Do current x86 architectures support non-temporal loads (from "normal" memory)?.
Success story sharing
SFENCE
may be not needed. At least in the same thread. Could you also look?sfence
is required for NT stores, whereas it is never required just for normal stores. NT stores aren't ordered with respect to other stores (NT or not), as seen by other threads, without ansfence
. For reads from the same thread that did the stores, however, you never needsfence
: a given thread will always see its own stores in program order, regardless of whether they are NT stores or not.Therefore non-temporal stores must be followed by an SFENCE instruction in order for their results to be seen by other processors in a timely fashion.
I don't know whynon-temporal stores must be followed by an SFENCE
. Thennon-temporal stores
doesn't allow memory reordering?