Intel Details Optane With Enterprise DC P4800X PCIe NVMe SSD

Optane is Intel’s application of the 3D Xpoint (crosspoint) architecture that was built in conjunction with Micron, and aims to fill the gap between memory and storage. Its support was originally announced with the launch of the Kaby Lake CPU and supporting chipset, but this is the first time the technology has been used in an actual product.

As with many new technologies, Optane will start its life out in the data center, rather than in the consumer space. The first official product will be the enterprise targeted DC P4800X drive, which is a PCIe 3.0 x4 NVM SSD in an HHHL package (Half Height Half Length). The first available units will be 375GB SSDs costing about $1500, or $2000 with accompanying software – which we’ll get into in a bit. Other densities such as 750GB and 1.5TB will be made available in the future, but not right now.

At face value, the DC P4800X looks like an overpriced enterprise SSD, but it’s performance puts it more in-line with memory than an SSD. In fact, that was the whole point behind 3D Xpoint, to be known as non-volatile memory. The DC P4800X is the first generation product that will sit in a PCIe slot, but moving forward, Optane will be found in M.2, U.2, and surprisingly, DIMM slots (such as DDR4 memory).

Optane is largely in an active experimental phase, not because of a lack of maturity (the technology has been in development for the last 10 years, partnership with Micron in 2013), but because it’s a rather unique (and potentially disruptive) technology. It’s an architecture that will weave memory and storage together and its uses are myriad.

To start with, the DC P4800X can play the part of a high-speed SSD, providing a pool of storage for extremely aggressive databases. This is something we need to clarify as most home users will never be able to push these drives to the workloads it was designed for, as we’re talking 2GB/s continuous simultaneous random read and write operations, 24/7, with very low latency. While other SSDs are capable of 2GB/s or more (with one recent SSDs reporting 6GB/s), the party trick is being able to perform at such speeds while under huge write pressures. The DC P4800X performs basically the same whether it’s read only or subjected to 600MB/s writes at the same time. This is random reads and write, not sequential, and an important distinction between other drives.

The second potential market for these drives is as memory expansion, and something that has a lot of active development taking place. The DIMM modules are not available just yet, but the DC P4800X can be configured as a memory pool addressable by the system over the PCIe interface, and this is where the software comes in.

It’s possible to expand a system’s memory pool with storage using the classic method of paged memory, where data that extends beyond physical memory is written to storage instead. However, not all applications will treat paged memory as real memory and will flat-out refuse to use it in some cases. Intel’s solution is to use middleware that hooks into the system before the OS loads, expanding the reported memory to include the Optane module.

Currently, this expanded memory is limited to about 8x the system memory, but the performance of it was rather impressive, at least by the methods and metrics Intel provided. A dual-socket system with 768GB of RAM was loaded up with a dataset of 675GB and then benchmarked. The same system then had all but a single stick of 64GB RAM removed per CPU (128GB total) and then had four 375GB Optane modules plugged in and configured as RAM with Intel’s middleware solution enabled. The matrix multiplication test showed a 10% performance increase with the expanded memory, and when used with a database, the transactions per second were only reduced by 20%. Under these specific workloads, the Optane as RAM was almost keeping up with the native RAM solution, but exceeded the performance with the multiplication (likely due to the larger memory pool).

Part of this near RAM speed is because of a change to the way the writes are handled. With a normal SSD, the entire block that data needs to be saved to needs to be cleared first, then written, then read to confirm. This is why TRIM and garbage collection are so important to maintain an SSD’s performance over time, as clearing blocks before they are written speeds up the whole write process. With Optane, the blocks do not need to be cleared, and instead can be written to on demand.

One performance aspect of Optane that Intel wanted to highlight was the latency and performance at low queue depths. Most SSDs are measured at obscene queue depths of 32 or above, as it allows a ‘best case’ scenario of what a particular device is capable of. These are unrealistic, even in a data center. The average home user that fully loads their storage will typically reach a queue depth of 2, maybe 4 with virtualization (although if you are just gaming, QD1). In a high impact database server, QD12 is about the limit, QD8 being more common. These are a far cry from the QD32 that SSDs are tested at (or QD128 in some cases). With Optane, Intel was reaching 500,000 IOPS, or 2GB/s, with a queue depth of 11, rather than 32 or 64 found in most other SSDs.

The DC P4800X will be the first Optane module to go up for sale, and we expect to see U.2 modules being released sometime later. For those that are unaware, U.2 has similarities to M.2 in that it has 4x PCIe lanes that it can take advantage of, but instead of being locked to any available real-estate on the motherboard, it uses a stacked connector, similar in height to a pair of SATA connectors. This is meant to replace and be faster than the now defunct SATA Express port. U.2 is mainly used in server gear, but has more recently started to show up on high-end prosumer gear, such as Intel’s X99 platform (Extreme Edition LGA2011 CPUs).

We will have to wait on further details when it comes to Intel’s more consumer friendly launch of Optane Memory that was lightly hinted at with the Kaby Lake launch of the Z270 chipsets, which Intel only mentioned M.2 interfaces at the time (no U.2 or DIMM modules it seems). For now, Optane is strictly server based and you’ll need a very large wallet to afford the cards, but there is nothing stopping you from buying one and plugging it into your own Xeon system, as long the motherboard is compatible (Xeon E5-x6xx V2 or E7-x8xx V2 or above).