More isn’t always better. AMD and Intel brag about their triple-digit server-processor core counts but also offer models with only eight CPUs. The matrix of customers and applications fuels diverse server requirements and, by extension, diverse processor requirements. For a given customer-application combination, a particular server configuration sits in the sweet spot, balancing capability and cost.
Processors with 100 or more physical CPUs, for example, enable cloud providers to carve up a system into smaller virtual machines rented out to multiple customers. Those with 8 or 16 cores but a full complement of PCIe lanes can attach to a multitude of solid-state drives (SSDs) and host a storage server. In between these extremes are AMD EPYC and Intel Xeons with various core counts. A large enterprise analyzing data from millions of transactions per day or planning production through fulfillment needs bigger servers with beefier processors than a smaller company in the same industry.
DRAM Dimensions Define Servers
The choice of scale affects other server dimensions as well, most prominently main memory (DRAM). To the first order, processing and memory requirements are correlated. A processor is like a water pump: high-throughput pumps (CPUs) need bigger reservoirs (memory) to hold water (data) than slower pumps.
Affecting the sweet-spot configuration, some applications need disproportionately large memory. In-memory databases, key-value caches, and enterprise resource planning software are all sensitive to DRAM capacity. Too little memory, and the application chokes, spending so much time swapping that it violates the IT system’s service-level agreements or fails altogether.
Redis and Memcached, for example, lightly load processors but are memory limited. Weaviate and SAP HANA heavily load processors and need big memories to hold their large data sets. Beyond the amount needed for the data, additional capacity must be available to handle occasional spikes from memory-intensive operations (e.g., copy-on-write doubles buffer sizes). Client connections, overhead for garbage collection, and unused portions of memory chunks all escalate memory-capacity requirements.
Overprovisioning Mitigates Risk but Raises Costs
A conservative approach is for an IT manager to estimate processing and memory requirements and configure a server, giving up some performance to reduce hardware expenses. To maintain SLAs and avoid catastrophic failure, the IT pro can add memory to achieve a safety margin. However, this raises capital expenditure owing to DRAM’s high cost. A server may integrate a $3,000 processor and have $6,000 worth of memory. Doubling the memory to be safe increases system cost by almost 50%. Mass storage is a lot cheaper, however. Less than $1,000 gets the IT manager a couple of SSDs holding a few terabytes.
In theory, virtual memory solves this problem. A processor’s memory-management unit (MMU) helps the operating system (OS) choose memory pages to swap out to storage, such as an SSD array, freeing them for other uses. Through virtual memory, a program can address much more memory than is physically installed. The focus of OS developers has been on selecting the best page to swap out, typically choosing one that isn’t frequently or recently used. More sophisticated analysis, however, could yield a different choice. Moreover, the question of which page to swap in is unaddressed. The OS doesn’t begin fetching a page until it’s needed, stopping program execution until the swap-in process completes, tanking performance.
MEXT Loads Pages Before They’re Needed
Startup MEXT addresses this challenge. Instead of only asking which page to swap out next, its software predicts which page to bring in next. A simple algorithm like choosing the least often or recently used page has been suboptimal but adequate for selecting what to evict. That approach, however, isn’t accurate enough to identify a candidate page to bring in. MEXT’s software, therefore, employs various artificial-intelligence techniques to learn memory-access patterns and make accurate predictions, preemptively loading pages. It runs alongside enterprise software and doesn’t require a modified OS kernel, simplifying its deployment.
With the MEXT technology, an IT manager no longer needs to estimate memory requirements to the precise byte and add a couple of DRAM sticks for good measure. After all, that extra memory would sit unused most of the time. Instead, IT can configure a Goldilocks server—not too expensive and not at risk of falling down during memory-intensive operations. When building a sweet-spot server, more isn’t always better.
MEXT sponsored this post. For more information about the company and its products, go to www.mext.ai.