A new method of managing heap memory for NUMA architecture

TECHNOLOGY DESCRIPTION

Today’s computers have multiple processing cores to improve performance; however, a bottleneck arises in multi-core processors where there is only one memory controller, as each processor would need to be able to access the same memory. The Non-Uniform Memory Access computing architecture, or NUMA, addresses this bottleneck by giving a separate memory controller to each processing core. Those computing nodes are connected, and memory allocator software dictates how the system allocates resources to computing tasks. Current memory allocators have substantial drawbacks for tasks that require a large amount of remote accesses, have load imbalance among memory controllers, or have interconnect congestion.

The inventors have created a patent pending NUMA memory allocator called NUMAlloc that solves the above memory management problems through four innovations: 1) Binding-based memory management; 2) Interleaved heap; 3) Huge page support; 4) Reduced overhead of migrating objects among freelists. These innovations result in a drop-in, scalable memory allocator that compared to the state-of-the-art allocator speeds computing performance by an average of 13% and speeds up to about 5x for memory intensive applications.

TECHNOLOGY DESCRIPTION

ADVANTAGES

• Increases computing performance over state-of-the-art: Up to 5x faster and 13% faster on average

• Drop-in, scalable solution that is ready to implement in standard operating systems (e.g. Linux, Windows, MacOS)

• First allocator specifically designed for NUMA architecture

• Greater benefits achieved in tasks requiring significant memory allocation management

APPLICATIONS

• Multi-core processor computers

• NUMA-based computers

ABOUT THE LEAD INVENTOR

Professor Tongping Liu is currently an Assistant Professor in the Department of Electrical and Computer Engineering at The University of Massachusetts Amherst, and he is directing MASS Lab (Massachusetts Software Systems Research Lab). Before that, he worked at Computer Science Department of UTSA for five years. He received his PhD from the CS department of UMass with Professor Emery D. Berger (the author of CSRankings.org) in 2014. He received his Master’s degree from Huazhong University of Science and Technology and BS degree from Harbin Institute of Technology.

Dr. Liu worked in industry for many years before he joined his PhD program. During his PhD, he interned at VMware (2009), Samsung Research (2011), IBM T.J. Watson (2012), Huawei US Research Center (2013), and NEC Labs (2014). His research tries to solve realistic problems and has made real impact on both industry and academia. He has been granted three US patents and has five patents applications.

AVAILABILITY:

Available for Licensing and/or Sponsored Research

DOCKET:

UMA 20-039

PATENT STATUS:

Patent Pending

NON-CONFIDENTIAL INVENTION DISCLOSURE

LEAD INVENTOR:

Tongping Liu

CONTACT:

Today’s computers have multiple processing cores to improve performance; however, a bottleneck arises in multi-core processors where there is only one memory controller, as each processor would need to be able to access the same memory. The Non-Uniform Memory Access computing architecture, or NUMA, addresses this bottleneck by giving a separate memory controller to each processing core. Those computing nodes are connected, and memory allocator software dictates how the system allocates resources to computing tasks. Current memory allocators have substantial drawbacks for tasks that require a large amount of remote accesses, have load imbalance among memory controllers, or have interconnect congestion.

The inventors have created a patent pending NUMA memory allocator called NUMAlloc that solves the above memory management problems through four innovations: 1) Binding-based memory management; 2) Interleaved heap; 3) Huge page support; 4) Reduced overhead of migrating objects among freelists. These innovations result in a drop-in, scalable memory allocator that compared to the state-of-the-art allocator speeds computing performance by an average of 13% and speeds up to about 5x for memory intensive applications.