LIGHTWEIGHT OUT OF ORDER SCHEDULER FOR GPUs

Lightweight Out of Order Scheduler for GPUs

Princeton Docket # 24-4082-1

Graphics processing units (GPUs) accelerate parallel applications, such as machine learning and autonomous vehicles. Despite abundant parallelism in these applications, many programs still exhibit significant stalls. Previous proposals suggest adding out-of-order (OoO) execution to GPUs, but their high hardware overhead make them impractical. Princeton researchers have developed the first low-cost OoO execution mechanism for GPUs. Its OoO engine supports only the most performant and cost-effective aspects of OoO. This invention applied to an existing GPU processor demonstrated a significant performance improvement with a very low hardware cost.

Applications:

• Improved performance for many existing GPU applications, including those in gaming, scientific computing, machine learning, virtual reality.
• Applicable to GPUs and other types of parallel processors.
• Improved fetch performance for parallel processors even without the out-of-order execution feature.

Advantages:

• High performance per watt and per area.
• Integrates easily with existing processors.
• Does not require complex hardware normally associated with out-of-order processing.
• Does not require programmer or compiler support.

Stage of Development: 

Researchers have tested the invention experimentally by modifying using the accel-sim simulator. For interested parties, researchers have RTL design and model available for testing.

Publication:

GhOST: a GPU Out-of-Order Scheduling Technique for Stall Reduction | IEEE Conference Publication | IEEE Xplore

Inventors:

Ishita Chaturvedi, Ph.D., is a Performance Engineer at Cerebras Systems. She completed from Princeton University advised by Professor David August. Her research focus lies in the domain of High Performance Computing, specifically the efficient utilization of computation hardware resources, performance extraction using GPUs, and unification of the GPU-CPU architectures. Before joining Princeton University, she completed her Bachelors in Physics at Delhi University, India.

Bhargav Reddy Godola, Ph.D. is a Senior CPU Architect at AheadComputing. He completed from Princeton University advised by Professor David August. His research focuses on CPU microarchitecture with a concentration on improving performance of CPU front-end for datacenter workloads. He completed his Bachelors in Computer Science at the Indian Institute of Technology, Hyderabad.

David I. August, Ph.D., is a Professor of Computer Science at Princeton University. He earned his doctoral and master’s degrees in electrical and computer engineering from the University of Illinois at Urbana-Champaign.  Among his professional activities, Professor August was co-program chair for MICRO 2009, and he served on the program committees for ISCA 2007, PLDI 2008, MICRO 2010, ASPLOS 2011, and Top-Picks 2012.  His “Revisiting the Sequential Programming Model for the Multicore Era” was chosen for IEEE Micro’s Top-Picks special issue of papers "most relevant to industry and significant in contribution to the field of computer architecture" in 2007.  He also won the Best Paper Award for “Fault-tolerant Typed Assembly Language” at the 2007 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), in June 2007. His primary research interests are in synergistic compiler and microarchitecture design.

Intellectual Property & Development Status 

Patent protection is pending.

Princeton is currently seeking commercial partners for the further development and commercialization of this opportunity.

Contact

Renee Sanchez, JD

Princeton University Tech Licensing & New Ventures • Phone: (609) 258 6762 • Email: rs1453@princeton.edu