CXL - Compute Express Link

Duration: 2 - 5 days

Student Review:

"Instructor provided a lot of additional information and implementation details that were not provided in the CXL specification." - Francisco (Broadcom)

Course Image

CXL Fundamentals eLearning Course

Course Image

CXL 3.1 Update eLearning Course

Course Image

CXL 2.0 eLearning Course

Overview

Compute Express Link (CXL) is a high-bandwidth, low-latency serial bus interconnect between host processors and devices such as accelerators, memory controllers/buffers, and I/O devices. CXL 3.0 is based on PCI Express (PCIe) 6.0 physical layer running at 64 GT/s with x16, x8 and x4 link widths. Degraded modes run at 32 GT/s, 16 GT/s and 8 GT/s with x2 and x1 link widths.

 

CXL interconnect adds coherency and memory semantics, thus allowing for its application in heterogeneous processing systems with a variety of host processors, memory subsystems and peripheral devices interconnected. CXL has applications in standard computer systems, Artificial Intelligence (AI), Machine Learning, communication systems, and High-Performance Computing (HPC). Emerging applications require a diverse mix of CPUs, GPUs, FPGAs, peripherals such as smart NICs, and other accelerators interconnected via an open industry standard protocol with the necessary features which CXL provides. CXL provides a rich set of three protocols that include 1) CXL.io based on PCIe TLP based transactions, 2) CXL.cache and 3) CXL.mem semantics. CXL uses the PCIe stack offering full interoperability with PCIe. CXL.cachemem protocols allow for coherent transactions in memory space. These transactions employ Flit-based packet routing on the Link.

 

MindShare’s comprehensive CXL 3.0 Architecture course provides a solid foundation of platform architectures and use cases of the three CXL protocols with Type 1, Type 2 and Type 3 devices in platform example. The course details the role of the Transaction Layer protocol, Link Layer including Flit packing and unpacking, ARB/MUX and Flex Bus Logical and Electrical Physical Layer of a CXL port design. We explain enumeration and configuration process during system bring-up with details of configuration and component registers. Other topics include switch architecture, reset, manageability, RAS features, power management, performance considerations and compliance testing. The course describes new features added to the CXL 3.0 spec such as 256B and Port Based Routing (PBR) Flits, CXL.mem Back-Invalidation, Dynamic Capacity Devices (DCD), LD-FAM and G-FAM Devices, Global Integrated Memory (GIM), Multi-Headed (MH) Devices, Hierarchy Based Routing (HBR) and Port Based Routing (PBR), Direct Peer-to-Peer Routing and lots more.

 

All of MindShare's classroom and virtual classroom courses can be customized to fit the needs of your group.

What You Will Learn

  • CXL system architectures with Type 1, Type 2 and Type 3 device
  • CXL transaction protocol for CXL.io and CXL.cachemem including support for back-invalidation to CXL.cache protocol and device scaling
  • CXL port design layers such as Transaction, Link, ARB/MUX and Flex Bus Physical Layers
  • CXL switch architecture including new features to CXL 3.0 HBR/PBR Switches (optional topic)
  • Enumeration and initialization issues with Configuration and Component register definitions
  • Power management
  • Reliability, Availability, Serviceability (RAS) and error handling features
  • Considerations to improve protocol performance
  • CXL 3.0 Spec added features such as 256B Flit support, G-FAM/LD-FAM Devices, Dynamic Capacity Devices (DCD), CXL Fabrics etc.

Course Outline

 

  5-Day Class 5-Day Class 2-Day Class 5-Day Class 4-Day Class
Topics CXL 3.1 w/ Switches CXL 3.1 w/o Switches CXL 3.1 Update CXL 2.0 w/ Switches CXL 2.0 w/o Switches
CXL Features and Architecture Overview          
CXL 1.1, CXL 2.0 and CXL 3.x Features X X X X X
CXL.mem, CXL.cache and CXL.io X X X X X
Type 1 (with cache), Type 2 (with cache and memory, Type 3 (with memory) Devices X X X X X
Switch and MLD device architecture overview X X X X X
CXL 3.x features overview X X X    
Layered architecture overview X X   X X
Transaction flows in platforms with PCIe, CXL Type 1, 2, 3 Devices X X Update X X
Cache coherency and MESI protocol tutorial   X   X X
Transaction Layer          
CXL.mem protocol X X Update X X
CXL.cache protocol  X X Update X X
Back-invalidation and CXL.cache device scaling X X X    
CXL.io protocol X X Update X X
PTH and UIO X X X    
Ordering Rules     X   X
Link Layer          
IO Flit Packing X X X X X
Cache/Mem Flit Packing X X X X X
Control Flits X X   X X
68B Flits X X   X X
256B and PBR Flits X X X    
Cachemem Link Layer Retry (LLR) mechanism   X   X X
Flow Control   X   X X
Viral Feature       X X
ARB/MuX Layer          
Arbitration and Flit Multiplexing/DeMuxing  X X   X X
vLSM and Link Power Management    X   X X
ALMP Flits   X   X X
Physical Layer          
Protocol ID  X X      
Flit Header X X X X X
NOP Flits, IDLE Flits, Latency Optimized Flits X X X    
CRC, FEC X X X    
Synch Header Bypass X X   X X
1b/1b encoding X X X    
Link Training X X X X X
Reset          
Cold reset   X   X X
Warm reset   X   X X
Hot reset   X   X X
CXL reset   X   X X
Function Level reset   X   X X
Power Management          
Runtime control PM          
Physical Layer PM          
CXL.io PM          
CXL.cachemem PM          
RAS and Error Handling          
EFN Message   X   X X
Viral Handling   X   X X
Error Logging   X   X X
Enumeration and Manageability          
Enumeration in 1.1 and 2.0 Platforms X X   X X
Memory Interleaving  X X   X X
Memory Interleaving in CXL 3.x X X X    
Control and Status Registers          
Configuration registers and DVSEC X X   X X
Component Registers X X   X X
Backwards compatibiity to CXL 1.1 X X   X X
CXL 3.x registers X X X    
CXL 2.0 Switches          
Architectural overview with configuration options X     X  
Various switch types X     X  
Transaction forwarding X     X  
Virtual Hierachy X     X  
Fabric Manager and commands X     X  
CXL 3.x Switches          
Hierarchy Based Routing (HBR) X   X X  
Port Based Rounting (PBR) X   X    
CXL Fabrics X   X    
G-FAM Access Endpoint (GAE) X   X    
FAST, LDST and ID-based Rerouting X   X    
CXL 3.x Device Types          
LD-FAM, G-FAM (GFD) X X X    
Multi-Headed SLD, MLD and GFD Devices X X X    
Dynamic Capacity Device (DCD) X X X    
Global Integrated Memory (GIM) X X X    
Shared Memory verses Pooled Memory X X X    

 

Training Materials

Prerequisites

Good working knowledge of PCI Express is a must. Computer architecture fundamentals. Knowledge of Intel, AMD or Arm processor architectures. Knowledge of cache coherency and MESI protocol is recommended.

Related Courses

Course NamePDFClassroomVirtual ClassroomeLearning
CXL 3.1 w/ Switch Architecture5 days10 half-days Show Me
CXL 3.1 w/o Switch Architecture5 days10 half-days Show Me
CXL 3.1 Update2 days4 half-days Show Me
CXL 2.0 w/ Switch Architecture5 days10 half-days Show Me
CXL 2.0 w/o Switch Architecture4 days8 half-days Show Me
CXL Fundamentals1 day2 half-days Show Me

CXL 3.1 w/ Switch Architecture

Class: 5 days

Virtual Classroom: 10 half-days

ELEARNING: Show Me

CXL 3.1 w/o Switch Architecture

Class: 5 days

Virtual Classroom: 10 half-days

ELEARNING: Show Me

CXL 3.1 Update

Class: 2 days

Virtual Classroom: 4 half-days

ELEARNING: Show Me

CXL 2.0 w/ Switch Architecture

Class: 5 days

Virtual Classroom: 10 half-days

ELEARNING: Show Me

CXL 2.0 w/o Switch Architecture

Class: 4 days

Virtual Classroom: 8 half-days

ELEARNING: Show Me

CXL Fundamentals

Class: 1 day

Virtual Classroom: 2 half-days

ELEARNING: Show Me

Related Items

Live Training Courses