This revision is from 2024/06/24 10:05. You can Restore it.

PCIe versions:

• PCIe 3.0: 8 GT/s per lane
• PCIe 4.0: 16 GT/s per lane
• PCIe 5.0: 32 GT/s per lane
• PCIe 6.0: 64 GT/s per lane

SLI (Scalable Link Interface):

• Older technology for linking multiple GPUs
• Limited to 2-4 GPUs
• Lower bandwidth compared to NVLink

NVLink:

• High-bandwidth interconnect for GPU-to-GPU communication
• Much faster than PCIe and SLI
• Supports up to 8 GPUs (depends on GPU model)

Impact on LLM training vs. inference:

Training:

• Requires high bandwidth for data transfer between GPUs
• NVLink is preferable for multi-GPU setups
• Higher PCIe versions beneficial for CPU-GPU data transfer

Inference:

• Generally less demanding on inter-GPU communication
• PCIe is often sufficient, especially for single-GPU setups
• NVLink can still improve performance in multi-GPU inference scenarios