RTX 5090 vs M5 Ultra: Which AI Powerhouse Wins?

Summary

This video compares the Apple M5 Ultra Mac Studio and the NVIDIA RTX 5090 for running large language models (LLMs) locally. The M5 Ultra offers significantly higher unified memory capacity (up to 256 GB) and memory bandwidth (1.2 TB/s), while the RTX 5090 boasts superior raw compute power (419 TFLOPS) and higher memory bandwidth (1.8 TB/s). Performance benchmarks for Qwen3.8-27B show the RTX 5090 leading in raw token generation speed, especially with Multi-Token Prediction (MTP) enabled, but the M5 Ultra's massive RAM capacity makes it superior for running larger or unquantized models.

Technical details

• Memory Bandwidth 30s

  M5 Ultra: 1.2 TB/s; RTX 5090: 1.8 TB/s.

• Compute Performance (TFLOPS) 60s

  M5 Ultra: ~110 TFLOPS; RTX 5090: ~419 TFLOPS.

• Power Consumption (Sustained Peak Load) 80s

  M5 Ultra: ~214 watts; RTX 5090: 300 to 575 watts.

• Qwen3.8-27B Benchmarks (Prompt Processing) 110s

  RTX 5090: ~3,000 tokens/second; M5 Ultra: ~1.7K tokens/second.

Mentioned resources

• Qwen3.8-27B (Model Repository)
• RTX 5090 llama.cpp runs of Qwen3.8-27B Q4_K_M (Discussion/Benchmark)
• MacStories M5 Ultra review (Review/Benchmark)

Channel & topics

• Hugging Face
• Local LLM Inference
• Hardware Benchmarking
• AI Compute Architecture
• M5 Ultra
• RTX 5090
• Qwen3.8-27B
• RTX 5090 llama.cpp runs of Qwen3.8-27B Q4_K_M
• MacStories M5 Ultra review