Tag: Scaling

2 articles

• LLM μP Hyperparameter Cross-Scale Transfer in Production: Reducing Large Model Tuning Costs with Base Shape Fingerprints, Coord Checks, and Proxy Sweeps

  A systematic breakdown of μP hyperparameter cross-scale transfer principles and engineering boundaries, covering how to solidify Base Shapes, run Coord Checks, organize proxy model sweeps with regression gates, and avoid misapplying width transfer to training duration, batch size, or architectural changes.

  07/28/2026 LLM training / μP / hyperparameter transfer / pretraining engineering / scaling

• LLM Serving Autoscaling in Production: Safe Scaling with Queue Depth, P95 Latency, and GPU Utilization

  A deep dive into production-grade autoscaling for LLM serving. Build a multi-layer control loop using queue depth, P95 latency, token throughput, and GPU utilization, with warm pools, admission control, cost boundaries, and cooldown strategies to meet inference SLOs under cost constraints.

  07/09/2026 LLM Serving / Autoscaling / GPU Inference / P95 Latency / SLO / Scaling