# Neural Scaling Laws (Lecture 2) - Institution: CERN - Speaker: Albert Aillet - Year: 2026 - Language: English - Source: [CERN](https://videos.cern.ch/record/3023585) - Licence: [CERN](https://copyright.web.cern) - Who can watch: Teens profiles and older (sources give no ratings; placed by Invisible Network from the source's own words) See attached PDF for full citations and bibliography. As the historic race to build AI infrastructure pushes projected hyperscaler spending near 7 trillion dollars by 2030, one of the primary justifications for such staggering capital deployment is the predictability offered by neural scaling laws. This presentation examines the scale-invariant nature of deep learning, defined as a power law of the form $ y= ax^b $, where scaling inputs yields predictable gains across orders of magnitude. The presentation first gives an overview of the field's trajectory by listing seminal work. A central focus is the resolution of the debate between Kaplan et al. and Hoffmann et al. regarding the optimal allocation of compute. We continue by challenging the Chinchilla paradigm through the lens of inference economics, showing that for widely deployed models, the optimal frontier shifts toward smaller, "over-trained" architectures to minimize inference-time costs. Furthermore, the presentation opens a discussion on the sociopolitical implications of these scaling laws. As the capital requirements for large model training skyrocket, we mention the risks of power centralization within a few major tech companies. In response, we explore emerging research on decentralized model training, investigating how scaling laws adapt to distributed compute environments and whether decentralized infrastructure can offer a counterweight to the current centralization of intelligence. The presentation will include plots and equations from the aforementioned academic work and follow a structure similar to the presentations of Pearce and Hashimoto. The presentation is mainly designed as a lecture. It can also include a practical exercise involving fitting scaling law curves to provided empirical data of large model training, depending on availability. 00:00:00 Slide 1 00:03:21 Slide 2 00:03:49 Slide 3 00:04:24 Slide 4 00:04:52 Slide 5 00:06:09 Slide 6 00:08:46 Slide 7 00:11:36 Slide 8… ## Lectures 1. [Neural Scaling Laws (Lecture 2)](https://invisiblenetwork.ai/learn/cern/3023585/5b8079b0b543), 59 min (CERN's player)