The whole world seems to be talking about AI. It writes for us, draws for us, and now we have entered the age of “agentic AI,” which can even operate our computers on our behalf. But honestly, can you feel it yet? I mean how responsive the smartphones and laptops we use every day really are when running AI.
That is because so much of this technology still depends on enormous cloud servers somewhere far away, rather than on our own devices. Recently, though, the processors inside our computers and smartphones have begun moving in an entirely different direction.
Apple’s approach is especially interesting. Its recently released M5 and A19 Pro chips can deliver more than four times the AI processing power of the previous generation. I tested them myself not long ago and came away very impressed. So what exactly has Apple done inside these chips?
On June 3, I had the chance to have an in-depth conversation with Doug Brooks, Apple’s senior product manager for Mac. A former systems engineer who has been involved with Apple silicon from its beginnings, he shared his perspective on where Apple’s AI chips are headed.
Neural Accelerators: An Extra Engine Inside the GPU
If you are interested in semiconductors, you probably know how much fun it is to study photographs of Apple’s chips, or “die shots.” But the M5 die shot presents a curious puzzle. The Neural Engine, a separate area dedicated to AI, is still right there. So why has Apple added another new unit, the Neural Accelerator, inside the GPU cores that handle graphics? And why put it there, specifically?
Brooks explained that the underlying design was about improving AI performance by increasing the GPU’s matrix-processing performance. At its core, AI is a collection of mathematical operations that demand enormous computing power. A Neural Accelerator sits inside each GPU core and focuses specifically on the large-scale matrix operations at the heart of AI. That, Brooks said, is how Apple achieved more than four times the AI performance of the previous generation.
Large language models (LLMs) and image-generating diffusion models have a distinctive characteristic in how they work. Starting from noise that looks like a blurry field of gravel, they must multiply entire tables of data, or matrices, over and over, billions of times per second, to arrive at a clear answer. For a chip, it is a relentless succession of repetitive calculations.
GPUs have always been good at tackling this problem because they have so many cores, but matrix operations themselves are not their original main job. So Apple embedded a dedicated accelerator inside each GPU core: the Neural Accelerator, built to race through matrix operations at lightning speed.
It is Apple’s familiar approach to chip design, once seen in the dedicated Media Engine it built into its chips for video editors, carried over directly into AI.
Apple Silicon’s Strength: The Flexibility to Take Any Shape
This is where Apple silicon’s formidable strengths come into play: flexibility and scalability. Conventional chip companies have to meet shared industry standards and price targets, so they cannot easily change a chip’s shape or design. Apple, however, looks first at the finished hardware product and designs its chips freely around that. It can make a chip as large as it needs to be or add pins to increase bandwidth, without anyone objecting. Even an unusual design that joins two chips together presents no obstacle to completing the product. As long as it delivers twice the performance, there is no problem.
This time, too, Apple chose to make each individual GPU core smarter rather than simply making the chip larger. As a result, across the M5 lineup from the base model to Max, AI matrix-processing capacity scales directly, and at a formidable pace, as the GPU core count rises.
Brooks ultimately brought the conversation back to the importance of the app ecosystem. Hardware matters, he explained, but developers also need to be able to build software around these chips. That is why the entire product family, from iPhone to iPad and Mac, shares the same architecture and frameworks such as Core ML and Metal. The bigger picture is to let developers build once and deliver powerful AI apps across every device.
In fact, this same Neural Accelerator technology is found not only in MacBooks but also in the A19 Pro chips inside the latest iPhone 17 Pro and iPhone Air in your pocket. It is a distinctive feature of Apple’s ecosystem, made possible by integrating hardware, operating systems, and software into one whole.
CPU, GPU, Neural Engine… Orchestrating the Best AI Experience
What I most wanted to understand was how the work is divided up. If a new accelerator has been added to the GPU, does that mean the existing Neural Engine is no longer needed? I had to ask what distinguishes the Neural Engine from the Neural Accelerator.
He told me that the Neural Engine remains a powerful dedicated engine, with thousands of models already running on it all the time. Meanwhile, the demanding workloads emerging from the open-source community, large language models, and the MLX framework require the GPU. Apple silicon’s AI strategy, he explained, is to have the Neural Engine handle light everyday tasks efficiently, while the GPU’s Neural Accelerators take on compute-intensive generative workloads.
After hearing his answer, the pieces seemed to fall into place. At heart, the Neural Engine is about conserving battery power. Once a CPU or GPU starts working, it consumes a lot of power. Rather than waking them up, the Neural Engine can keep running in the background with minimal power, summarizing text or cutting subjects out of photos in a quiet corner of your phone. Through Core ML, it also provides the foundation that lets developers use AI capabilities without having to build or train their own models.
By contrast, developers who bring their own open-source AI models to the Mac these days mainly use the GPU. Apple responded with the open-source MLX library, creating a setup in which the moment a developer runs MLX code, the Neural Accelerator boosters tucked inside the M5 GPU automatically wake up and speed up the calculations.
By default, when an AI request comes in, macOS or iOS determines whether it is suited to the Neural Engine or is a heavy workload that needs the GPU’s accelerators, and assigns it accordingly. But app developers can also send work directly to the GPU if they need to.
“Will Users of Older M1 and M4 Devices Be Left Behind?”
But if the M5 is this smart and fast, will people using older M1, M2, or M4 Macs, or older iPhones, be left out of new AI features in the future? Honestly, this was my biggest concern as I ran performance tests. Brooks told me there was no need to worry.
Every Apple silicon Mac, regardless of generation, is ready to run Apple Intelligence, he explained. The difference between the M5 and earlier generations is not one of “capability,” meaning whether something can or cannot be done. It is a difference in “performance”: how fast and smooth the experience is.
Earlier chips can also run local AI models perfectly well thanks to their unified memory architecture. The M5 simply adds another specialized engine, dramatically speeding up the time it takes for the first word to appear or an image to finish generating. For now, that means there is no fragmentation that draws a line between which features a device can and cannot use.
For enormous AI workloads that exceed a device’s capabilities, Apple also combines on-device processing with the power of Private Cloud Compute (PCC), its own server infrastructure that guarantees strong privacy protection. With this hybrid infrastructure spanning devices and the cloud, users of older devices can enjoy a consistent AI experience without being left out.
The Triumph of Chips Guided by the “Job to Be Done”
In the past, the semiconductor race was a fairly one-dimensional contest: how tightly could you pack transistors using smaller manufacturing processes, and how much speed could you squeeze out of them? Today, the paradigm has shifted. The competition is about how intelligently you arrange dedicated blocks inside a chip to perform the specific jobs consumers want.
Rather than buying the same general-purpose chips as everyone else, Apple designs its silicon backward from the “job to be done”: “What experience should this device ultimately deliver to the consumer?” Dynamic Caching, the Media Engine, and now the M5’s Neural Accelerators… Built around this philosophy, Apple silicon has become the Apple ecosystem’s most powerful asset, one that rivals cannot easily challenge.
How will the new Apple Intelligence and foundation models unveiled at WWDC change our everyday lives, with such powerful hardware behind them? It is one more reason I am looking forward to the keynote.
