Key Takeaways from Hot Chips 2024

Earlier this week, our Head of Research, Xianxin Guo, attended Hot Chips 2024. As one of the most important events in the semiconductor industry, it was the perfect place to discuss innovation in AI accelerators and compute architectures.

Here are some of Xianxin’s key takeaways from the event: 

1. The market focus has shifted from AI training to inference. Hot Chips 2024 was all about inference performance.
2. Heat and energy are holding back major silicon advancements in AI.
3. New architectures outperform GPUs: substantial performance gains can only come from new hardware.
4. Chips continue to struggle to get the additional performance needed – lots of workarounds are being applied to try and facilitate improvements.   
5. Optical interconnect is coming of age with big hitters like Broadcom and Intel presenting their optical interconnect solutions. Optical interconnect integrated with optical computing will maximise overall compute performance.

Companies are trying many different ways to solve today’s compute problem. Startups are trying new architectures; companies of all sizes are working on optical interconnect; there is even work going to develop a proprietary communication protocol to replace the TCP/IP protocol. Equally, there is a lot of work underway exploring and deploying low-precision AI. 

Optical computing is set to be part of this innovation, with Lumai enabling AI deployment at scale with the fastest, most energy efficient AI processor. 

Get in touch to learn more.