Thursday, November 16th
Today marked the last day of SC23! We wanted to thank the thousands of folks who stopped by our booth throughout the week.
The real life Groq llama we brought to the show drew a lot of intrigue (and we realized we should have had “Bunny” come the third day as folks missed seeing her!). When people arrived at our booth, the first question was usually, “So, what’s an LPU™ Inference Engine?” Folks were also curious about storage, on premise versus cloud deployment optionality, and the research community was visibly hungry for compute.
There was also a lot of conversation about quantizing an LLM to FP8 or even INT4. Would this impact quality to the point that it should or shouldn’t be done? The debate was about the tradeoff between speed and quality however when folks learned we were running Llama 2 70B in FP16 with no quantization, while being able to achieve our level of speed (over 300 tokens per second per user), they were very surprised. It continued to get us that “Wow!” that we build for.
Over the course of the last four days our team had the chance to deliver hundreds of demos of our LPU™ Inference Engine, host over 25 customer meetings and eight in-booth sessions, and present at four talks:
- Programming Novel AI Accelerators for Scientific Computing with Sanjif Shanmugavelu
- Exploring Converged HPC and AI on the Groq AI Inference Accelerator with Tobias Becker
- Strong Scaling of State-of-the-Art LLM Inference with Groq Software-scheduled Deterministic Networks with Igor Arsovski
- From Stencils To Tensors: Running 3D Finite Difference Seismic Imaging on the Groq AI Inference Accelerator with Tobias Becker
Beyond the technical conversations at our booth, we were also giving away free Groq swag and actually ran out! We passed out water bottles, beanies, touchscreen-friendly gloves, LLM-themed sunglasses (just like the ones from our press announcement graphics), and for elite swag bags, blankets and Groq orange ceramic camp cups! We saw fellow exhibitors giving out swag as well, and one of the most popular giveaways were the Star Wars legos! It was awesome to see that inner child come out in the attendees when they received such a nostalgic gift (and we’re taking notes for next year!).
With that, we wanted to say another huge thank you to all the new faces we met, to all the old friends who stopped by to say hello, and to the incredible team of Groqsters, both in attedance and working geoagnostically, who made our presence at SC23 possible. We’re already looking forward to next year and can’t wait to see again you then!