Qualcomm Says Hybrid AI Is Becoming Crucial: Here's Why

Qualcomm asserts that the hybrid AI approach is applicable to virtually all generative AI applications and device segments.

Qualcomm, the renowned chip maker, has emphasised the importance of hybrid artificial intelligence (AI) architecture in the face of the rising popularity of generative AI and chatbots such as Microsoft's ChatGPT and Google's Bard. According to Qualcomm's white paper, hybrid processing is becoming increasingly crucial as it enables generative AI developers and providers to leverage the computing capabilities of edge devices, resulting in cost reduction.

What Is Hybrid AI?

In a hybrid AI architecture, AI workloads are distributed and coordinated between cloud and edge devices, rather than relying solely on cloud processing.

This collaboration between the cloud and edge devices, including smartphones, cars, personal computers, and Internet of Things (IoT) devices, enables the delivery of more powerful, efficient, and optimised AI solutions.

One of the primary motivations behind hybrid AI is cost savings. For example, the cost per query for generative AI-based search is estimated to be ten times higher than traditional search methods. Qualcomm emphasises that this cost difference is just one illustration among many generative AI applications. With hybrid AI, devices and the cloud can simultaneously run models, with devices handling light versions of the model while the cloud processes multiple tokens of the full model in parallel. The cloud can also correct device-generated answers if necessary.

Remarkably, AI models with over 1 billion parameters are already operational on smartphones, exhibiting performance and accuracy levels comparable to those achieved in the cloud. Furthermore, models with 10 billion or more parameters are expected to run on devices in the near future.

Qualcomm asserts that the hybrid AI approach is applicable to virtually all generative AI applications and device segments, including phones, laptops, extended reality headsets, cars, and IoT devices.

The company's white paper highlights the potential of hybrid AI architecture to harness the strengths of both edge devices and the cloud, enabling cost-effective and efficient deployment of generative AI solutions across various industries and use cases. As generative AI continues to evolve and gain traction, Qualcomm advocates for the adoption of hybrid AI as a key strategy to optimise performance and overcome the cost challenges associated with cloud-centric AI processing.