[ad_1]
//php echo do_shortcode(‘[responsivevoice_button voice=”US English Male” buttontext=”Listen to Post”]’) ?>
Despite many new and novel ASIC designs on the market today, GPUs are still extremely popular for both data centers and edge applications like robotics.
That was how Nitin Dahad, editor-in-chief of embedded.com and an EE Times correspondent, opened a panel discussion on whether any novel architecture can replace the GPU. The conversation was part of EE Times’ most recent AI Everywhere Forum.
“Are new types of chips making any progress, and in which markets?” Dahad asked the panel’s speakers. “Which types of chip architectures are showing the most promise? And how do we design new chips to tackle an ever-evolving workload like AI?”
One expert who weighed in on the questions Dahad asked sees an opening for AI hardware vendors who have innovative ideas.
“It’s hard to not speak about machine learning [ML] and speak about the elephant in the room that is Nvidia,” said Ian Cutress, chief analyst for More Than Moore. “The number that I always get quoted is something like 90% of the training market is currently hosted by Nvidia. But when I speak training, there’s obviously the whole world of inference that sometimes we forget about.”
Nvidia has inference accelerators, the T4 Tensor Core GPU and the A10 Tensor Core GPU, and they do very well in data center applications, Cutress acknowledged. However, there are many more real-world needs for inference, and Nvidia has no plans to meet them, he said.
“The devices that we hold in our hands, the devices on the edge and even going in to solve the data center market, there’s a lot more malleability there for these new AI hardware vendors to play in, to take advantage of, to find cost-effective solutions—and optimize solutions with customers,” he said. “That’s where I see the biggest opportunity to kind of battle the Nvidia juggernaut.”
Data centers need GPUs, but edge devices are where opportunities lie for new architecture, Kinara CTO Rehan Hameed said.
“I think data center GPUs are still harder to displace today,” he added. “I think just because of the massive software ecosystem that exists, and pretty much every new model that needs to be developed makes use of that extensive CUDA library that exists there, especially on the training side.”
One of the advantages of GPUs is that ML models have been trained on them, so the ML pretty much works out of the box on the inference side on a GPU as well, Hameed said. There isn’t any porting required.
“I think that is the dominant reason why it’s so hard to displace GPUs so far in data centers,” he said. “But I agree that edge is a completely different story.”
Hameed asserted there are many markets where the cost and power profile of a GPU doesn’t work, including in cameras and retail checkouts. In robotics, GPUs can be used in prototyping but are not a solution for mass deployment, he said.
“What we have seen is that for most practical deployments, a dedicated AI accelerator along with an embedded SoC [system-on-chip] is today the best solution for AI deployments on the edge,” Hameed said.
There are alternatives to GPUs, said Bill Jenkins, director of product marketing for the AI, military and financial industries at Achronix.
“As we look at large workloads where we need more efficient compute, FPGAs have been playing in that space for a really long time,” he said. “We don’t get the same traction because they’re a lot harder to program. So people have to do a little more work. I’ve also seen a great deal of traction from graph processors.”
However, Jenkins said, the question remains: Are graph processors suitable for the types of workloads that people want to productize?
“I’ll go back to one of the biggest problems,” he said. “Not many people really know what they want to do. There are just so many ways and so many things that they could implement. You know, I look at the GPU, the CPU and even the FPGA as that flexible architecture that can handle everything. And then the question is: Does it need to do something really well, and is there an alternative dedicated piece of hardware for that something?”
What are customers saying?
Dahad pointed out that there is a lot of expertise required on the customer side in using the hardware and software for AI. He asked the panelists what they are asking for from the industry.
“I would say the No. 1 thing is, ‘I’ve got a model. How do I implement that on your architecture?’” Jenkins said. “And then they’re going to compare that performance against where they are today. So if somebody can provide [a product that is] going to be lower-latency, lower-power, higher-performance and turnkey … they’ll take it and tweak it over time.”
[ad_2]
Source link
