They're gonna need a heck of a lot of memory bandwidth – not to mention capacity – to do it
Within a few years, AMD expects to have notebook chips capable of running 30-billion-parameter large-language models locally at a speedy 100 tokens per second.
In this regard, LLM performance on Strix Point is limited in large part by its 128-bit memory bus – which, when paired with LPDDR5x, is good for somewhere in the neighborhood of 120-135 GBps of bandwidth depending on how fast your memory is. When it comes to demonstrating the value of AI PCs, AMD is leaning heavily on its software partners. With products like Strix Point, that largely means Microsoft."When Strix initially started, what we had was this deep collaboration with Microsoft that really drove, to some extent, our bounding box," he recalled.
The good news is there are quite a few ways to do just that – depending on whether you're trying to prioritize memory bandwidth or capacity.One potential approach is to use a mixture of experts model along the lines of Mistral AI's Mixtral. These MoEs are essentially a bundle of smaller models that work in conjunction with one another.
Singapore Latest News, Singapore Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
The Sunday Times 100: Three Leeds businesses named among top 100 fastest growing in UKThree Leeds businesses have been named among the top 100 fastest growing in the UK.
Read more »
Bench Across Britain: After 100 years of dominance, will Labour retain its grasp on Wales?Denbighshire in North Wales is the latest stop in Sky's Bench Across Britain series. While Labour look set to win more seats than in 2019, on Pontcysyllte Aqueduct there remain plenty of floating voters.
Read more »
'I've got 100 questions' about lying Northants police chiefNick Adderley's replacement as chief constable said trust in the police was lower than ever.
Read more »
Sunday Times: top 100 fastest growing companies list revealed - including Jeremy Clarkson's Hawkstone brewery🚀 From digital banking pioneers to healthcare innovators, these are Britain's fastest-growing companies! 🌟
Read more »
Shoppers say £12 hair serum gives 'outstanding results' that last for 100 hoursAussie has launched a new hair serum, the SOS Super Serum Leave In Treatment, which claims to give 100 hours of hydration – and shoppers are already raving about it
Read more »
Incredible video shows football pitch COLLAPSING as 100-foot sinkhole swallows middle of the fieldA shocking video has shown a USA football pitch collapsing as a massive sinkhole destroys the middle of the field.
Read more »