It seems that generative AI is everywhere these days. From AI-generated news articles to photograph-like images used as headshots, it’s becoming quite pervasive. It should therefore be of little surprise that companies like Amazon are getting into the mix. The multi-billion-dollar company recently announced it would soon be introducing its own AI shopping assistant for its customers. Named Rufus, Amazon’s AI assistant will not only help customers find specific products. But it will also guide them to items that best meet their individualized needs. Trained on a trove of data that includes personalized shopping information, this could be a game-changer. Amazon certainly thinks so, but trusting AI assistants to guide well might be a challenge.
(Generative AI and copyright law are clashing–read up on the conflict in this Bold story.)
Over the past few years, Amazon’s intentions when it comes to customers’ well-being has come into question. The company has been accused of swaying its site visitors toward those products that benefits Amazon the most. Whether it is products providing Amazon the best profit margin or those generating the most ad dollars, there’s a question of trusting AI assistants. Thus, it’s worth considering how Amazon’s AI assistant might further such practices should Amazon be so inclined. When it comes down to it, the recommendations that Rufus makes depends on its AI training. And to date, Amazon has provided limited information about the algorithms under which Rufus will operate.
What We Know About Rufus
According to the company, Amazon’s AI assistant will serve mainly as an adjunct to site visitors. As a conversational shopping assistant, users will be able to communicate their wants and needs. In turn, Rufus will guide them to the highest quality and best priced items to satisfy their needs. Sounds pretty simple, right? Greater customer engagement, increased shopping conversions, higher customer satisfaction, and of course, higher revenues. Assuming Rufus has consumers’ best interest in mind, then Amazon’s AI assistant offers enhanced value. But trusting AI assistants, including some related to Amazon’s product and book reviews, have been problematic. This will be something the company will need to address to win over customer use.
As with any generative AI tool or platform, the question of AI training is raised. This is raising new questions about copyright infringements. Companies like Open AI and others have been fairly tight-lipped about training algorithms for AI tools. Amazon is no different in this regard as it too has not revealed details about Amazon’s AI assistant training protocols. But it has offered some insights related to Rufus’ training. Reportedly, it has been trained on Amazon’s product catalog, reviews, web information, as well as questions and answer sections. Using such data, it becomes possible to see how Amazon’s AI assistant could make good recommendations to users. But this may not be the only data utilized in Rufus’ algorithms. And this is why trusting AI assistants like Rufus should be performed with caution. This especially true give the volumes of data Amazon has about individual consumer purchases and preferences.
Amazon’s Troubled Past with AI
Part of the problem in trusting AI assistants for Amazon relates to its past history. In recent months, the company’s bestseller lists have been contaminated with fake book reviews and recommendations. At the same time, well-known authors with a publishing following have reported AI-generated books posing as imposters on Amazon. Understanding this, there are questions as to how well Amazon can oversee AI-related content. And if this is an issue, then how adept is the company at overseeing Amazon’s AI assistant? Notably, the fake reviews and false identities linked to Amazon’s books aren’t under their immediate control. But it still raises issues about the company’s level of commitment when it comes to AI-related integrity. Greater transparency and oversight would be a welcomed change in this regard.
(AI-generated books are corrupting the Amazon best seller list! Read all about it in this Bold story!)
These are not the only concerns that make trusting AI assistants at Amazon challenging. In addition to these issues, Amazon has been accused more than once of biasing recommendations to consumers. Products labeled as being “Amazon’s Choice” presumably are those with the best consumer reviews, product quality, and value. But some have found that Amazon operates a “pay-to-play” game with advertisers. Those providing higher ad revenues to the company receive preferential consideration for the “Choice” status. At the same time, Amazon has also been accused of doing the same for its own products. This has prompted an antitrust lawsuit by the Federal Trade Commission in 2023 that is still ongoing. If these self-motivating practices are true, trusting AI assistants at Amazon will pose similar threats.
Proceeding with Caution
There’s no question that Amazon values its ad revenues. In the last quarter of 2023, the company saw a 27% increase in ad revenues reaching a total of $14.7 billion. At the same time, however, Amazon also needs to maintain customer loyalty and confidence. Thus, trusting AI assistants that the company offers will require an ethical approach to using generative AI. But simply because the company starts out that way doesn’t mean it will continue down the same path. At some point, Amazon will be attracted to the dollars advertisers are willing to spend for sponsored Rufus content. As a result, don’t be surprised if Amazon begins strong but strays as consumers become accustomed to having Rufus around.
There is data to support a strong level of caution when it comes to Amazon’s Ai assistant. Past research has demonstrated that sponsored listing on Amazon’s sites far exceed other retailers. For example, the number of sponsored products on Amazon doubles those found at Walmart. They exceed those found at Target’s site four-fold. This suggests that Amazon uses its listings to attract higher volumes of advertisers. Of course, this may remain limited to routine search for product listings. But in all likelihood, Rufus will eventually take over this role for individual consumers. While this level of personalized AI assistance is attractive, Rufus might not be steering customers in the best direction. Just as it’s difficult now to determine validity of product reviews at Amazon, so will be trusting AI assistants they offer.
