We tested AI shopping chatbots. This is what we found.

We’ve all found ourselves scouring malls and websites for the perfect last-minute present.

So as more retailers roll out OpenAI-powered shopping tools, we wanted to see how well chatbots could recommend gifts.

Minneapolis-based Target, Walmart and others are betting big on generative AI, launching platforms in ChatGPT, developing in-house chatbots and investing in thousands of GenAI licenses. McKinsey estimates the technology could give the industry a $310 billion boost.

But we also wanted to address lingering privacy concerns by testing how much personal information shoppers would have to share to get quality recommendations.

“Chatbots tend to solicit a lot more information from users than search engines — your health, your finance, all sorts of things,” said Dr. De Liu, professor of information and decision sciences at the University of Minnesota’s Carlson School.

“Now, imagine you’re also telling them your shopping habits, your budgets. Once they know that about you, the recommendations get more personalized, and that’s exactly what makes it harder to leave them.”

For our experiment, we asked participating reporters for these five prompts:

The suggestions were OK, if unsurprising. The chatbot largely echoed what reporters had already said, which makes sense because these systems rely on pattern recognition and predictive text.

For example, when someone said they enjoy experiential gift cards like orchestra tickets, the recommendations echoed that preference.

Another said they would like high-quality year-round clothes, such as Patagonia vests or other gear that aligned with their various hobbies, which included biking, running and fly-fishing. The chatbot’s output was a fly fishing vest, running vest and a guide for the “50 best fly-fishing gifts.”

One surprise was an idea to pay someone’s race entry fee — a suggestion that seemed to spark a little more creativity than the others.

But the underlying feeling from reporters was that the ideas didn’t involve enough personalization, while others were flat-out wrong.

“If you use a generic chatbot, they typically don’t go out to search for these products, other than maybe some web articles or something on the internet,” he said. “But they aren’t specifically designed to go search the catalogs and give step-by-step recommendations.”

OpenAI recently launched a shopping tool trained to compare deals, find similar products and surface new items, potentially generating more accurate responses.

Because it could require more information, I tested it using my own interests:

The chatbot asked follow-up questions about budget and book genre, then offered the option to approve or reject several ideas before taking a few minutes to generate a curated list.

Each item was grouped into its own section that included where to buy it and for what price, why it might be a good fit, trade-offs and who it was best for. The chatbot also gave a short explanation of how to choose among the options.

“The best way to think about this kind of chatbot is to think of it as being a personal shopping assistant,” Liu said. “You tell them what you want, your priorities, maybe your budget, and then they will start doing research based on the criteria laid out.”

After several retailers announced partnerships with OpenAI to enable shopping through ChatGPT, questions quickly followed about how the platforms would work and whether generative AI would be a new way of shopping.

There are also concerns about personal data storage and the environmental impact of data centers. Still, surveys by PwC and Deloitte show a growing number of shoppers are using generative AI, especially when comparing products.

Target’s in-app platform with ChatGPT remains in beta and should be available to all users, a spokesman said. The company has seen most people use the tool for finding deals — something Target executives have said shoppers continue to prioritize amid economic uncertainty.

Walmart’s recently launched “Instant Checkout” integration lets shoppers buy individual items directly in the chatbot. It also announced a partnership with Pinterest that allows users to tap a pinned recipe and add ingredients directly to their cart.

Target and Walmart have both debuted their own in-house AI shopping tools. Walmart introduced Sparky in June, promoting it as an AI assistant. Target launched the second iteration of its gift finder in November, which is still labeled as beta and will be phased out after the holidays, a spokesman said.

After prompting both for gift recommendations for a “23-year-old who enjoys reading and making home-cooked meals,” Target’s gift finder said it was unable to help with books. A spokesman confirmed the chatbot can’t respond to all categories, including books.

Walmart’s chatbot provided a detailed list of ideas, including individual recommendations for cookbooks, recipe journals and kitchen gadgets, and a link to a page with hundreds more.

Liu said that AI agents will become more common, much like search engines, but there will likely always be two types of shoppers: those who want items as fast as possible, such as for routine grocery shopping, and those who still want the traditional discovery experience.