Log In
In the Convening Leaders 2024 competition, all but one team used Gevme’s Spark AI, a gen AI tool developed in collaboration with PCMA, to come up with a marketing plan for an event. (Whatever Media Group)
A competition in the Tech Playground at Convening Leaders 2024 in January pitted event-planner participants against others using generative AI. In that battle to create the most innovative marketing campaign for an upcoming event, dubbed “Human Heroes vs. AI Wizards,” judges ruled the all-human team outperformed the teams using AI.
A win for human creativity, no? After all, conventional wisdom tells us that the best use of AI is to perform rote tasks, rather than to generate ideas. A recent study, however, turns that notion on its head.
In a working paper — “Ideas Are Dimes a Dozen: Large Language Models for Idea Generation in Innovation” — researchers at The Wharton School and Cornell University shared the results of their study exploring whether ChatGPT-4 is better at ideation than humans. The study compared 200 ideas — for a new retail product targeted at U.S. college students for under $50 — produced by business school students alone against those generated by AI.
According to the study, ChatGPT-4 can generate 200 ideas by interacting with one human in about 15 minutes. A human working alone can generate about five ideas in 15 minutes. “Humans working in groups do even worse,” the paper’s researchers said. “In short, the productivity race between humans and ChatGPT is not even close.”
But quantity isn’t the same as quality. So how did the AI tool perform on that front?
“In most innovation settings, we’d rather have 10 great ideas and 90 terrible ideas than 100 ideas of average quality,” wrote the researchers, who examined ideation productivity and quality separately and found that ChatGPT also came out on top when evaluated for the latter (which analyzes, among other factors, average purchase probability). “Combined, the effects of much higher productivity and the higher quality of the best ideas will likely completely trounce human ideators,” they said.
RELATED: Who Organized the Event Better — Humans or AI?
The researchers noted that prior to 2022, it was generally understood that creative work would remain the domain of humans with AI in the workplace. “In some ways, the opposite is true of LLMs,” they wrote. That is partly due to LLM’s lack of judgment when ideating, which, at least in this setting, contributed to extreme productivity and high variance in quality, resulting in “creativity greater than that of the average human.”
Wharton professor Christian Terwiesch, one of the authors of the white paper about the study, told Knowledge at Wharton that there are three things the study teaches us — the first is a “no-brainer.” Everybody, he said, should be using ChatGPT to help them come up with ideas. “It’s cheap. It’s fast. It’s good. What’s not to be liked? Worst case is you reject all of the ideas and run with your own. But our research speaks strongly to the fact that your idea pool will get better.”
A second takeaway, according to Terwiesch, is that AI can help us move faster “through the bottleneck of innovation, from creating ideas to evaluating them.” Evaluation is far more challenging and more critical for success, he said.
And lastly, “I think we should not boil it down to man versus machine,” he told Knowledge at Wharton. “We need to find ways in which the AI becomes your creative co-pilot. Together, you can become a more innovative team.”
Michelle Russell is editor in chief of Convene.