Published: Wed, 10 Sep 2025 03:45:00 +1000 Duration: 1:00:45

Description: While most conversations about generative AI focus on chatbots, Thomas Wiecki (PyMC Labs, PyMC) has been building systems that help companies make actual business decisions. In this episode, he shares how Bayesian modeling and synthetic consumers can be combined with LLMs to simulate customer reactions, guide marketing spend, and support strategy. Drawing from his work with Colgate and others, Thomas explains how to scale survey methods with AI, where agents fit into analytics workflows, and what it takes to make these systems reliable. We talk through: Using LLMs as “synthetic consumers” to simulate surveys and test product ideas How Bayesian modeling and causal graphs enable transparent, trustworthy decision-making Building closed-loop systems where AI generates and critiques ideas Guardrails for multi-agent workflows in marketing mix modeling Where generative AI breaks (and how to detect failure modes) The balance between useful models and “correct” models If you’ve ever wondered how to move from flashy prototypes to AI systems that actually inform business strategy, this episode shows what it takes.

LINKS: The AI MMM Agent, An AI-Powered Shortcut to Bayesian Marketing Mix Insights AI-Powered Decision Making Under Uncertainty Workshop w/ Allen Downey & Chris Fonnesbeck (PyMC Labs)

Upcoming Events on Luma 🎓 Learn more: Hugo's course: Building LLM Applications for Data Scientists and Software Engineers — https://maven.com/s/course/d56067f338

  1. Thomas Wiecki: "They're trained on the entire human text data. And in a way, you know us better than we know ourselves, right? Because they have really consumed like everything we've ever produced."
  2. Thomas Wiecki: "Bayesian modeling, where you embrace subjectivity, iteratively improving the model very closely with the data, the focus is more on learning and getting a useful model rather than a correct model."
  3. Thomas Wiecki: "And also, I mean, intelligence is intelligence. So whether that's artificial or human, I don't think it's I mean, there are definitely differences, but I don't think it's worth it to make categorical difference."
  4. Thomas Wiecki: "one of the first things it could do was, like, write beautiful poetry and be just very, very well spoken and really interpret the nuances of what people are saying, but couldn't count how many Rs on strawberries."
  5. Hugo Bowne-Anderson: "it is like very, like, Silicon Valley engagement bait y stuff in in in some ways."

Full Transcript

Thomas Wiecki (00:00 - 00:40): So, the question there was, can we instead ask synthetic consumers, namely AI systems, and get the same answers? And turns out we can. And that's fascinating research, and I'm happy to dig more into that. But this is really just another way, right, to allow businesses make more informed decisions. So it includes sophisticated simulations, you could say, right?

And these simulations could come from causal Bayesian models that model the data generative process and can generate new data, they can make predictions, they can provide insights, they can provide counterfactuals and what if scenarios, but also includes more realistic simulations where we have an LLM act as a human and respond as a human.

Hugo Bowne-Anderson (00:43 - 02:16): That was Thomas Wiecki, founder of PyMC Labs and coauthor of PyMC, talking about how his team worked with Colgate using AI to simulate customer reactions to new toothpaste ideas, including mango flavored or even glow in the dark toothpaste, and how closely these synthetic surveys matched what real consumers said, even across different demographics. In this episode, Thomas and I get into how companies can use generative AI not just for chatbots, but to test product ideas, guide marketing spend, and actually support business decisions. We also talk about where these systems break, what it takes to make them reliable, and how Bayesian modeling fits into the picture. This was originally recorded as a livestream, and I'll link to the full video in the show notes. I'm Hugo Bowne-Anderson, and welcome to Vanishing Gradients.

Hey there, Thomas, and welcome to the show.

Thomas Wiecki (02:16 - 02:17): Thanks so much for having me.

Hugo Bowne-Anderson (02:18 - 03:36): Such a pleasure to be here with you and take, you know, one of our many conversations that I've always loved public. Welcome to everyone who's watching the livestream. Great to have you all here on YouTube. As I've written in the description and in the chat, our Q and A will happen on Discord and I've put that link in the description and in the chat, so please join us there and ask questions. And we do it on Discord so that we can have threads and we can respond later and we can continue the conversation.

So I'm really excited to be here today with Thomas to do a livestream of the Vanishing Gradients podcast, which originally was a data science podcast, then it became a machine learning podcast. Now we've rebranded as AI, of course, But it's data and data powered software and decision making all the way down. Thomas is the author of PYMC, the leading platforms for statistical data science. He has a PhD in computational cognitive neuroscience from Brown. Former VP of data science and head of research at Quantopian where he built a team of data scientists to build a hedge fund from a pool of 300 ks crowd researchers.

And I know Thomas through the PI Data ecosystem as well and Thomas has spoken at open data science, a lot of different PI Data conferences, at the Strata Conference, and I've known Thomas on and off for the past decade. So it's so much fun to be here today with you, Thomas. I'm wondering if there's anything I missed out in my introduction.

Thomas Wiecki (03:36 - 03:59): I think that was perfect. And yeah, I'm also very excited to be here. I have very fond memories of when we just got on a random call and we're like, oh, where are you right now? And you're like, I'm in New York. I'm like, I'm also in New York.