The Shiny New Object Podcast - Predictive Modeling of Conversion Events
An in-depth conversation with Tom Ollerton (Founder of Automated Creative) about marketing analytics and the future of data-driven marketing. Key topics include:
- Learning from 100-year-old marketing books for timeless insights
- Daily data analysis practices for developing marketing instincts
- Predictive modeling of conversion events to optimize advertising algorithms
- Understanding incrementality in marketing attribution
Video
Summary
Learning Philosophy
Reading marketing books from 50-100+ years ago offers better signal-to-noise ratios than modern books that quickly become outdated. Recommended: “Reminiscences of a Stock Market Operator” for insights on purchase psychology and narrative construction.
Daily Data Practice
The core advice for data-driven marketers is developing an intimate “relationship with the data” by examining dashboards daily. This builds instincts for spotting anomalies that others might miss.
Predictive Modeling Approach
Instead of reporting raw conversion values to advertising algorithms, marketers can apply predictive adjustments based on:
- Expected lifetime value
- User behavior patterns
- Business context (e.g., multiplying new customer values by 2x if likely to return)
The Incrementality Challenge
Understanding what conversions truly represent value driven by advertising rather than organic behavior - illustrated by the “pizza flyer” anecdote where a teenager handed flyers to customers already entering the restaurant.
Introduction
Tom: Hello and welcome to the shiny new object podcast. My name is Tom Ollerton, I’m the founder of Automated Creative, and this is a podcast about the future of data-driven marketing. I have the privileged position of being able to interview some amazing people from across the industry about their vision for the future of this industry, and this week is absolutely no different. I’m on a call with Boaz Sobrado, who is director of marketing analytics at Capital.com.
So Boaz, for anyone who doesn’t know who you are and what you do, could you give them a quick background?
Boaz: Hi Tom, it’s absolutely great to be on the podcast. I’ve listened to a good few episodes of your podcast and I’ve really enjoyed it - I’ve learned a lot, so thank you for inviting me on.
In terms of my role and myself as an individual and as a professional, I work for Capital.com and I’m responsible for marketing analytics. The way I usually explain what marketing analytics does is I use that old phrase that was attributed to a retailer over 100 years ago, which is “I know that half of my marketing budget is wasted, I just don’t know which half.”
The way I explain what marketing analytics does is trying to figure out which half of the marketing budget effectively is wasted. And this is a much tougher task than you might think because it’s very not obvious what exactly drives performance and what doesn’t. So that is the sort of thing that me and my team try to help with here at Capital.com.
Learning Sources and Approach
Tom: So are you a marketing book reader, or do you watch videos or conferences? How do you get your knowledge about how to get better at your data-driven marketing job?
Boaz: There’s a wide range of sources. So I do very much enjoy listening to podcasts. I also spend probably more time than I should on Twitter, engaging in the conversations that are there around marketing, around technology, etc. But I am also a very big fan of books, and I’m a fan of books that are actually perhaps slightly different to the ones that most people would read.
Simply because I think the difficulty with marketing books is that particularly digital marketing things move so quickly, so fast, that by the time that someone has the time to write a book and publish it and then someone actually gets to the point of buying it, a lot of what has been written is kind of no longer actual or out of date.
So one thing that I’ve particularly liked about books is looking at books that have been published a long time ago. So you can very easily go to your local library, look up the marketing section, try to look at books that are 50, 60, 70, 80, maybe even 100 years or more old. And the very interesting thing about them is that obviously there’s a lot of stuff there that’s out of date, but it’s very obvious what’s out of date, so it’s actually quite easy to get signal and the signal-to-noise ratio is a lot better. And there’s a lot of things that if they were valid 100 years ago, they’re very much valid still today, then they’re particularly true.
Tom: How did you come up with that idea? That’s such a kind of odd approach - did you just stumble across that as a source of insight or did someone suggest it to you?
Boaz: So this is something that I’ve heard recommended by other people in other fields as well. I mean it’s no coincidence that people to this day like reading the classics, so to speak, both in literature but in other fields. But the way I personally came across it is I was wandering around in the library and I came across a very old direct response handbook from over 100 years ago. And as I started reading through it, I realized “oh actually, a lot of the stuff that is mentioned here, if it’s not very clearly obviously out of date, then it’s actually very useful.” So that’s how I stumbled into this little niche.
Tom: It’s interesting, and I’m writing a book myself at the minute about the relationship between data, creativity and marketing, and it’s really frustrating because everyone wants to talk about generative AI, which I do as well, but by the time it gets published next year, that’ll be a whole different beast - may have faded or morphed, who knows. So yes, you have to think like “okay, how can I put stuff in this book that hopefully would be useful in 100 years time?”
Book Recommendation
Tom: Is there a specific book that you would recommend?
Boaz: One of the books I recommend to all of the new joiners here at Capital is a book that’s not actually related directly to marketing. It’s called “Reminiscences of a Stock Market Operator” and it was written almost exactly 100 years ago. But although it’s not directly about marketing, I think it touches a lot of marketing topics very well. It touches the importance of word of mouth, of how different narratives get constructed and how different narratives get disseminated. But also - and it’s very very powerful on this - it touches a lot on the instincts that drive people to make a purchase.
And although the absolute details of the companies and markets and products that this book talks about are entirely out of date - you know, it’s talking about companies that existed 100 years ago that no longer exist - nevertheless, a lot of the mechanisms and the procedures and the way this information spreads is very very much topical. And I recommend it to everyone that joins the company.
Advice for Data-Driven Marketers
Tom: So what advice would you give to today’s data-driven marketer to be better at the discipline?
Boaz: I think one of the big advantages we have today is that whereas 100 years ago - this is something that’s mentioned in the book - 100 years ago you had to pay someone to make you a chart about whatever you wanted to see and it took a couple days or maybe even weeks to get your chart. Today you can sit down at your laptop and look at your chart as many times as you want.
And I think that starting your day like that - starting your day looking at dashboards, interrogating them, digging through them, seeing how different things move together - is basically the fastest and easiest way you can have to becoming a good data-driven marketer.
Because if you develop a relationship with the data, if you look at it every day, if you interrogate it every day, if you look at how different variables move together, you will start developing instincts around how things move together and what happens when odd things happen.
You can look at a dashboard one day and you can see “oh, that’s kind of odd - usually when that line goes up, this other one also goes up, but today it hasn’t gone up.” And then you have to interrogate it a little bit, dig through it a little bit, and then it turns out that for some reason there was an issue with the Android version of your app, or there is some language that’s been mistranslated and there’s been performance issues in some countries, or something like that.
The point being that if you develop a good instinctive familiarity with the data - and you can only do this by looking at it every day - you will start spotting things that other people wouldn’t spot, and that will be an advantage.
Real-World Example
Tom: How do you do that specifically? Could you give me an example, maybe in a previous role, where you’ve had that kind of daily view of the data and you spotted the anomaly, you’ve dug into the anomaly, and you found an insight that’s been beneficial to the growth of the business?
Boaz: Yeah, I’m happy to share in general terms. The way I like to approach it is looking at the top-line business metrics that the top business leaders would be interested in looking at - the revenue numbers, the spend numbers, all of the important things that happen in between, all of the onboarding flows, split across different channels, regions, etc. So just getting an instinctive feel for the heartbeat of the business.
You know, are Mondays usually stronger than Tuesdays? Or is Friday the strongest day? Is the last Friday of the month unusually strong, or is it the first Monday of the month that’s unusually strong? Just sort of getting an intuition around the data heartbeat of the business.
And once you’ve done that, once you’ve acquired that instinctive feeling for it, you can immediately spot trends. So in particular, there’s an anecdote where I saw that our conversion rates in a previous business were dropping significantly. And I managed to, after digging around a little bit, figure out that there was a relatively niche issue in one of the implementations of the onboarding flow that impacted a subset of users that would have otherwise taken a significant amount for someone to spot because it was such a niche issue.
But nevertheless, because I had this intuitive feel for what was going on, I managed to figure out that something was wrong and relatively quickly identify it. Even though as a marketer I wasn’t strictly responsible for the onboarding process, nevertheless it impacted the performance significantly, so I could drag it out and try to figure it out with the responsible product manager.
The Shiny New Object: Predictive Modeling of Conversion Events
Tom: So we’re going to move on now to your shiny new object, which is predictive modeling of conversion events. Now I think I know exactly what that is and I’m really buzzed to talk to you about it, but can you explain to the audience what that is for someone who may have never heard of it?
Boaz: Yeah, I think I first started thinking about this concept when I worked at my previous company, which was a consumer credit business, which effectively you can think of as kind of like a credit card. And when you’re selling credit cards, when you’re quote-unquote “selling money,” when you’re giving money to people, the outcome is actually quite different than when you’re selling furniture, right?
Because if you’re in the credit card business, you have a customer that comes in and takes 100 pounds. Well, they can be - if they’re a loyal customer and they stick around for years and they pay back their debts on time, etc. - they could be a very valuable customer. But they could also be a customer that comes in, takes 100 pounds, never pays anything back, right? So effectively you’ve got this range of potentially negative values for a conversion event, which is not the case in e-commerce.
Right? In e-commerce, if you sell a chair, if you sell a table, if you sell a bed, whatever piece of furniture, then generally you know how much you sold it for and you know exactly what your margin on that is, and you can try to estimate it, and that’s the conversion value that you feed back to the advertising platform and that you optimize against.
Now I know that that’s not entirely true because occasionally you get returns and etc., so there’s some complications there, but on the whole it’s a little bit more straightforward. However, when you work in a fintech company and you’re looking at the value of the user that you’ve just acquired, the range is a lot bigger and in the case of credit can potentially be negative as well.
So I then started thinking, “okay, well how do I do a better job at actually giving the algorithms that end up optimizing all the ad delivery, etc., the data that they need in order to be able to optimize better?” And that’s where my thinking and obsession with these predictively modeled conversion events started.
Tom: I would say that was a fairly long answer, so could you be more specific? Could you just help someone - give me a basic description of what it is?
Boaz: So the basic description would be that instead of feeding back the raw data of what actually happened on your web platform or your app or your website, you take some function of that data and you modify the raw data based on what you think is going to be the real value and not the actual observed value.
So to give you an example, sticking to the furniture business example: when you sell a table, say online, and you know that your margin on that table is, say, 20 pounds, instead of giving the platform the 20 pound value that they can then optimize against, you give it some function of the 20 based on maybe - if it’s a new customer and you expect them to shop again, you multiply the 20 by two and you feed it back 40 pounds. Whereas if it’s a repeat customer that buys from you every week, you apply some sort of discount to it and you feed it 10 pounds, for instance.
So effectively it’s taking the observed values that you get from your advertising activities and modifying the real value to generate some sort of predictive or synthetic value that hopefully will drive better outcomes for you within your marketing process.
Tom: So basically you’re taking conversion events that happen on site - furniture, financial services, or otherwise - and then you’re predicting the lifetime value of that customer or a short-term value of that customer and reporting that back to the business?
Boaz: It’s effectively kind of like that, except you can also… it’s basically that. So if you simplify it to individual users, let’s say your goal is user acquisition - you want to acquire a user. Instead of looking at the first transaction that a user makes, you can also try to make an estimate of the user’s lifetime value upon the first transaction and feed that to the algorithm itself. And that is a perfectly legitimate approach and a lot of companies do this. In fact, it’s a very valuable thing to do in cases of, say, sparse data where you have relatively few values to actually go off, so you actually need to give the algorithm a bit of a confidence boost and give it a big number or as you only have small numbers.
It could also be valuable in the case in the example that I mentioned earlier with negative lifetime values. So for instance, if a user is probably actually not worth very much based on the behavior you’ve seen from them, then instead of giving a potentially high value that their first transaction would warrant, you could actually feed the algorithm a low value.
But in general, it’s not just using the LTV predictions - it’s about modifying the real values to make them a little bit more palatable to the algorithm that’s effectively making the bidding.
So to give you an example, I don’t know if you’re familiar with UAC campaigns or Universal App Campaigns, but effectively these are campaigns that are trying to optimize on Google towards an in-app event. Now I’ve heard from people that used to work at Google that effectively any value over $500 that is fed to this algorithm is actually just truncated, so it will just take the… you know, if you send it $510 as a conversion event value, it will just truncate it to $500, right?
So there is a limit to the values that the algorithm will actually accept, and there’s various reasons for that. But the underlying reason for it is that the algorithms that are used to optimize advertising performance in this sort of black box world that we live in are algorithms that make certain assumptions about the distribution of the underlying data.
Now what that effectively means is that if Google’s doing it on their end, then maybe there’s something that you can also do on your end to optimize the algorithm’s performance. So truncating values, for instance, or normalizing them, or all sorts of things like this is a good example that is actually relatively commonly used across performance marketing to try to nudge the algorithm into finding the results that you wanted to find.
Adoption and Implementation
Tom: And so how common is predictive modeling of conversion events and how much of a shiny new object is it in your experience?
Boaz: I think it’s pretty shiny object in the sense of a lot of people want to do it, a lot of people try to do it. I think it is potentially something that can be quite tricky to do, simply because depending on the infrastructure and the size of the team that you might have, you know, you might not actually have a dedicated data scientist out there being able to actually put together machine learning models to try to predict the value of every single account or user that you acquire.
That being said, what I think is quite common is a much simpler version of this, which is some sort of simple lead scoring algorithm. I think most companies that do user acquisition have some sort of lead scoring whereby they draw these lines of “if the user meets this requirement then it’s worth X, and if the user meets that requirement then it’s worth Y.” From that perspective, it’s I think relatively common.
Implementation Advice
Tom: So how would you advise someone… advise two people, so the team that is big enough to have a data scientist and a team that isn’t - how would you go about kicking off predictive modeling of conversion events in those two scenarios?
Boaz: I think the underlying philosophy - the most important thing - is to be familiar with the underlying philosophy, which is the goal is to make the advertising platform’s algorithm work for you as a business, not necessarily for the advertising platform itself, right? So you have to keep in mind that most of the time there’s alignment of interests, but there are some sort of scenarios in which it’s not necessarily the case.
I think perhaps there’s this familiar anecdote - I don’t know if you’re familiar with it, Tom, but I’ll repeat it for the sake of the audience - of the pizza flyer operation. There’s this person with a restaurant and they hire these three teenagers to hand out flyers in front of the pizza shop to hopefully get people into the restaurant. And the three teenagers go out, and every single flyer has some sort of unique identifier so you know you can tell which customer came from which teenager.
And then what you see is: first teenager at the end of the day gets three customers in, second one gets four, and the last one gets 34. And the restaurant owner is quite surprised and he asks them, “okay, like, how are you getting so many people to come in?” And effectively the teenager says, “well, I hand them the flyer before they step into the restaurant.”
So that’s sort of a long-winded anecdote towards incrementality, but the point I wanted to make is you have to be aware of the dynamics of incrementality when it comes to the actual conversion events that you’re feeding back to the platform, such that whether you’ve got a data scientist or not, you need to be aware of what is actually driving value for you and your business.
Now if you do not have a data scientist, I think keeping taking this to heart and being very very conscious of the fact that not all of the conversions that the ad platform takes credit for are actually real and they can legitimately take credit for them - I think that’s the single most important thing you can do. And if you keep that in mind and you interrogate the data based on that notion, you’ll do very well.
With regards to when you actually do have a data scientist and you can actually work towards building these predictive lifetime value models, I think there’s two things you can focus on.
First of all, generally when it comes to user acquisitions, what people use as a signal to estimate the lifetime value of the user is the first or the first couple of transactions that a user makes will very much inform the value of the user going forward. Now that’s kind of the baseline that the data scientist needs to work on top of and need to try to improve that model iteratively, right? Go through it: “okay, I know that if you make two transactions in the first day then you’re most likely to be worth X. Okay, well how can I improve this model? Does the amount of times you log in before transaction inform it?” etc., etc. And then that’s the actual value, if you’re technically capable enough and have the right infrastructure, that you can feed back to the advertising platform. That’s one thing that I would focus on in that respect.
The second is actually just testing - trying to test as much as possible what is it that the algorithm will work well with if you feed them these values versus what are the things that will simply not be functional.
And the last point I wanted to make is generally the more data points you can feed an algorithm, the better. So the more you can multiply your conversion events and not necessarily just limit them to the sparse high-value ones, but rather to the more frequent, concurrent ones, even if they’re not necessarily truly events that drive value, that will definitely help improve the quality of the ad campaigns.
Now I appreciate this has been a long answer to a simple question, but hopefully I haven’t rambled too much.
Conclusion
Tom: Well look, unfortunately we are at the end of the podcast now, and you’ve opened my eyes to some things. And I have to say I’m not sure I totally get all of it, and that’s how you learn, right? So thank you for showing me a different way of thinking about what is a niche of data-driven marketing. So I really appreciate that.
So if someone wants to get in touch with you about anything you’ve talked about, be it books or predictive modeling, how and where is a good way to get in touch with you?
Boaz: You can find me on Twitter @boazsobrado. You can find me on LinkedIn looking for Boaz Sobrado - there’s no one else in the world with that name. And you can also find my website boazsobrado.com.
Tom: Brilliant. Boaz, thank you so much for your time.
Boaz: Thank you.
About the Podcast
The Shiny New Object Podcast explores the future of data-driven marketing through interviews with industry experts. Host Tom Ollerton brings together insights from across the marketing technology landscape to discuss emerging trends and practical applications.