.png)
Episode 094
Connor Joyce is the author of "Bridging Intention to Impact" and a Senior User Researcher at Microsoft on Copilot Team, where he is advancing the design of AI-enhanced features. Passionate about driving meaningful change, Connor advocates that companies adopt an Impact Mindset, ensuring that products not only change behavior to satisfy user needs but also drive positive business outcomes. He is a contributor to numerous publications, advises emerging startups, and lectures at the University of Pennsylvania. Based in Seattle, Connor enjoys exploring the outdoors with his dog, Chai, and a local event organizer.
CHAPTERS
0:02 Introduction
1:12 Connor Joyce and Bridging Intention to Impact
4:41 Creating the Impact Mindset
5:31 Contextualizing Behavior Change
5:50 Importance of Behavior Change in Product Development
9:30 Defining User Outcomes
10:18 Identifying User Outcomes through Research
11:08 Finding Measurables for Behavior Change
15:42 Transitioning Raw Data into Functional Metrics
18:38 Pitfalls of Prioritizing Usage Alone
23:33 Focusing on User Outcomes for Organizational Change
TRANSCRIPT
[0:02] Welcome to Product Voices, a podcast where we share valuable insights and useful resources to help us all be great in product management. Visit the show's website to access the resources discussed on the show, find more information on our fabulous guests, or to submit your product management question to be answered on our special Q&A episodes. That's all at productvoices.com. And be sure to subscribe to the podcast on your favorite platform. Now, here's our host, JJ Rorie, CEO of Great Product Management. Hello, and welcome to Product Voices. We've got a great episode today. We're talking about behavior and intention and product management, and it really is going to be a fun conversation. So I've got a very cool guest with me today. day. Connor Joyce is joining me. Connor is the author of Bridging Intention to Impact. He's also a senior user researcher at Microsoft on the co-pilot team.
[1:06] He has got a fascinating background and has done really cool things in product management. So I'm so excited, Connor, for you to be joining me for this conversation. Thanks so much for being with me on Product Voices. JJ, I appreciate it. I'm really glad to be here.
[1:22] So, let's start with your book, right? I love to have authors on the podcast because it's just such a fun process to write a book. All of the research and all of the hard work. Fun is one word, right? Right. But but tell me, tell me about your book. Tell me about, you know, my my favorite question is, you know, what led you to write it? Right. Why this book? Why this topic? Tell me a little bit more about your process, about, you know, how you got to to writing your book. Yeah, absolutely. I love the question. And I could probably write a thesis on writing this book. So I'm going to give it to you as quick as possible, and then we can dive into any of the details that you're curious on. I did my master's in behavioral science at the University of Pennsylvania, and one of the things that I saw there was behavioral science has the potential to really change behavior.
[2:22] And there's a lot of examples in academics of people creating interventions that actually have a pretty strong effect at driving at a certain new behavior they want people to have. And I thought, this is where I want to build my career, but I want to do it at scale. I want to do it in a way that I'm going to be able to impact as many people as possible. And so I thought, well, the natural place to do it would be technology. So long story short, wound up at Microsoft soon after that and was excited to see that there were a lot of people thinking about about how to build features to drive behavioral change, but at the same time that the metrics being used were not always those that would be able to ultimately go and again.
[3:06] Measure to maximize the behaviors being changed by the specific features. So while I was at Microsoft for my first time, I had the opportunity to work on some projects there that began to drive at this. What would it look like to measure that actual behavioral change? So we were ensuring that we would build features to drive those new behaviors. And I found a lot of individuals, especially the product managers, very excited by this work. Product managers, engineers, designers, really all the groups, but especially the PMs, because it was PMs are driven generally by the metrics that they are ultimately trying to maximize. And so I thought, this is really interesting, but is this just a single company problem or is this something that we're seeing across the board? And so I left Microsoft and I was at a couple of different companies, startups through midsize, and I continued to see this challenge over and over of we have the intention to build a product that's going to help users do X, Y, Z. But the metrics that we're using to actually tell if it's doing that are usage and satisfaction. And so I set out to write a book about a new philosophy that I call the impact mindset that's all about creating those metrics to really ensure that the products are changing behavior that ultimately satisfies those user outcomes.
[4:30] But does so in a way that they continue to stay with that business. So in other words, you're building something that works for the user and in
[4:38] turn is profitable for the business. And so that's what I set out to do. And I'm excited that the book will be coming out in August and sharing that message.
[4:47] That's amazing. It's really cool because I think it connects next, you know, two really critical pieces, which is identifying the behaviors that we want to change in the first place, right, which kind of gets wrapped up in our feature work and that sort of thing. But to your point, I don't know that enough product people think about it this way, and hence your work and the book, which is, are we measuring it in the right way, right? So we We may have intentions of changing a certain behavior, but how do we know we're doing that in the right way? And so I think this is very exciting, and I'm sure a lot of folks will find
[5:30] a lot of value in your book. So let's dig in a little bit. Like, talk to me a little bit and set the stage or the context, if you will, of behavior change in the first place. Like, why is it so important to have that type of mindset as opposed to, you know, they ask for this feature or this is a needed feature,
[5:49] we're going to throw it in the product? Like, tell me about the, how can we contextualize behavior change in the work we do and the products we put out for our customers? The way that I look at it is that if you ask somebody why they use any product, and we don't even need to talk about digital products here, just truly any product.
[6:13] They're going to have a reason. Now, the reality is, is they may not always know actually why they're using it, but they're going to have a reason. And the more that you dive into the actual factors that are compelling them to go and choose to spend their time, spend their energy interacting with something, there is a purpose behind it. People don't use products just to use products for the sake of themselves.
[6:40] It is about satisfying some sort of outcome. That in some cases can be entertainment, in others it can be information. In the book, I use a lot of health and finance because they have some of the most easy to ultimately talk about outcomes, but really every field has some sort of outcomes that they're driving their users for. If it's a fitness app, it means that the outcomes people want are more fitness activity. If it's a finance app, it's more savings or more investments or whatever it might be. So people have those outcomes. And so that's where I really start the book around is thinking about these outcomes. But outcomes don't happen automatically.
[7:22] By their own. They don't just materialize. It requires manipulating an individual and the environment that that individual is ultimately in. And I say manipulating is definitely a behavioral science word. It also has a negative connotation. So I just want to, when I say that, I mean, truly it means changing somebody's mental model of how they're approaching their life or changing their environment or having them change their environment in a way that they take a new behavior. And so that is the basis of the book comes down to what I call the user outcome connection. And the first piece of that user outcome connection is identifying the user outcome that person's trying to drive. And then from there, the next piece is defining what specific behaviors that person has to change in order to drive that outcome. So again, to really illustrate it with a simple example. If you have a fitness app that is intended to get somebody to level, say like a, I think they're a couch to 5k is one or, or my fitness pal, a lot of these that are intended to help people get into a better level of physical fitness.
[8:35] That's the outcome. It could be weight loss. It could be higher endurance. It could be a variety of health outcomes. Each of those have very distinct behaviors that are going to ultimately lead that person to that outcome. And it's all about identifying those behaviors, because those are the measurable pieces, as are the outcomes. But the behaviors are the measurable pieces that really matter when that person ultimately is interacting with that digital app or product. So how do we as product teams, because you mentioned, sometimes people.
[9:14] Don't even know or at least can't articulate why they're using a product. Right. Or even what their outcome, what their desired outcomes could be. So how how can teams identify? And sometimes maybe it's simple and sometimes maybe it's not those user outcomes.
[9:30] Right. And, you know, do we have to do some facilitating, some interpretation, even some creation ourselves and then educate the user on it? But tell me a little bit more about that, the creation, if you will, or the defining of those user outcomes. Yeah. And it takes, I think, somebody who, while I have worked in product for a good chunk of my career and I've built my own products, I am also the majority of the roles that I've held are researchers. So it probably takes somebody who has this hybrid to come in with such a research-focused approach to product developments. But I believe that that is where the real profit is to be made is, again, to really satisfy those user outcomes.
[10:15] But to do that, you do need to understand them. So there's a lot of directions I could ultimately take this. And I think it really depends on where a company is in their lifecycle. So it really starts with product market fit.
[10:29] And very early on, companies are going to be thinking about, should be thinking about, how exactly are we solving a single or couple of very tight problems for that user? Well, if you do that correctly, you should begin to, right there, begin to understand what are those outcomes that are trying to be driven. If we are building a brand new app to go and help somebody, better yet, to be very specific about it, to help low-income individuals build an emergency savings, well, then that is the outcome that ultimately we want to drive after with the
[11:06] behaviors that we're trying to create. I use the app Acorns is, I think, a great example of that specific example of them being able to find a very niche market and then find simple features that can change behaviors to help people create that emergency savings. But getting back to answering your question, it really is. Once a company has a product in the space, it's about user research and it's about understanding from the users, the future potential users, and competitors.
[11:42] How users are interacting with competitors, to really understand why are they choosing to spend their time on those specific products, and which features are they ultimately using within those products, to better understand at least why do you think you're using this? That's a great place to start. And then, And ideally, you can go and abstract that further and begin to drive at the mental models that are driving them to choose to use a product differently.
[12:16] Ultimately, with the aim of identifying what are they trying to do in their lives that are going to that that is yielding that usage. So it's simply put, it's it is getting in front of users, but it's not you can't always just ask them, what are you trying to do? It's about triangulating multiple different points from interacting with users to really begin to understand. Again, we only have 24 hours in the day. we all make many decisions throughout those 24 hours about how we spend our time. Why is that person choosing to interact with that piece of software? That is probably the outcome that they're trying to achieve. You know, again, there's not this perfect answer in anything in product management, I think, right? Where, oh, you do A and B and then you'll get C every time. No, that's not how any of this works. And so same with this. It's like there's not, you know, of one answer to this. It depends on the company. It depends on the place they're in, the maturity, et cetera, the product. But that makes a lot of sense. So let's connect. Let's connect that, right? So the user outcomes and the connection, right? So how do teams go about really finding the measurables that are going to drive that behavior change? What does that process look like? And as opposed to some of the measures that we know and love, what are the process that we go out there to find the ones that are actually going to make a difference?
[13:44] For this, I'm over here trying to remember how exactly to say it because I'm embarrassed that even though I've been writing about it and I still struggle to pronounce the word ethnography, I'm saying it incorrectly, but there is a field of study that is all about actually watching people and understanding how they act in their natural environment. And so I start with that frame of reference. In a perfect world, if you have an app in the market, you could go and watch, and I should say a product of any kind, not just app, in the market. If you could go and watch that person interact with it, just sit over their shoulder and see what they're doing with it, And that would be the prime way to understand what behaviors are being changed by the software in its present form. And so the reality is that, yes, someone could do that at a small scale. And again, that's where ask your local user researcher, and they will probably be familiar with some of these more ethnographic study types.
[14:59] That's the magic of and why I wrote the book mostly for digital products is because digital products offer the opportunity to do that exact same thing, but at scale with things like event data. So whether it be a segment, an M particle, or another one of these products that are able to capture a lot of data on every action that a user takes, that is what you're working with as a product team now. Now, obviously, it's not that easy. And you still need to take that raw data and turn it into metrics. And that's a lot of what I talk about going from having huge sums of data to actually building functional metrics and thinking through and then ultimately
[15:40] technically creating those metrics. That's where a good chunk of the book rests at. But it starts with recognizing there is actually a lot of raw data that is potentially out there. It's just about identifying what you actually want to measure. And so to identify that, again, it's first, whether it be user researchers, whether it be product members, whoever can, if you can get in front of a user and see how they're interacting with a product. Meanwhile, asking them with the knowledge of the outcome they're trying to drive, do they understand how these behaviors that they're taking are connecting to that outcome? What other behaviors are potentially available for them to be able to do to change that outcome?
[16:28] And then it's talking to research participants. It's also just doing some desk research that's potential here. If you are building, again, a fitness app, you can do some desk research. In other words, look up some academic articles on how do you drive weight loss? How do you drive better eating habits? So to the point of the metrics, first, you need to define the behaviors. If you define the behavior as well, realistically, most companies in the digital realm will be able to find the raw data to measure those behaviors. And then it's about actually creating those metrics. Yeah, yeah, I love that. And, you know, it's interesting because...
[17:09] I think as educated and somewhat sophisticated as most product people are, meaning they worked on, you know, technology or some sort of complex products, we get intimidated quite often by, you know, some of the more human side of things. And I think that's a pretty general statement. But I think you know where I'm going. I think sometimes we tend to, even when it's data-driven, even when it's kind of scientific-driven, we get a bit intimidated, I think, by, you know, behavior and the human aspect of things. And it's so, so important. Everything we do is build products for people, right? And so I think it's, I love this conversation. One thing I want to dig into before a question about, you know, kind of how teams maybe can go about this or taking the first step, if you will. One of the things I like about your book is that you do talk about how it's, you know, the pitfall of only looking at usage, right? Right. And I think it's so easy for us as product folks who are who are ourselves, right, our products and our our success is measured on product success and product usage for the most part to simplify it. It's easy to say, well, usage is up, so we must be doing it all right.
[18:30] And that's that's obviously erroneous, but it's easy for us to say or, you know, who cares because usage is up.
[18:38] So talk to me a little bit about your research and about your philosophy on why we as product folks and product teams need to avoid that pitfall, even when usage is going up, if we're not doing the right things for the long term. Talk to me about those pitfalls of prioritizing only usage. From all the research that I've done, and this comes from the product management reports that come out, this is both industry level and independent with my experiences. I believe the biggest pitfall that product teams fall into is the assumption that usage alone means a product is working. Right.
[19:21] And what that ultimately means is that to the point that you just made and some of my earlier points, when somebody chooses to use something, that means in that moment, they're being drawn to it. And our modern UI and really even UX these days with all the gamification elements and with all of these other pieces that are constantly drawing people in behavioral design patterns. I mean, this is an area that I've done research into. Everything is being created to draw us towards really heavy engagement, which is all right to an extent. If it is, no one's going to complain, again, about that weight loss app. Or better yet, I talk about Duolingo a lot. You don't hear that many people complain about Duolingo, even though in many ways it is one of the most gamified and quote-unquote addicting apps.
[20:15] But it actually works. And there's a study I referenced in it, and I wish I could remember the authors, but it pretty much suggested that using Duolingo for a period of time, and it was a shorter period of time than two years of college, and it was more effective. So it was a very strong effect that using Duolingo, I think it was six months, but don't quote me on it. And so nonetheless, using Duolingo is actually very effective. Perspective, if somebody is consistently using it. And Duolingo does a great job at actually getting that consistent usage. But the important piece there is that people stay on the platform because it's working. And if you look at, especially, I believe it was, and I can't quote the specific reports, but I know that ProductPlan, I believe Pendo, a few of these product management related companies have these big reports they release every year. And a reoccurring theme for the last three or four years has been retention. Why do we keep having leaky buckets?
[21:22] At the same time, we're seeing the ability for companies to build new product is beginning to decrease with all of this AI, all of the augmentations that AI offers and efficiency gains that AI offers. So we're seeing more product, more features, but at the same time, people are not retaining to these products. Why? My main hypothesis, and again, what I spell out in the book is the fact that because it's not actually satisfying those outcomes, so it's very easy for someone to say, I'm going to switch to the next best thing because this current thing is not actually working for me. So yes, I'm getting drawn in and we see this spike in growth. And to your point, whether it be product teams, investors, finance people, everybody loves to see a chart that's shooting up to the top right. But that is just one metric.
[22:17] And the other metric, again, is that retention. Are people actually staying on? And then is it actually like you can also look at things like upgrades and other business metrics that you'll begin to question? Ah, well, yes, they're using it. But are they actually finding it beneficial enough to want to stay and continue to grow with this product? And you only can really draw at that when you begin to satisfy those outcomes. So yeah, so ultimately, it is about making sure that those behaviors are being changed, and that that is what is driving usage, rather than just the design elements. Yeah, the behaviors being changed that are yielding the user outcomes.
[22:57] And that is what's driving the actual usage that one is seeing, rather than design elements and other factors that are that are there to kind of juice usage for a little while, but don't really offer any longevity. Longevity that's that's so spot on and makes so much sense um so so final kind of question i want to ask you is just to bring us back to the the work right that that we do and you know i i'm sure there are lots of folks out there um listening who who love the idea totally buy in are going
[23:31] to go get the book because we're going to link to your book um but but what do teams and organizations need to do, need to change to get started and really focus on measuring these outcomes and being kind of this keen user outcome connected organization?
[23:50] What do they need to do? Yeah, the basis of it is that.
[23:56] I see this user outcome connection, which I already introduced. Behavioral outcomes connect to user outcomes, which ultimately connect to business outcomes as that third piece. So in other words, if we're trying to drive weight loss, what behaviors can we do to do that? It could be increased workouts. It could be less calorie intake, etc. Okay, we're seeing that when When people do that, it's changing user outcomes. And ultimately, when people on our platform see that they are losing weight, they decide to continue to use the product. So that would be that user outcome connection. I believe in the philosophy of the book is all around that user outcome, that that is the place to start, is to ensure every feature has that user outcome validated. Now, building it out is the first step, but validating it, you need to have the data to know, yes, the feature is changing the behaviors.
[24:55] Yes, when those behaviors are changed, it changes the outcome.
[24:59] And yes, changing that outcome leads to those positive business outcomes. Outcomes that is that that's the core philosophy from there it is about building an evidence-backed decision-making culture and that requires experimentation because one needs to do some experimentation to create those validations it also requires a culture of using that evidence in decision making which is in the word but it actually is a cultural aspect to it also and and And I talk about it in the book about a variety of different ways that that can happen. I encourage a grassroots movement to your part. Hopefully there are people that are listening to this podcast that go, huh, this is actually something that I'd like to try to see. I've built the book to make that person, make you, listener, that are excited about this, be the grassroots movement for this type of thinking. And the book comes with a variety of different, lots of case studies, but also it has some some complimentary elements to it, some templates that someone can use so that the day that they start reading it, if they want to go and work along each chapter so that they can start to build that movement and build their first user outcome connection, they can do so. And that's my aim with it ultimately, is to be a tool that one can use to build that type of culture.
[26:27] I love that so much. And I think it's so important. One of the things that's so great about the book is that you do allow, um.
[26:35] Different roles or different reader personas, if you will, to grab a hold and make change in their own world, right? Because a lot of times, you know, anytime a culture is involved, we think that it comes from the top down, or we think that the leadership has to impart some big change or something to be able to do that.
[26:57] And the truth is, there are lots of ways that we can embed and change culture from the bottom up, from the middle, you know, up and down and in a grassroots way. And so I think that's very, very important. So I encourage anyone listening who knows this is a good thing to not be intimidated by the fact that, you know, yes, it may take some work, but you can do it, whatever your role. If you're a product leader, business leader, definitely go get the book and read it and see what you can do for your organization. But, you know, product managers, designers, et cetera, we all can make a difference. And I think that's really important. And one last point to something you said, I think it's important that we hold ourselves to account for making, using the data and the evidence in our decision making, right? Right. It's so easy to go through the process of getting data and evidence and then default to our kind of old way of thinking and actually making decisions. And I think that's that's an experiment in itself. That's a culture change in itself. And so hold ourselves to account when we're not doing that, when our peers aren't doing that, even when our leaders aren't doing that, you know, making decisions contrary to the evidence. You know, let's let's talk that through and see.
[28:15] Are there intentional business decisions there or are we just falling back to the old way of doing things? So I think that's a really, really great point. And again, the book I know is going to be so valuable and I've had a sneak peek of it and it's amazing. And so, Connor Joyce, thank you for joining me. Thank you for sharing just a snippet about the book, Bridging Intention to Impact. We will have links to it. Please, everyone out there, go get it. It's amazing. Connor, thank you so much for writing the book and for joining us and sharing a little bit about it on Product Voices. I really appreciate it, JJ. Yeah, it was great to share about it. And if anybody wants to, if you read it and you get excited about it, my company's desired outcome labs that you can reach out to. I'm also frequently on LinkedIn. So feel free to reach out on LinkedIn. And yeah, I would love to hear how the tools and the techniques in the book are being implemented. So please feel free to reach out. That's awesome. And we will definitely link to your LinkedIn and to your company. And of course, on how to get to the book. So again, Connor, thank you. And thank you all for joining us on Product Voices. Hope to see you on the next episode. Thank you for listening to Product Voices, hosted by JJ Rorie. To find more information on our guests, resources discussed during the episode, or to submit a question for our Q&A episodes, visit the show's website, productvoices.com. And be sure to subscribe to the podcast on your favorite platforms.
[29:43] Music.
Comments