Full Transcript of Episode 04: SuperBowl, Sports Analytics, and Measuring What Matters:
Lizette: [00:00] Podcasting from Nashville, Tennessee, this is “Keep Asking,” the weekly podcast that helps you dive in a bit deeper and wider into the research providing insights into today’s church in culture. Welcome back. This is a little bit of a bittersweet week, the official end of football season, with the Super Bowl coming up.
[00:39] Since we’re going to be talking about the Super Bowl and a little bit of data we found out about what churches do on Sunday nights when they adjust their schedules, or those that don’t, I wanted us to start out with, “What are some of your favorite Super Bowl memories?”
[00:55] You don’t have to go into the whole Super Bowl, but which Super Bowl and what was it that was your favorite?
Scott: [01:03] Definitely for me, it would be when the Broncos finally won their first Super Bowl, John Elway leading the Broncos over the Packers. That’s because the two teams I followed the most are the Minnesota Vikings, who are 0-4 in the Super Bowl, and the Denver Broncos are my favorite team, and at that time, they were 0-4 in the Super Bowl.
[01:26] I endured a lot of losses in that key game, until we finally came through and ended up winning a couple in a row. That definitely stands out as the Super Bowl day, for once, not being a sad day.
Lizette: [01:42] All right. Thanks, Scott. Casey?
Casey: [01:44] This one’s a really easy one for me.
Lizette: [01:46] I think I have a guess.
Casey: [01:49] The Giants’ 2007 season was an incredible run. They beat the Cowboys at Dallas, they beat at Green Bay, when it was freezing cold in Lambeau, but then to see the culmination of that by beating the undefeated Patriots team in the Super Bowl, I feel like I’m going to be…
[02:06] The rest of my life, as a fan, I’m never going to have quite that same experience as what that was, so that’s as good as it gets.
Lizette: [02:15] Mine is Super Bowl XVI. I, as you all know, fell in love with Joe Montana a couple of weeks before that, in the NFC Championship Game, where they beat Dallas with “The Catch” catch from Joe Montana to Dwight Clark.
[02:30] I, as a sixth grader — don’t do math if you’re listening. No calculating Super Bowl XVI, sixth grader — as I say that to two math people, but just a very exciting thing. Contrary to your story, I jumped in as a football fan, and then had several glorious years with Joe Montana being amazing.
Scott: [02:57] Was that the year they played the Bengals?
Lizette: [02:58] Yes.
Scott: [02:59] That will totally fit our opening data, because I remember having to go to church that evening, sitting in a worship service, and quickly running home afterwards to catch what part of the game I could that evening.
Lizette: [03:17] Look at that. We recently dropped some data on what churches do. One of the questions that we asked, “What do you typically do during Sunday nights?” The responses went from evening worship service, small group Bible studies, student ministry, children’s activities, or other, or no Sunday evening. Among those who have Sunday night activities — is that right?
Scott: [03:46] Mm-hmm.
Lizette: [03:46] It’s any kind of Sunday night activities, we asked was, “Does your church typically continue, cancel, or adjust your Sunday night activities on Super Bowl Sunday?”
[03:56] 59 percent said they continue normal Sunday night activities, 5 percent said they cancel Sunday night activities, 24 percent say they adjust Sunday night activities to include watching the game, and 12 percent adjust Sunday night activities in other ways.
Casey: [04:37] I’m really not sure what I expected to see on this question. I think probably I would have expected maybe to see more churches cancelling than are. I do think that that highlights that there are churches with Sunday evening activities that don’t really feel like they need to make alterations based on other priorities people in their congregation might have.
[04:57] I don’t think we asked this question with, “Here’s the people doing the right thing. Here’s the people doing the wrong thing.” I think we asked it more in a conversation-starter. Whatever decisions churches are making, hopefully, this question just highlights those different perspectives, and maybe encourages intentionality in that decision making.
Lizette: [05:16] Scott, one of the things, as I look at this, I think this could be a hot topic for some folks, because I know that there are pastors who feel very strongly one way or the other, just like I know there are pastors who feel very strongly, “You shouldn’t cancel church if there’s ice on the roads.”
Are you just stirring up trouble by asking these questions? What are you trying to accomplish here?
Scott: [05:47] There are emotions around a lot of these topics, but there’s also practice, and we wanted to find out, “What are people actually doing when it comes to Super Bowl?” We’ve asked whether churches are open on Christmas Day, when it lands on a Sunday.
[06:05] A lot of discussion sometimes happens without the facts of, “How many people are actually doing one thing or another?” We always feel that facts are helpful to have in conversations.
Lizette: [06:18] Casey, I know you’re busting at the seams to talk about sports analytics in general. Go for it.
Casey: [06:27] I love numbers in general. That’s obviously the field I went into, as a statistician. I also like statistics for the practical component of it, that you’d say, “Oh, I can see how this helps.” I think sports analytics is one of those places where you see people make strategic decisions based on data and new information that they have available to them.
[06:49] We’ve seen people be extremely successful, the Billy Beane, Oakland Athletics, the Moneyball approach there, where incorporating data into the way that they operate, and seeing benefits from that. That’s one of those reasons why sports analytics is exciting to me, is that it’s readily apparent how it impacts things.
Lizette: [07:11] Pretend you’re explaining the phrase “sports analytics” to an eight-year old. I’m not saying anyone looks at it like that, but what is that? And when you say Billy Beane, who sounds like a cartoon character to me, what did Billy Beane do, what did he use? Is that the whole Moneyball movie? I’ve not seen it.
Casey: [07:32] That is pretty much the movie.
Lizette: [07:31] We don’t have to review the movie, but back up and tell me about sports analytics.
Casey: [07:37] I think, again, there are really well-defined rules to most sports. You know this. You have three outs in an inning. You know that 90 feet to first base, all those different aspects of it, but strategy within sports is always evolving.
“[07:53] Within the confines of these rules, here’s what’s allowed.” You have a million decisions that you’re making in terms of, “What should we do within the confines of those rules that’s going to give us the best chance of succeeding?”
[08:04] That’s one of those places where, I think, people are trying to come up with data-driven solutions, or trying to use all the available information they have to maybe make better decisions towards succeeding.
Lizette: [08:18] When you say “data-driven decisions,” sometimes I picture the little computer printer, the dot matrix, just pouring out page after page. Could a data-driven decision be like the card, the magical card that I hear is out there on whether you should kick the extra point or throw, try for the two points?
[08:37] Is that, “If the score is this,” or, “If your difference is by this much…”? You know what I’m talking about. I don’t know what it’s…
Casey: [08:44] Yeah. I think that there’s two aspects to that. One is that that card, I think actually, like you said, is mythical in the sense that I think that everyone has their own card, and I think that sometimes there’s variations on that, and I think that…
Lizette: [08:59] That really disappoints me, because I literally thought there was a, “This is when you do, when you do it” card out there that coaches used.
Casey: [09:07] I think that that probably existed at some point, at least somebody had to be the first one to say, “Here’s what you should do in each of these situations,” but again, there’s nuance to that. If there’s six minutes left in the game versus two minutes left in the game, there’s, “How’s their defense doing? How’s their offense doing?”
[09:22] There’s a lot of other pieces to that, so it’s one of those places where, “Yeah, let more data inform your decisions on whether you go for two or not.”
Scott: [09:34] One of the ironies is that small market teams — Moneyball was about a small market team in Oakland — some small market teams are using big data, some huge amounts of data, to make some innovations.
[09:50] It’s amazing in some of these sports that have been around for dozens, even a hundred years, there are a lot of new innovations of data, even though the sport really hasn’t changed that much.
Casey: [10:03] I think, unfortunately, for instance in the movie Moneyball, you sort of see that dumbed down to, “On-base percentage matters.” I feel like if I were going to try to say, “How did they use data in that, based on the movie alone?” Obviously, it’s a lot more nuanced than that.
[10:17] Yeah, to try, and a lot of times even going across from those traditional metrics of batting average, home runs, and RBIs — those are the three that people are looking at and saying, “Oh, this is how we evaluate what a good player is,” none of those take into account how often is somebody getting a walk.
[10:38] We know that there are pieces missing that are still meaningful in terms of player evaluation. Some of those more advanced metrics, just trying to incorporate all the information you have about a player to maybe do a better evaluation.
Lizette: [10:51] When you say metrics, again, let’s pretend we have an eight-year old here — and eight-year old is just a simple way for me to say, “I would like to get my head wrapped around that a little bit better.” Sometimes I have questions that are planned. I’m laying the groundwork, and sometimes I’m like, “I need a little more clarity.” I need a little more clarity when you say metrics.
Casey: [11:13] I just mean measuring things. That’s really…
Lizette: [11:15] OK, right there. I think that’s good. When you’re measuring things, perfect.
Casey: [11:21] You can’t measure everything, and I think that we’re aware that there are intangibles in sports, but that doesn’t mean that there’s nothing that you can’t or shouldn’t measure, either.
Lizette: [11:35] In sports, they’re really getting to where they’re measuring everything. There’s a group now that’s measuring and ranking every player in the league, and you’re hearing a lot more talk about measuring every aspect, counting every aspect.
Scott: [11:56] If you know for certain that a particular player shoots a 12-foot jump shot at 80 percent on the right-hand side of the floor, but at 70 percent on the left-hand side of the floor, which side of the floor are you going to try to get that player to?
[12:11] You can respond to it. That little 10 percent swing can be enough between winning the game and losing the game. We see in baseball that a lot of teams are now shifting their defense, where normally you have two infielders on the right side, two on the left side.
[12:27] They’re actually putting three, or even all four on the right side, because that’s where the player always hits the ball. By making those small adjustments to your strategy based on data, you’re increasing your probability of winning that play, and therefore winning the overall game.
Lizette: [12:49] That plays into probably one of my favorite books of all time — well, two of them — Football for Dummies, and, Take Your Eye off the Ball, two books on the fundamentals of football, learning the details.
[13:04] I think it was actually Take Your Eye off the Ball. I was going to say written by Howie Mandel, but I know that is not correct. What’s his name? He played for the Raiders.
Scott: [13:14] Howie Long?
Lizette: [13:15] That would be it.
Lizette: [13:16] I think one of the more fascinating things is he really breaks down what’s happening. I was in band. I didn’t play on football, so I’m wanting to understand a little bit more of what I see in plays. He breaks down what is happening on the offensive line.
[13:36] When I was reading that, it revealed to me — talking about strategy — these players, at a professional, college, and even at a high school level, they’ve studied what their opponents are going to do. They have a plan. They have basic things that they’re going to do.
[13:51] They line up. They prepare, but then as soon as they take that first step, they’re adjusting. They have a plan based on the data, but then they’re also skilled enough to adjust. For me, that is a huge key, I think, for how we, in the church, or how we in Christian organizations and non-profits can be thinking about blending strategy, learning from data, but also taking action when things change and happen upon us.
[14:23] You know, we get questions all the time. Should we even be involving data with spiritual things with the church? Do you have some thoughts on that?
Scott: [14:33] The reality is your ministry encounters and generates a lot of data every day, whether it’s your accounting folks, whether it’s emails that are coming in, or letters. Your website has a lot of information on it, social media. Even how you spend your time. That’s all data.
[14:53] The question is, “Are you learning from it?” and really as a leader, are you leading your organization to learn from that data? Three thoughts that I would offer to a leader in trying to help your organization learn from data. The first thing to do is identify somebody in your organization right now who currently is learning from data.
[15:15] They’ve shared an insight with you. They seem to be curious about that. Celebrate that. Support that, those efforts. The other thing to do with them is to give them access to more data. Most of our data naturally is separated.
[15:34] Our ministries do a lot of different things, so our accounting data’s in one place, our website data’s in another. If we have both products and events, those tend to be in two different systems. Our donor information’s in one place, and information on the people we serve is in another.
[15:53] Our data tends to be in different places. That brings up the second piece of advice, which is to begin to bring that data together, and to be intentional about where it’s stored and making it accessible.
Lizette: [16:04] What if someone sees my data? As you’re saying that, I’m having flashes of conversations we’ve had with clients who, they have access to this data, but not to that.
[16:17] I’m thinking of all the stories we’ve heard of rivalries in Christian non-profits. These are good, these are not bad ones, it’s just the reality of people with people. We’re protective of our Lifeway Research data.
Scott: [16:30] Exactly, and that’s where leadership comes in. You’re exactly right. That’s going to generate some turf wars, because I may be passionate about the products we have. You’re passionate about events that we have, and we don’t want our data to meet.
[16:44] In reality, that’s where synergy comes in. As a ministry, you can be learning from that data that’s joined together. There’s a lot of discussions today about Big Data. Honestly, for most of our ministries, it’s going to be hard to get to the point of what other people are talking about when it comes to big data, those kind of volumes.
[17:04] The more we start doing what we’re talking about here, we’ll realize that the tools we have — just an Excel spreadsheet — may not be enough to analyze that. That’s when we start to get into discussions of big data.
[17:18] I think that the third piece of advice I would give to leaders would be to be intentional about what you use to measure outcomes. Of all the different types of things you could measure, some of them are going to be what you consider outcomes, or the things that you say matter.
[17:37] Those are so important, because people’s actions in your organization are going to respond to what you’re measuring, and what you say matters. That can be, number of people in attendance may be a metric that you use, and that may be very important, or it may be the giving that’s coming in from your donors.
[17:57] It also can be some of those mandates that you put out there of, “Never let that phone ring more than two rings.” That mandate becomes a metric, and having conversations about those metrics are absolutely important, because your organization will make decisions you don’t even see in order to accomplish those metrics.
[18:22] Making sure you have the right metrics so that your organization is heading in the right direction is absolutely critical.
Lizette: [18:30] The next suggestion that I want to make, tying on one of your first points, is probably going to horrify the two of you.
Lizette: [18:39] When you said ID someone who is learning from data, celebrate and support them, my first thought was a lot of times — and this is a little bit of stereotyping — a lot of times, those people can be introverts. A lot of times, those people can appear maybe grouchy.
[18:57] If you’re not a data person or a numbers person, and you’re the big dreamer or idea person, those people are not necessarily fun to be trapped in a room with. But I do think it’s important that they are hearing about the vision, and that they’re also speaking into the vision, and they have time with the dreamers.
[19:22] What are good ways for…I’m thinking of Todd Atkins, who is not a data person here at Lifeway, but one of our more extraverted, guy in charge of leadership. What advice do you have for a Todd Atkins-type of person, or the more gregarious non-data person?
[19:44] How would you guys want to work with someone like that? How would you want to be approached? Not that either of you is grouchy.
Casey: [19:50] I do think that some of the skepticism there comes from a place of personality differences, but I also think there’s some of it that’s…We’re talking about metrics right now. We’re talking about quantifying something related to their ministry efforts.
[20:11] I think some of that skepticism is an awareness that not everything they do is quantifiable, and a concern that their contributions might be undervalued or put down based on arbitrary metrics.
[20:21] That’s the last thing we want is for people to say, “Well, we made up something to measure, and you didn’t do well at it. You’re not doing a good job in your ministry.” I think there’s some of that that’s kind of the underlying fear there.
[20:34] We’ve been talking about sports analytics. People talk all the time in sports about intangibles, things that we can’t quantify or measure. We know how much more in spiritual organizations are the things that are in that space of intangibles.
[20:47] Again, Scott was saying that there are some reasonable things, that you’re building metrics whether you know it or not in some sense, and I think what we want to strive for is to help people do assessment well.
[21:04] Those people that you’re talking about, whether they’re grouchy or not, can help benefit. Come alongside of them, and hopefully be able to change that scorecard, measure what it makes sense to measure, and make decisions.
Lizette: [21:16] The example that I have in mind to make it feel accessible is just a team knowing if they’re running the ball, a football team, and they know that they gain four yards on average every time they run the ball versus if they know they’re gaining one yard on average every time they run the ball.
[21:36] You’re going to have a lot more confidence running the ball if you know you’re getting four, five, or six yards, six being pretty phenomenal. If you’re running the ball and getting six yards on average, you’re going to be pretty excited.
[21:50] Just having that one measurement can help you have confidence, even in those times when you’re maybe losing a yard, or you’re not quite gaining. You’re getting two yards, but you know on average, you’re good enough that you’re doing that.
[22:08] I think that a person in an organization or a church, if they think, “OK, what’s one thing we can figure out what we’re doing, are we making gains? Is it getting just two more people involved in groups? Is it three more people on our staff filling out their time cards in a timely manner?” Not that we would know anything about that in our organization.
[22:38] What are those basic steps? Do you guys have anything? Casey, you played sports. Any way to translate it to the person who’s thinking, “I can’t do all of that.” If you can track sports data, if that makes sense to you, if you listen to sports talk radio, you can do this. You can do this.
Casey: [22:57] I think what it points out as well is that you mentioned the one-yard loss and right after that, feeling like maybe we don’t want to try this again. I think that that’s we’re extremely influenced by a recency bias, by anecdotal evidence. “What’s the last thing we saw? That’s fresh in my mind.”
[23:20] Sometimes we’re making decisions off of the most emotional experience to us, or the most recent experience for us, instead of based on all the information available to us. I think that’s one of the places where you can say, “Yeah, if I can incorporate some sort of measurement of, ‘How’s this going on average?’ not just how it went the one time that I remember it going poorly?” that’s definitely helpful.
Scott: [23:42] Most of our hunches about how ministry works are not 100 percent accurate, and data is what shows us those gaps. That would have been a perfect spot for me to make up a statistic, to say that most of our hunches about ministry are 10 percent off.
[23:58] In reality, just being off a few percent in those assumptions about how things work can be the difference between a donor mailing losing money or making money, the difference between an activity being worth doing or not being worth doing, or whether we can get enough people to that event or not.
[24:21] Those little data points that seem small at the time are really critical to how our ministries operate.
Casey: [24:27] Pointing back to what you said, Lizette, about knowing, “Four yards a carry. That’s helpful.” I think in the same way in ministry, we’re not saying, “Hey, every decision you make should be driven by data.” You’re not going to want a computer to run your ministry for you. That’s not, I don’t think, a reasonable suggestion.
[24:50] I think, again, having a few road markers to be able to look to and say, “This helps me frame this problem that I’m dealing with.” The best example I have for that is going back to that movie, Moneyball.
[25:05] You see that example of somebody finding success by ignoring the supposed wisdom of the baseball guys, relying entirely on the cold, hard facts of statistical analysis, the message being that those with experience in their field have nothing to add, and the data should drive all the decisions.
[25:23] They also made another movie called Trouble with the Curve that was based on the exact opposite movie. Basically, you see an organization failing by ignoring the wisdom of an old baseball scout while listening to the cocky know-it-all and his computer algorithm, the message being that data is useless, and undermining the wisdom of those with experience in their field.
[25:42] I don’t think the either of those things are the case, and I think that we recognize that’s not the case. You see sports organizations recognize that that’s not the case. They see there’s value to be found in the world of statistical analysis, and the world of experienced practitioners in their field, and I think striking that balance can help ministries succeed as well.
Lizette: [25:59] These movies that you keep talking about, there’s not going to be any racy scenes our listeners are going to encounter, are they?
Casey: [26:05] I think they’re both PG-13.
Lizette: [26:05] Excellent, because I just got alarmed by that.
[26:09] It’s time for our segment on listener feedback. Technically, this question came to me via Twitter before we launched the podcast, but I’m certain he’s a loyal listener by now.
[26:19] This one comes from @phudchambers, a pastor of Remedy Church in Rock Hill, South Carolina. Married, six kids, and a South Carolina Gamecocks fan. Let me clarify. He’s a really obnoxious Gamecocks fan. Full disclosure, Fud and I were in the same PhD cohort, but anyway, here’s his question.
“[26:40] What’s the most important food to eat when doing research?” and follow up, “How much coffee can one drink in a day doing research?”
[26:48] When you guys are working on a project, and it’s one of those long grind-it-out, never going to end, what gets you through?
Scott: [26:57] Totally depends on the time of the day. If it’s morning, donuts. You can never fail with donuts. If it’s late at night, to me, that’s when Doritos come in.
[27:06] [background music]
Casey: [27:06] I would say Dr. Pepper is kind of the staple if I feel like I need some more energy into whatever I’m working on.
Lizette: [27:17] Excellent.
Lizette: [27:18] Good job. All right, if you liked what you heard today, we would greatly appreciate it if you could pop in to iTunes and rate the podcast positively, if at all possible, but feel free to give us a critique directly via email or Twitter.
[27:36] You can email us your feedback and ask questions about this topic or questions about research in general at firstname.lastname@example.org. Let us know. Do you find it challenging to think about data in research and statistics in a way that relates rightly with the church, with letting God be at work and in your organization? Let us know.
[28:00] You can also find us on Twitter @lifewayresearch and @smmccon, @statsguycasey, and @lizettebeard. We will look forward to talking to you next time. Keep asking, learn more, and do better.
Links of interest:
Most Churches Hold Sunday Services, Super Bowl or Not by LifeWay Research
Data Tells a Story … Why Churches Should Track Everything by Carl Kuhl, Outreach Magazine
An Introduction to Data-Driven Decisions for Managers Who Don’t Like Math by Walter Frick, Harvard Business Review
What Role Does Data have in the Church? (ACS Technologies blog on Jan 27, 2017)
By the Numbers: Billy Beane’s 19th Season Leading Oakland A’s by Rich Campbell (Forbes.com)
Caution: Predictive Analytics May Miss One Important Thing by Jeff Weinberg
When teams should go for two (and why Jack Del Rio was right) by Nasir Bhanpuri (NFL.com)
Disclaimer: These are general links that expand on different topics discussed in this podcast. The inclusion of a link is not an endorsement of a website or an organization. It is simply a simple way to provide additional information for listeners wanting to explore these topics further.