Fireside Chat: Taj Carson on Nonprofit Data Strategy

Listen to Podcast

Like podcasts? Find our full archive here or anywhere you listen to podcasts. Or ask your smart speaker.

This time on Transforming Nonprofits, Kyle catches up with Taj Carson, CEO of Inciter, to geek out on data. What does data mean for information strategy, technology, and the nonprofits of tomorrow? The topic of data is only becoming more and more important, but for so many nonprofits it can be an elusive topic. How are nonprofits making meaningful strides in data?

Data has always been important – from monitoring and evaluation to getting an annual report right. With the increasing availability of AI, data is more important than ever.

In this conversation, we learn Taj’s approach to data for nonprofits, her thoughts on the impact of technology and how she thinks good data helps nonprofits choose technology in the future.

Taj Carson is the CEO and Founder of Inciter. Inciter maps data, integrates data systems, and automates data processes. The outcome? Beautifully designed, accurate reports. Effortlessly. Inciter is essentially a Managed Services Provider (MSP) for your data, supporting nonprofits across the many systems they use, regardless of what those systems are.

Our Fireside Chats are designed for audiences with varied experiences with technology. In this Fireside Chat with Taj Carson on nonprofit data and strategy, learn data basics, how Artificial Intelligence (AI) is changing the value of your data to your own nonprofit and to your community and the sector, and how to harness your data to make strategic decisions.

Transcription

Kyle Haines: Hey, everyone, welcome back to another episode of Transforming Nonprofits, where we explore topics related to nonprofits and technology. This episode is all about data, and I’m joined by Taj Carson from inciter.io, and we get to geek out on data. I was especially excited to do this episode because I wanted to chat more about data and what it means for information strategy, technology, and the nonprofits of tomorrow.

As many of our listeners know, the topic of data is only becoming more and more important. But for so many nonprofits, it can also be a very elusive topic. I wanted to have this conversation and likely more conversations in the future about how nonprofits are making meaningful strides with data.

Data’s always been incredibly important, from monitoring and evaluation to simply getting an annual report right. But with the increasing availability of AI, data’s getting more important than ever. This conversation with Taj gave me the opportunity to better understand how she’s approaching data with nonprofits, her thoughts on the impact of technology, and how it might help nonprofits choose technology in the future.

With that, let’s get into the conversation.

Thanks so much for being willing to be on Transforming Nonprofits. Everyone at Build is excited to hear our conversation. We’ve been sort of circling around this conversation for a while, and I just really appreciate you making time to connect today.

Taj Carson: Yeah, I’m really excited to be here, and I can’t wait to see what kind of questions you’re going to fling at me.

Kyle Haines: Hopefully, it is nothing that stumps you too much.

Taj Carson: I don’t think it will be.

Kyle Haines: Excellent. Well, I think a great place to start is telling me, or telling our listeners, rather, a little bit more about what led you to found Inciter, what Inciter does, and I think in your answer, people will understand why I was so excited to talk to you today.

Taj Carson: Sure. So originally, Inciter was founded as an evaluation consulting firm. So that was my background. My background is as a social scientist, and I started it to help nonprofits with evaluation. And I founded it because there just weren’t enough consulting firms that could work with nonprofits that needed support on data and evaluation, but they didn’t need a huge, randomized control trial. But they also didn’t really need a sole practitioner.

They needed a firm that can work with them to meet them where they were and come up with an approach to their data collection, evaluation, and performance measurement that would fit them. How do we come up with a strategy or a solution that fits where you are and not just what’s reliable and valid? What’s the cutting-edge thing? What’s the funder want? And always sort of around capacity building. I come from a space where I’d worked on federal government contracts and other work that really involved capacity building.

And I really was excited about that. So that was 21 years ago, which feels like a lifetime now. And we’ve evolved a lot since then.

We still do evaluation work, but pretty quickly that same drive led me to see that there’s also a lot of nonprofits that had trouble with data, right? So not just the data for evaluation, but with their data in general. And again, you’d have these IT firms that could build things. And then you would have software providers that could sell you software. But the whole idea of what fits for you? How do you need to manage your data? What are the different strategies and tools that you need?

We initially started, because I felt people weren’t getting excited about their data the way that I do, and it’s maybe not just because I’m a dork. And I thought, maybe data visualization, maybe that would really help. The communication aspect of data and how can I help by visualizing information and helping people visualize. And that was great.

And at the same time, I was thinking a lot about those three parts. And again, this was probably 15 years ago.

1. How are people getting data into systems?
2. How are they storing the data?
3. How are they getting data out?

Which is that reporting and visualization piece.

And the question I kept asking is, what technologies are out there that nonprofits could take advantage of? And what are the ways that we could use or repurpose and make it fit for them?

And so that’s continued to evolve to what we do now, which is a lot of times when we’re doing an evaluation or other performance measurement work, we are storing it in, for example, a database or a data warehouse.

We do a lot more data strategy, data governance, working with all those different systems to say, how can we pull those together and deal with data integration without building something brand new in terms of the input? And so we do a lot of data warehousing and still do a lot of data visualization to support that, but in a way that fits where they are and what they need and what’s most important to them.

Can a Technology Solution Solve Your Data Problem?

Kyle Haines: It feels to me like many nonprofits have made significant investments in predominantly software applications or solutions with this notional idea of it’s in service of data. Three to five years later, they don’t feel like they’ve really solved the data problem. And I think I heard you say a bit of that. I think I heard you say a version of that earlier. Has that been your experience as well?

Taj Carson: 100%. Yeah.

And it sounds crazy, but this is what happens. They’re like, will it do these things as opposed to sort of thinking about their, I like to call it the data ecosystem. And thinking about all of that, what’s important? Does this even need to go anywhere else? Or is it fine where it is? Being able to make those decisions.

I do see that a lot and I see people going from, and the hardest part is to see nonprofits who don’t have a ton of resources going from system to system to system. That’s the really painful thing where switching systems is not going to save you. You need to really take a step back.

And sometimes they’re in a system that actually will work for them if they could get it configured properly or stop asking it to do things it’s not designed to do. So yeah, I see it all the time.

Kyle Haines: It makes me think about one of our clients who I work with. There’s a lot of value in understanding program evaluation data. And just to put that into plain words, people come to educational programs, and they fill out an evaluation form. I think so many organizations focus on, “I’ve got to bring that information into Salesforce, and I have to associate it with that person so that I can evaluate it later.”

And what I heard you just say, or at least what I think I heard you just say, is why not leave that data in place? There is inherent value in that data, but so many people keep chasing these solutions, they’re going to bring data into a solution like a CRM solution or bring data into an association management solution or whatever the right solution is.

Taj Carson: Yeah, so then the question you’re asking me, so I assume you’re not talking about paper surveys, right?

Kyle Haines: It’s just a simple tool like SurveyMonkey, which is a point…

Taj Carson: Or an Excel spreadsheet.

Kyle Haines: Yeah, exactly. I mean, it is a purpose-built solution that does something really well.

And I think this client fortunately doesn’t do this, but many of our clients are like, “I’ve got to find something that integrates with all of my solutions so I can bring it all. It has to move through all of these different systems so that it resides in different places.”

And if they had a data strategy, going back to something you said, they might realize we need to retain that data and occasionally evaluate it, but it doesn’t need to live in all of these different solutions.

Taj Carson: Right, because here’s a perfect example. You’re doing a training or a community education workshop or a public health workshop, whatever, and people go to the training, and they fill out the survey, and maybe you’ve even got a pre and post-test survey, and you never see those people again. You do not need to make that data relate to other data in your system.

I think what you’re saying is, you can totally just run the descriptives, take a look at the open-ended, get the information you need, and it doesn’t need to play well with the other systems. And this is really the important question. And I can’t tell you how often I’m like, it might be fine where it is, right? Everything doesn’t need to go into one place. And that’s a really good example, when you’ve got people that there’s not, you’re not going to see them again, or they’re not going to show up in any other of your systems or any other data sources, or you don’t need to report on it that way.

This goes to that data use thing, right? What are the decisions you need to make? And what is the information you need to do that? And that helps you decide, do I need to actually bring it together?

Kyle Haines: The thing that I keep thinking about, and we’re actually starting a project that is about what you talked about earlier, defining a data strategy. And for me, in the last couple of years, that’s become a big light bulb for me. That absent a data strategy, technology decisions become much more difficult. Unless that technology…

Taj Carson: They’re easy to make. They’re just not easy to get right, right? It’s easy to make the decision.

Foundational Questions About a Data Strategy

Kyle Haines: And maybe a refined way of saying that is, perhaps if the solution is meeting some operational need. And of course, data comes out of that. You don’t have to start with a data strategy, but it seems like a miss for so many organizations to not articulate what data is important to run the organization, what data is important to execute your mission or to delight constituents or whatever those questions are.

And it seems like those are foundational questions that you want to answer. And then think about the technology and the reporting and the data and all of the things that you need to put in place to get that data. Does that resonate?

Do you agree with that? You’re allowed to totally disagree with it or say that’s only part of it.

Taj Carson: No, no, I think that’s true. And I think people here, I think there’s a lot of words we’ll probably use today that scare people a little bit. And I think people say data strategy, and then everybody gets overwhelmed immediately, right? Like, oh my God, it seems like this really huge, enormous thing.

I would say that we run into organizations where the finance people have their data in one place, and the evaluation department has their data in another place, and the fundraising people, and they’re just not, nobody knows about it all.

It doesn’t even have to be this huge year-long effort. It could just be, can we get all the people with all the different data uses to sit down, and can we just document it? We’re big fans of documentation. Just say, okay, what is it? Where is it? And how are you using it?

And that is an amazing start, and you would be surprised how many organizations don’t have that.

And I think people think that a data strategy or a data system is I’ve got to make this giant plan that takes us to the next 20 years, and then I’ve got to implement it all at once, right? And I would say that data strategy and data governance are really just document where you are, because that way you don’t have somebody’s random software that they purchased with their company card storing some very important piece of data, and nobody knows how anything’s defined and they don’t know where it is.

And you really can, just getting it down on paper, so to speak, it can just bring up a lot of light bulbs, and it doesn’t mean you then have to go and implement some big technology solution, but you need some way to have some communication around it.

And somebody’s got to hold that. And I think that’s the piece that often is missing, is the IT department’s holding the network administration and some other things, and then the finance department holds some, and the fundraisers hold some, and then the evaluators hold some. But sometimes in leadership, you don’t have somebody who says “my job is to make sure I know where everything is and that we’re talking to each other.” And that can be a really big missing piece, is like kind of who holds that.

And that’s how you get these dark, shadowy corners, where you have these data systems that nobody knows about. And then you also have, there’s a whole other topic around legacy systems that somebody built 15 years ago, and there’s only one person in the organization who knows how to operate it.

And data governance is the same way, right? You can’t do anything without good data governance. And that’s the other thing people trip over.

They say, we’re going to buy new software and that’s going to fix it, but our data quality is terrible, or we have no idea whose responsibility it is to enter this data correctly, or to define the data, or secure the data, or even decide what our data security ought to be. We don’t know who does it, it’s not clear, we’ve got three people that are doing it three different ways, and now you’re going to put that all into a new software system, or a data management system, or God forbid a warehouse.

But it’s the same thing. People think this has to be this huge lift, and it’s really just figure out where you are, and then just figure out how to make it a little bit better.  

And I think for nonprofits in particular, that applies to data strategy, it applies to data governance, and it applies to data integration, right? Don’t integrate everything all at once. Pick a couple of things that are really valuable and start there.

Kyle Haines: Yeah. I’m just reflecting on examples where organizations took a first step in creating a data strategy, and what enormous dividends it paid. I think what I heard you say, and I would support this idea, is that it wasn’t a lot of work. It was a couple hour long meetings to talk about what data was important and what we wanted to measure.

And then we went out and built the processes and the operations to begin to capture that data. And truthfully, we didn’t have to make a ton of technology investments. We just had to sort of start connecting things that were already in place. It wasn’t, “oh my gosh, we have identified this high-quality data that we need. And oh, by the way, we need a half million dollars in technology investments to be able to get there.”

Taj Carson: Yeah, I agree. It doesn’t have to be an enormous lift. You can really just start with where you are and do what you can do and go from there.

And yeah, I think people get really caught up in the tools coming out of the private sector in particular, this whole, “we need a data warehouse, we need a data lake.” You might.

“We need an integrative tool, we need all kinds of tools. We need to use AI. We need to use …” You just, you don’t know what you need if you don’t have a data strategy.

You have no idea what the next step is going to be until you sit down and put it together.

But you probably don’t (need an expensive new system.) Or it’s the same as “I want to take all of our data and put it in one place.” It’s that same dilemma of thinking that the technology, especially new cutting-edge technology, is going to solve your problems.

Kyle Haines: Yeah, and I can’t believe we got 30 minutes into this conversation and AI just came up. I mean, we’re probably going to lose a ton of credibility because we haven’t thrown around AI all over the place, since that seems to be what everyone’s talking about these days.

Data Warehouse

I actually do want to go to AI, but I was thinking for folks listening who don’t know the difference between a data warehouse and a data lake. Is there an easy way to explain it? I have a way that I might explain it, but I think you’re closer to it. I’m curious how you would define the difference between those and maybe talk a little bit more about knowing when you need one of those, or you might even know when you need to start having conversations about potentially needing a data lake or a data warehouse.

Taj Carson: Sure. So a data warehouse, it’s a centralized repository of data. Usually, historical data and not one-time data.  When you need to collect data over time, usually from multiple sources you use a data warehouse.

When we’re talking to people, if you only have one data source, we’re going to tell you you probably don’t need a data warehouse. We’re kind of cautious about when we think people need a data warehouse, because once your data is in there, you’ve got to maintain it, or you’ve got to have a vendor partner that maintains it. You have got to think about it.

But a data warehouse is generally built and organized to provide analytics and reports on the data that’s in there. Usually, a data warehouse is hosted on the cloud. I was talking to somebody yesterday that has their own little data warehouse on premises, but that’s very uncommon these days.

You’ve got three big providers. You’ve got Amazon, Microsoft/Azure, and then the Google cloud platform. Those tend to be the three.

And the reason those have become so popular is because they’re really scalable. You can have a small one, you can have a big one. They also support all the maintenance of the hardware, the security. A lot of that backend stuff is easier with those providers. You really need, you would need a lot of data maturity and a deep understanding of your data to build and organize and manage a data warehouse. You can, of course, partner with someone like us to do it, but regardless, you got to have strong data governance practices in order for your data warehouse to have the integrity you need to be able to get the analysis and reports that you need out of it.

Data Lake

The other piece is, people talk about the data lake, or just to really get fancy, they talk about the data lake house. And a data lake is a data warehouse that’s got less structure. So maybe the data you’re storing, it’s not organized, maybe it’s not relational, maybe you have flat files and raw survey results and PDFs and images and videos, and you can just kind of dump them all in there, right? Google Drive kind of operates like a data lake house. It’s a data lake. That’s a good way to think. You just put everything in there, right? It’s not necessarily structured and related.

And so, you know, your data can exist anywhere from totally unstructured to it’s ready to be placed into relational tables. And then you might have a hybrid, which is a data lake house. The lake is flexible in what you can store, but it’s also kind of messy.

And it can be, you know, by definition, kind of unorganized. And so it can turn into a data swamp and things in there can get lost or forgotten. I think asking the question about data lake or data warehouse, usually it’s a data warehouse for our clients, but that is the definition.

And just in case people are interested, we just finished a white paper on how to decide whether or not you need a data warehouse, including a really fun graphic decision tree tool. If people are interested in that, they can get in touch with me, and I will send them that.

Kyle Haines: I love decision trees. This is an aside, but I used a decision tree to help one of our clients make a decision earlier this year, because I was even having a hard time understanding what all of the inputs were. I love decision trees.

Taj Carson: They’re good visual tool.

Kyle Haines: The last one I did just to continue the aside was how to navigate the Salesforce ecosystem before there was nonprofit success pack. And there are all of these vendors who offered fund raising CRMs and including BlackBaud and Round Corner and I’m trying to remember the other two, CauseView. And I’m forgetting the last one. But yeah, I probably spent way too much time on the decision tree making it really pretty.

Taj Carson: You know, as a person who’s done a lot of information visualization, if it’s not, there’s a way to make it pretty that actually is very functional, right? There are some ways in which you visualize information that actually makes it easier to understand. So just because it’s pretty doesn’t mean that the pretty is not helpful.

Artificial Intelligence (AI)

Kyle Haines: That’s I think that’s right. You brought up AI earlier. And just so you know, Taj, my colleagues rip on me all the time because they think that I’m a one-person Harvard Business Review quoting machine and that’s the only source that I ever have. But for some reason, their stuff resonates with me.

I’ve certainly seen this in other places that even in the for-profit commercial sector that many CIOs, and I think that the one statistic was more than 70% of CIOs in the commercial sector said before they could even get to AI, they had huge data problems. Taj is nodding strongly to that idea.

Taj Carson: Yeah, if the commercial sector is saying that, then with all the resources that they have towards things like data and technology, you can imagine what’s going on in the non-profit sector. Yeah, I think that’s absolutely what’s going on.

Everybody’s talking about AI, and everybody’s saying our products have AI. But I think most organizations would be, I think the term is way out ahead of their skis on this, because they usually don’t have the amount of data, the data quality and the data governance that they need in order to take advantage of generative AI that would say, we’re just going to dump everything in here and it’s going to tell us everything we need to know. Because the AI isn’t that good at meaning making quite yet, right?

I think it really depends on what you mean, because we also are talking about AI and it’s this big subject. If you’re talking about like prediction value, analysis, then yeah, I would say most organizations don’t have the governance or the quality. And sometimes they don’t have the amount of data that would benefit.

Now there’s other ways in which people are using AI. It’s built into different software tools and that can be really helpful for making your work easier. When you’re writing, when you’re doing simple tasks, I think machine learning has been used for a while, but maybe this is a good parallel, right?

For machine learning, if we want to process data using machine learning, which is a step down from AI, then the data needs to be coming in in the same “wrong” way each time, right? If I want to use machine learning, if I want to use Python to clean your data, it’s got to be that the transformation is the same every time.

So that’s a really good example.

If machine learning is a challenge because the data is messy enough and inconsistent enough or the quality is poor, it’s missing, then it’s really going to be a challenge for AI.

AI, Cybersecurity Concerns, and Sensitive Data

The other thing that people are talking about is if you’re using AI to analyze your data or to do anything with your data, it’s sort of a black box, right? You put the data in and then the thing happens, and you don’t know what other sources of information the AI is using to work with your data, and you don’t necessarily know how the AI is using your data in its work with other data sources.

I would say AI is a definite no if you’ve got sensitive data because it’s proprietary, we don’t really know how it operates.

Get Your Data House In Order

I think the best approach I’ve seen is when I’ve heard a couple of organizations saying, “We want to start to get our house in order because AI is coming.”

Will you ever be able to use AI? Who knows? But I think that’s a great approach, right? Because there’s no downside to getting your data house in order.

If you’re getting your strategy, your governance, your data quality, all that stuff in order it will pay off now in so many ways, as you referred to with the data strategy work with the organization you were mentioning.

My advice would be if there’s some ways in which AI can make your day-to-day life and your job a little bit easier without putting your data into that system, by all means, take advantage of it. Just get your data house in order, and when the time comes, you’ll be ready.

Kyle Haines: I trade and work in metaphors and analogies, and what comes to mind is, even if AI is 40 years off for you, and here’s the comparison. It’s the comparison to retirement. You should start saving today. You should start doing the things you need to do today. If you’re a young person, even if you can put $50 a month into a retirement account, that’s a start.

And I heard you say, even if you never see the value or there is no value in tapping into AI, all of this work is going to have value.

And my metaphor is falling apart a little bit, but this is akin to doing the work early on so that later you can tap into it, much like retirement. And maybe you’ll never need retirement. Maybe your rich uncle or aunt will leave you millions and millions of dollars.

The work you do around data today, it’s going to pay dividends, irrespective of whether you have enough data to do something like machine learning. It’s just inherently valuable.

But I think that for organizations, obviously, that are thinking about AI, and hopefully most are, this is a necessary first step.

Taj Carson: Yeah, it absolutely is. And again, I think you just have to watch out for the sales pitches, because people are going to say, it has AI, it’s going to do everything. And hopefully people are figuring out by now that it’ll (only) do some things.

Technology, it’s a tool, right? It’s like that hammer is not going to build your house, you need an architect. So yeah, absolutely.

And I think for some reason, I’ll give you another terrible analogy maybe. But it reminds me of biohackers who are like, “I’m going to do this $10,000 experimental stem cell treatment.” And you’re like, how about you just go for a walk, right? But they’re not doing the things that are easy and affordable, and obviously we know work, right? It’s kind of like that. It’s like, just go for a walk.

Just start now doing the stuff that you know works, that you know you need to do. And that’s a precursor to anything that might be more advanced.

Kyle Haines: Yep. I really hope that in post-production editing, somebody can take that metaphor that I did, untangle it, reassemble it so that it actually makes sense, because I’m not sure it totally landed. I think for many organizations, this is a transformation in thinking about data first, rather than technology first, and that there’s so much important work to be done around data. 

That doing that, and by doing that work, I should say, it’s going to unlock the potential of that technology. And then maybe in some cases, it de-emphasizes the importance of that technology. It becomes much more about data.

Taj Carson: I mean, it’s very similar to our earlier conversation about choosing a data management system. It’s very similar, right? If people think, “I’m just going to choose this data management system, Salesforce or whatever, and it’s going to fix everything, but actually, I don’t have any half decent data to put in there, and I don’t know where anything is or what it means or what its value is.” And that’s not going to solve your problem.

AI is going to be the same, right? It’s that same “technology is not going to solve your problem” answer.

Software Selection and Data Strategy

We really like collaborating with people who do software selection for nonprofits, because they often encounter this, where they’re trying to work with the client around software selection, and the consultant understands the client has a data problem. They’ve got a data strategy problem. And so we love to let collaborating with other consultants to handle what software does what and is best fit.

And then at the same time, what we’re saying is, “how does the software fit into your data ecosystem?” to help the client get the right fit for that and not just focus on the technology.

Kyle Haines: Build does those selections and Taj, something really cool is that when people reach out to us, in their introductory email, they’re (now) leading with, we have a data problem. So that’s why I was so excited about today’s conversation. I think the sector is beginning to realize, technology advances data, but we have a data problem first. And I’m tired of lurching from technology to technology.

Taj Carson: Well, if you think of it, it hasn’t been that long, and maybe “that long” for me isn’t long because I’ve been doing this for 21 years. But it hasn’t been that long that there have been solid data management systems for non-profits, right? If you think back 15 years ago, what did they have?

You could build your own SQL database and use Excel or have somebody custom build things for you. But that market of software like Social Solutions and Salesforce, well, Salesforce has been around, but Salesforce for nonprofits, Raiser’s Edge, BlackBaud, I could go on and on, right? There are so many offerings now.

And I think we might just now be at a point where they’ve been around for enough time that people have had the chance to lurch from one to the other and get to a space where they understand “This isn’t actually going to solve my problem. I’ve really got a data problem.”

Kyle Haines: Yeah. And the stakes are enormously high because I feel that fatigue. I think that’s the word I’d use. There’s this fatigue with the promises of technology.

You hear leaders talking more and more, at least in our conversations about this, “Our staff are complaining about this technology, but we just put it in five years ago and they’re saying the same thing that they did five years ago, what’s going on here? Are we just making the wrong technology choices?”

And our experience at Build is that it’s so infrequently that they made the wrong technology choice. Something else wasn’t in place. And often it’s the data strategy piece that you talk about.

Taj Carson: People love to hate the technology tool, right? Like they love to say, “It’s the software, it just doesn’t work. It just doesn’t do the thing.”

And we’re always a bit skeptical of that because, did people get the data governance training they needed? Do we tell people how to enter the data? Did we set up the fields to be validated properly for our data model? Do people get repeat training to understand how to put data in and get data out and what goes in and what doesn’t?

We do always look closely at that to see. And sometimes it’s the configuration. Did you configure it properly for your use case? It’s a good place to start by saying it might not be the technology.

Are You Able to Answer Data Questions of the Future?

Kyle Haines: Yeah. The last thing, you know, I wanted to share something that a client said that I really wish I’d come up with myself because I thought it was a really articulate way about talking about data.

We were doing a selection for them, and we were in a meeting, and the topic of reports and dashboards came up. And people went around the room and talked about the dashboards and the reports that they needed.

And their leader said, “I think this is a valuable conversation, but reports and dashboards oftentimes just confirm what we already know. Data answers questions. And I’m looking for tools that allow me to ask questions in the future that I don’t know are going to be questions that I have today.”

I thought that was really a profound way of thinking about the value of starting to think about a data lake or a data warehouse.

Some of this is about capturing data, I think, to answer the questions of tomorrow so that you’re prepared to answer the questions of tomorrow.

And I’m wondering what your thoughts are on that idea.

Taj Carson: So, my first thought is we have a little joke in-house about people who say, “I need a dashboard,” right? Because people come to us all the time and they say, “I need a dashboard.” And they never just need a dashboard.

We always go in, and we say, okay, let’s get out the Power BI, let’s get out the Tableau. And they don’t have what comes before the visualization, right? They say, “I need a dashboard,” because if they have this idea, that the Power BI or Tableau is just going to magically give them the answers they need, even if they didn’t think through what data they need to answer what questions or how it’s stored, or if it’s good quality or if there’s enough of it.

What data are you collecting and why?

First there’s that, right? You have to really be thoughtful about, are you collecting what you need to collect? And is it in good enough shape for you to answer the questions?

If you’ve done that, and then you’ve got it connected to a BI tool, which usually there’s some piece of that first part missing, then you can get answers to questions.

But this whole idea of, “we already know the answers,” I’m not so sure about that, right? Because even if you get what you expect, if you see what you expect to see, it just means things are going the way that you expect them to go, which is not a bad thing.

Data Swamp vs Thoughtful Data Lake

“We want to answer questions in the future that we haven’t even thought about asking yet.” Well, that’s when you get into a data swamp where people start by saying, “well, let’s collect that just in case. Let’s store that just in case. Let’s collect,” right? I think you’re going to need to be really cautious about, “I might want this.” It’s data hoarding, right? “I might need this someday. I might want to ask that question. I might want to know that thing.”

That can get very, very expensive.

But if you’re talking about longitudinal analysis, if what they’re saying is, “you’re telling me what I know because you’re just giving me this quarter. And what I want to do is two years from now, be able to look back.”

That’s an issue of longitudinal data. That depends on your software. Warehouses are really good for storing, they’re really good for helping with longitudinal analysis.

If what you’re saying is, “we know what’s important, but we want to make sure we can look at it longitudinally historically over time,” then that can be a really good case for a warehouse.

But just be careful of thinking, “we’re going to collect everything just in case we might want to ask that question” and be careful of thinking that a BI tool is going to solve all of your problems, even if you don’t have everything. 80% of the work of a BI tool comes before the data even hits the BI tool.

To Dashboard or Not to Dashboard?

Kyle Haines: Yeah. I think I got really geeked out on Power BI years ago, and it is a very cool tool, but then I realized, you got to have something to feed it. You can’t visualize something that you don’t have the data to support it.

And maybe talk about that client. It was a museum actually, and he used this specific example. “I don’t need a dashboard to show me that attendance was down. I walk through the museum every day. Like I can see it. It’s just confirmatory. And I can probably extrapolate that museum store sales are going to be down, and the restaurant sales are going to be down, and that membership sales were down that day,” which I thought it was a funny way to think of it.

I think in this example, he was maybe thinking about the early stages of a data strategy. “What do we need to collect about people who participate in our educational programs? Because someday I might want to market to them in a different way and understand how that marketing brings them back into educational programs or brings them back to the museum.” I think what he was intending when he was saying, “I might not need a dashboard for that today, but I might have a question about that in the future.”

Taj Carson: I think you can still imagine a way in which you could construct clear KPIs for that, right? And maybe what he’s trying to get at is the why. Why did people come? Why did they come back? What did they like? But those could be constructed in clear KPIs that you might want to collect over time and use.

I’m very suspicious of the, “I’ve been walking around the museum, so I know what’s going on,” because of confirmatory bias and about a million other neuroscience-based principles. I mean, this is the reason social scientists are always doing studies on things that sound dumb. Often we find that, “oh, it isn’t actually what it seems to be or what you think it will be,” or the relationships are not as expected.

The other thing is, sure, maybe that museum director can walk around and get a sense of it, but what happens when that person leaves, right? You’re still going to need that dashboard to tell you when attendance is down.

And somebody else in the organization also might need that rather than having to go ask the museum director every time how’s attendance going?

Kyle Haines: All very, very fair points.

Taj, I think I could go on for hours and hours asking you questions because your perspective is so interesting, and I really value your experience in this.

Kyle Haines: Maybe we were separated at birth. I mean, who would do a non-color-coded diagram, Taj? I mean, they’ve got to have colors in them and arrows and things like that. That’s outrageous.

Taj Carson: Nobody that works here, that’s for sure.

Kyle Haines: A quick question about something you just said. You talked about two important things.

One is ranking the quality of the data and the importance of the data.

Is there any sequencing in that? Because I would assume that there is a lot of meat on the bone around the conversation around what’s important or not, and that that could be an area of disagreement within the organization. That Taj thinks this is incredibly important data, and Kyle says, I don’t see how that data is valuable at all.

It seems like there is a big facilitative effort along those lines. Has that been your experience? Is that right?

Shadow Data

Taj Carson: It can be, right? And the thing to realize here is the number is less important than the conversation around how do you get to the number.

That conversation about is it important or is it not important, who is important to, really uncovers what you’re using it for and who is using it. People often don’t realize.

This is one of the reasons somebody will have their little shadow Excel sheet. There’s a lot of good and bad reasons for the shadow data Excel sheets, but sometimes when you have that conversation, somebody will say, “but I need this.” And then you can sort of have that inquiry, and they say, “if I don’t have this, this is what happens.”

And this is where you find out whether they’re thinking, “I need this because I’m afraid the system is going to break,” which is a whole conversation, right?

Versus, “I need this because otherwise I have to go through these 12 steps, so it will take me eight hours to get the data that I need to do this thing.”

There’s not a one size fits all on how to rank it, but the conversation is everything in terms of what you discover about people in your organization and their data and the uses and how people are using it differently and valuing it differently.

Kyle Haines: Yeah. Yeah, I love those conversations because I think that they unlock so much and they begin to create a shared understanding of what’s important, and perhaps they uncover areas where people are locked in on something that is truly important that nobody else knew about. Or they’re locked in on something that nobody else sees as high value or important, and it gives them the ability to let go of something that is not truly that meaningful or important.

Taj Carson: Or the ability for someone to pry it from their hands, which sometimes has to happen too.

Kyle Haines: Yep.

Taj Carson: I’ve seen it multiple times where somebody is keeping a spreadsheet because they don’t trust the system. And that’s an interesting organizational culture issue; that’s a leadership issue. It can be a technology issue.

Did a thing happen and you lost everything? Is there a way we can reconfigure or do something with the system to make it meet your needs, so you don’t have to do that extra work? That’s just one example, but yeah.

Kyle Haines: I can think of an organization, and it came down to data. They didn’t trust the data. And that lack of trust was well justified.

And so we had to figure out how to answer the data problem. And this was just in a single system. This wasn’t even part of a larger data strategy, but that’s probably a topic for another podcast about data quality just within a single system.

But yeah, I mean, it was justified. When people said, I keep this in an Excel spreadsheet, it was not going to be possible to pry it out of their hands because they couldn’t get access to the data they needed. And they needed their version of data to actually make decisions.

Taj Carson: And once you know that, you support them in that. You can give them the support that they need, as opposed to, “what is that random spreadsheet you’re still working on?”

Kyle Haines: Well, Taj, I really appreciate your time today.

Taj Carson: It was really fun. Thank you so much for inviting me.

Kyle Haines: Of course, there’s going to be a part two, three, four, five, six, and seven, because this conversation has my brain totally activated. And I’ve learned a lot. I really appreciate all of your perspective and experience.

And thanks for being on Transforming Nonprofits.

Taj Carson: Yeah, absolutely. Thanks for having me.