How the Art Institute Uses Data to Predict the Future

Erik Gensler: Andrew, thank you so much for being here. I’m really looking forward to chatting with you.

Andrew Simnick: No, it’s my pleasure. I appreciate the opportunity.

Erik Gensler: I met you at a conference where you shared a presentation where you outlined this amazing model in data visualization tool that you build to predict visitor attendance at the Art Institute, including factors like weather, time of day, day of week, type of exhibition. Can you tell us about that project. What it is, why you built it, and, and what it does?

Andrew Simnick: The attendance modeling project started with a need to understand the basic factors that drive attendance to the Art Institute. What we tried to do was take, an anecdotal understanding of the effects of exhibitions, seasonality of weather, but to really understand how much each of those factors impacted visitation. At the time, we were just starting out path towards using applied data and analytics within the organization. My college and I built a model that tested 200 factors, both internal and external anywhere from exhibition type, to time of year, to weather, to events in Chicago, to the Chicago public schools schedule, tourism, etc. Narrowed it down to 20, and found out that there are different factors that matter in different ways based on two things: 1. Geography and 2. Channel. So tickets versus membership. And while in total it was really messy, there’s lots of rising, falling, year-over-year changes, break it out by geography and channel and apply the only the factors that statistically matter. It wasn’t that hard to break apart and understand. And where we really turned a corner was the idea of data visualization within the museum. We ended up taking the model output, doing our first major, effort in data visualization and converting to a dashboard that was incredibly easy and intuitive for the end user to see when there is a meaningful change in attendance overall, by geography, by channel. And taking all of this very rigorous mathematical modeling and regression analysis, but present it in a way with not much resource investment for staff around the museum to share the same language. And also to make decisions at all move to the end goal of allocating resources effectively across the organization.

Erik Gensler: You found that there were 20 factors that actually impact attendance and you could then slice and dice that by a couple of factors and very reliably predict attendance for any particular day.

Andrew Simnick: We model by the week. And we found that that gives the best balance of a usable output while still having clean enough information to make good decisions.

Erik Gensler: How close do you usually get?

Andrew Simnick: Our first full year of using the model for prediction, we were within 1% of total feet in door, paid, ticketed revenue, and per-cap revenue.

Erik Gensler: That’s incredible. What were some of the major drivers? You listed a few, but what were some of the ones that you found out were not impactful, and what were some of the things that were really impactful?

Andrew Simnick: We found that we rise and fall with tourism to Chicago, and the Art Institute is a wonderful organization, but we’re unlikely to drive an individual to come to the city just for us. So, by monitoring tourist volume to the city, we purchase hotel data and feed that into our model. We’ve done a pretty good job of projecting our market share of tourist, and also being able to normalize tourist visitation to how the city is doing overall. So it is a factor that we consider outside of our control, but meaningful enough where we need to watch it and where external trends can really change our performance, both from an attendance and a financial perspective. Seasonality matters a ton. We see spikes in attendance really tightly packed around the few days after Thanksgiving to the last week in December between Christmas and New Years. So, for anyone thinking about visiting the museum, come outside of those times if you want to get lower attendance, and that showed up as factors in our model. An example of a factor that we did not see come up and it’s ticket pricing. Something we need to think mindfully of as part of our mission, but we have not seen a link between our ticket price and attendance, since we’ve had the model up and running.

Erik Gensler: Once you’ve determined all these different factors that were driving the output, you then had to find a way to get that data into the model. Did that prove tricky?

Andrew Simnick: I wouldn’t say tricky, although my colleague, Matt, may disagree. Our first version the modeling was done very manually. We could extract from our ticketing systems, and external data such as weather, or mapping zip code to geography, that’s relatively straight forward. We had to do some internal build out, data input platforms to just get the information we needed for the modeling, but it was all built in-house using systems that we had without much financial investment. The first push was really brute force, and now we’ve made it a lot easier there’s new products on the market that allow for automation. But when we built this thing it was just a lot of sweat equity to get this thing off the ground.

Erik Gensler: How much of an impact do you see weather has on attendance?

Andrew Simnick: Weather definitely has an impact. Our sweet spot for attendance is when there is slightly unfavorable weather that’s still in line with the season. What I mean is, a rainy day in the summer is great for attendance for the Art Institute. We don’t always like as Chicago residents, but it’s great for the museum. But when we see the weather get to be so off the norm for the time of year, it goes negative. One of the factors that’s come up in our model is, we’ll call the Polar Vortex, when Chicago had a really aggressive cold snap back. I believe in 2014. And that came out as a factor in our model. The weather was so unseasonable that it just prevented people from leaving their homes and leaving their hotels, which is bad for us. It’s a question we’ve had internally of how sophisticated do we need to get around modeling weather. I think for now, we’re keeping it pretty simple and then if something breaks with some very unseasonable weather, we just take that as something outside our control and move on.

Erik Gensler: Museums have their permanent collection. You have major exhibitions, you have often like smaller exhibitions. How do you go about figuring out what kind of exhibition something is gonna be? Because I imagine that that is an input to the model as well.

Andrew Simnick: It is, and it’s a great question. So for us, an encyclopedic museum, we’re committed to a wide, diverse exhibition series. Not all perform the same from an attendance standpoint, but all of them have a reason to be in the portfolio of programming that we put forth. What we’ve done, in terms of the model is using past attendance data from exhibitions for 15 years back from when we initially did the classification. We use a scale, so we define it one, two, three, four, and this is where the art meets the science. Our marketing team will create exhibitions one through four with three and four having a disproportionate lift by geography and by channel, depending on the program. And not everything will hit that level. While it is an art, we’ve been on point the vast majority of the time we’ve probably missed on two exhibitions since the model’s inception, which is pretty good, and not by a long shot. We’ve also had a couple exhibitions over perform, which is always great to see. Again, nothing where it would upset the projection. There’s certain exhibitions, notably our Van Gogh’s Bedroom Show, that, attendance during the run of that was ten standard deviations above base line, which we didn’t know as mathematically possible until we saw it. That’s a situation where we’ve maxed out the model. The performance of that is something we hadn’t seen before, and one, we’re gonna celebrate that success, but two, we’re also gonna understand that that is outside the norm, and not bake that into our annual projections going forward as year-over-year performance.

Erik Gensler: Do you have any quantitative results of this effort?

Andrew Simnick: When we first launched this in 2014, it was right before the Art Institute won the #1 Museum in the World Award on TripAdvisor. And that’s something we took a lot of pride in, but we didn’t quite know what it would do to visitation. The timing was very, fortunate for us, as we had this base line model that could detect changes and performance. And what we saw is the week after the story broke and got a lot of local media coverage, our local ticketed attendance broke positive. It was the only channel that broke positive. We didn’t see tourism volume change, we didn’t see member behavior change, just local ticketed attendance, which the proxy for us for new visitation. We saw that made a bet, an educated bet, that shifting our marketing dollars and message towards that #1 ranking versus focusing on programming, which was a shift for us. No additional dollars but just changing the message. When tourism season came around, we saw the model break positive, not only in local but also in tourist volume. And that carried through, from the early part of spring through November. We can’t prove that it was TripAdvisor, but the changes in the model coincided with the announcement of the ranking and the launch of the marketing campaign. We look at the area of the curve of the model, we estimate, we saw a little over $2 million net increase in our, our admissions revenue during that period in time, which to be able to do that without spending additional dollars or at all deviating from admission is a huge win. And that was also the moment where this modeling approach really became the foundation of how we understand or earn revenue now as our organization.

Erik Gensler: It’s such challenging thing to predict the future and to be able to do it in such a way and be so confident in it is really so valuable. And I’m sure since this $2 million example, there probably been many more where you’ve, found ways to quantify this. Are there other ways you looked at our had wins from the model that you can share?

Andrew Simnick: Recently, we’ve extended the model to link our onsite retail performance to attendance. Effectively have that same normalized base line from which we can run experiments, which has allowed us to really push the envelope in experimentation around the physical retail space. We’ve had tremendous performance through what I call, complexity reductions, so we’ve been able to eliminate about two-thirds of SKUs. So our shop, we’ve reduced our inventory levels a significant amount. We’ve even reduced our storage needs to the point where we’re able to divest a warehouse., and all of this while watching that top line revenue normalize for attendance and seeing that it’s still falling within the bounds of normal variation. It’s really changed the way we think about how we can use data from an operational sense in that it’s helped us with space allocation with retail optimization.

Erik Gensler: This was your first project, right? You’ve worked on some other data projects. I think I remember you talking about analyzing visitor’s routes and dwell time, using Wi-Fi. Could you talk about that a bit?

Andrew Simnick: We’re really interested in how visitors use the galleries and our space in general. Visitor experience means a lot, not just to TripAdvisor. If someone is willing to give us their leisure time, we want to make sure that we provide the best experience possible, and encourage that repeat visit down the road. One of the ways that we always thought about doing this was to observe visitor behavior in the galleries. And this used to take the form of volunteers with clipboards, stopwatches, intercept interviews. What measuring Wi-Fi signal allows us to do, is to estimate the percent visitation overall, as well as dwell time within different galleries in the museum. Which allows us to experiment with different types of programs as well as understand which galleries are getting the most foot traffic, holistically. We don’t measure at the individual level. We just look for broad trends based on the Wi-Fi signal and what’s interesting is we found that small permanent collection installations can significantly, change the visitation patterns we typically see in the museum. Which is a pretty big insight for us, that we’ve been pushing over the past few years.

Erik Gensler: Have you ever tied marketing efforts to visitation times. So for example, if you do a social media campaign talking about a specific work or specific group of works, and then tie it back, is that driving people to specifically seek out those works?

Andrew Simnick: We haven’t specifically tied outreach to visitation time. A permanent collection show called, Flesh, which was a show about Ivan Albright, who has deep ties to Chicago, is well represented in our collection, and his images has always generated a positive response on social media. What we did, marrying social analytics, once the show is installed, there is a significant push on social around the imaged in the show, which are very, very recognizable and sometimes striking in either a positive or negative way, depending on how you respond to the art. What we saw during the run of the show, was that not only was the online response incredibly favorable and higher than normal for a permanent collection rotation, we also saw that dwell time in the gallery itself went up by 25%. And while this is, on it’s own, a great sign that individuals are engaging, these are also the galleries that are the home of American Gothic, Nighthawks, Mary Cassatt. So there are icons in the space and it’s still able to push the dwell time higher than that. So as one of these examples of, we’re gonna try something new, we’re going to look both qualitatively and quantitative of how it works, and we’re gonna make sure that it is purely aligned with our vision. Our collection of what makes the Art Institute great. So just this wonderful example of how it all ties together.

Erik Gensler: The term “big data” is certainly overused. If you’ve ever been to a marketing conference you’re sure to hear it, and I’m curious, what you’re giving us is examples of where you’re actually using big data in useful forms. How do you see big data in the realm of how art institutions should be contextualizing it and using it?

Andrew Simnick: Yeah, I’m not a huge fan of the phrase “big data.” The size doesn’t matter when you’re just getting started the most important thing is to understand what question you’re trying to answer. And this is in any organization, whether it’s arts and culture, a different non-profit, a for-profit. Starting out with the right question is the most critical thing one can do. You don’t necessarily need data to have that right question. For us, the question was “What’s driving attendance?” Then the question comes “What information do we have to test that?” And even before the multivariate regression, it was as simple as taking a printed report, converting the numbers into Excel and running a data table or looking at a bar chart. The desired output is insight that can be applied to the business. Data is the raw material you need get there, but the way I look at it and the way my team looks at it, you just gotta make sure you have the information that’s required in a good-enough format to answer the question you’re trying to solve. So, it’s great to have big data solutions, but it’s more important to have the question right and the ability to put that insight into play. Everything else in the middle. Just use what works.

Erik Gensler: I can imagine organizations that are hearing these amazing case studies around the predictive model, the data flow through the museum and saying “Oh my gosh, this is so beyond the realm of what I could even fathom for my organization right now.” So, what can you offer as a first step to being more data driven and thinking about building some of these models?

Andrew Simnick: We stated five years ago, data was novel. We were ahead of the curve and a lot of the questions we got were around what do you mean by data? Why is it important? What can you do with it? You need to have data to run your business. And one thing that was great about our culture and continues to be great, is there was a willingness to ask the question and to test the question. So the Art Institute’s always been a place of improvement and constant knowledge generation, and that made it a natural fit. For organizations that are just getting started down this path, the first thing is to ensure the culture is ready to put insights into action. If that’s not there, you can invest all the time and money into collecting data, and even structuring questions. The cultures not ready to put things into action, that’s a part that can start or change without any investment. It’s a mindset. But it’s necessary to really move for an organization looking to do these other things.

Erik Gensler: I’m curious the evolution, you say it starts with, with an openness to change and growth mindset which I’m a hundred percent with you. So I’m curious as to understand how things evolved at the Art Institute. Did it start with a higher? Did it start from someone on the leadership team saying we need this? How did the department start? How was it grown? How has it evolved and what is your role within the internal structure of the organization?

Andrew Simnick: I oversee, strategy, financial planning, and operations. And that’s grown over time. In parallel to the growth of our analytics efforts in some ways, when I joined the museum five years ago. I quickly found a colleague Matt Norris, he’s amazing. Who was a believer in using systems to provide data and to answer business centric questions in new ways. And coming from the strategy side when I first joined the museum, I was living in the world of forming questions and I finally met a colleague who could help get the information, and the two of us together just became a true partnership in applying data in new ways and figuring out ways to communicate insight into the organization. What’s interesting is looking at how we’ve expanded and grown our approach. Our team is still lean. We started between the two of us, we’re now up to three FTs, and two of them had been added in the last twelve months, so our team is still very lean. We’ve scaled up our analytics platform, which is still in a home-built, it’s a sequel data warehouse with different packages added to it for external data, for automating preparation, for helping with data visualization reporting. I wouldn’t call any of it state-of-the-art. What’s special about it is we’ve grown the platform to meet our business needs as we take on more questions look at more areas of the organization. But ultimately, that pattern of working together as partners, figuring out the right question figure out the information we need, and to apply insight into the organization. It’s still going strong, and there’s no shortage of questions we can answer. In fact, that’s probably one of the limitations. There’s too many. So we still have to be just as judicious as we were at the beginning. The mindset hasn’t changed. The methodology hasn’t changed. We’ve gotten a few more people engaged, but I think ultimately, we’ve just scaled the platform to address more areas of the business in a culture that has become more and more receptive to using applied data for decision making across the institution.

Erik Gensler: I imagine you’re now getting all sorts of questions institutionally, which means you did a really great job of socializing these results and getting your leadership team bought in. I’m curious how you liaise with the senior leadership of the team and the board, and if they’re behind this?

Andrew Simnick: The senior leadership team is bought in, the board is bought in, and, my teams are bought in. But I think that one of the things that we’ve done is figured out from a communication’s standpoint, and a socialization standpoint, what’s worked and what hasn’t. We haven’t always done this perfectly. I think the common element when we’ve gotten something out from incubation, into practice, is that we’ve communicated, in a way that the audience can understand. Data visualization is amazing for this when done correctly. There is also times when a bullet point is just as, important, if not more useful than data visualization. And we’ve also tried to keep the human elements at play. Applying data in new ways correctly, you’re going to find things that aren’t always the status quo, and you are likely to be challenged. I have phenomenal colleagues that really believe in this organization. Everybody’s committed to improvement, doesn’t mean that we’re still people at work. And challenging the status quo sometimes comes with uncomfortable conversations, and we try to be as open minded and as flexible as possible, while still making sure that whatever insights come out are going to the end goal of improving our performance, making sure we’re all going in the same direction, and turn the Art Institute into the best museum it can be.

Erik Gensler: Do you use web analytics data in your model? Like number of visits to the website?

Andrew Simnick: We currently do not link the web analytics to our attendance model. However, we are preparing to relaunch our home page at artic.edu. The goal is one, to just give a fresher look and our visitors an easier way to navigate our collection, two, to make it much more mobile friendly, and three, to be able to capture data in new ways that we couldn’t do on the previous site’s architecture. So, while it is an area we haven’t explored as much as we’d like, we’re very excited to be putting the structure in place to be able to learn new things about how our visitors engage with us online in a much easier, faster way.

Erik Gensler: We’ve come to your last question and this is your CI to Eye moment, and the question is, if you can broadcast to the executive directors, leadership teams, staff, and board of a thousand art organizations, what advice would you provide to help improve their businesses?

Andrew Simnick: With all of the excitement and energy around applied data and analytics, the most important thing still remains the ability to ask questions and to act on insight. So data is an asset. If used correctly it can provide a tremendous value to an organization, but even more important than the data, smart structured questions upfront and a culture that’s willing to put insight into action in a fast and responsible way. That’s never going to change to matter what trends happen around data information analytics. The start and the end, to me, are gonna remain unchanged, no matter what type of organization you’re in, and that includes museums like ours and whoever else is out there listening.

Erik Gensler: Amazing. thank you so much. This was super interesting.