Factoring in Multiple Perspectives When Measuring Success

May 05, 2021

by Jay Ro, VP, Experience Analytics, RAPP LA

Perspectives: We all have (at least) one; we all need (a lot) more. Have you noticed that there is no single way to define and measure "success?" It takes a comprehensive understanding, and oftentimes multiple points of view, to determine the effectiveness and success of our marketing efforts.

Contextual understanding brought to performance analysis is paramount in establishing deeper measurement and answering questions such as:

• Did I perform just as (or better than) expected?
• How about when compared to the last campaign? Or to last season or year?
• How did I perform against my competitors?

Part of the Story

Marketing strategies, tactics, and campaigns are all built with specific objectives in mind, many of which are unique to their purpose and place within the marketing funnel. However, the increased availability of data-supported marketing metrics — from upper-funnel awareness to lower-funnel conversions — has created challenges in measuring “success.”

For one, a lack of data can hinder primary KPI measurement. Lower-funnel conversion volumes can be insufficient for robust measurement, whereas upper-funnel engagement metrics are plentiful. In many instances, upper-funnel engagement metrics are summarily dismissed because they don't directly apply to the bottom line, but there certainly is value to be found in tying upper-funnel actions to lower-funnel conversions.

Additionally, getting ahead of data and tech capabilities can be an issue. Big data and robust martech stacks can do a lot of things, but they can’t do everything. What often happens is that KPIs and benchmarks are proposed and approved without a proper understanding of present data and tech capabilities and limitations.

Increased confusion, crunched deadlines, and misaligned expectations are the end results. Besides, metrics alone don't tell the whole story. There need to be valid benchmarks by which to measure and evaluate performance to deepen contextual understanding.

Differentiated Benchmark Perspectives Pay Off

Evaluating success against a single POV can create skewed data or inherent biases — a big mistake as marketers are faced with decisions that are becoming more and more multifaceted. Year-over-year growth, category share, media optimizations, and plenty more all play a role in underlying performance success.

Benchmarks across differing comparison frames can be effective in building a comprehensive understanding by providing multiple data points that can confirm and triangulate measurement.

For our University of Phoenix clients, RAPP has been successful in meeting these challenges through the combined efforts of various Marketing Sciences disciplines. Our Analytics Enablement, Business Intelligence, Data Science, and Competitive Marketing Intelligence teams have all been instrumental in building a comprehensive RAPP offering that enables our clients to view and compare their performance against a number of benchmark perspectives. This allows them to then measure success according to the various ways they need to answer and inform their decisions, big or small.

Key benchmark options include:

• Against Expected.Predicted (or projected) benchmark values for KPIs are generally derived from robust prediction models that take into account various factors that could impact performance (including external). These factors could include fluctuations in market pricing for media, economic indicators, significant events/milestones, or even demographic variables. The models themselves can also vary in complexity, but model robustness is usually dictated by data availability or analytics capabilities.
• Against Previous/Prior. These benchmark values can range in time frames (week-over-week, month-over-month, year-over-year, or prior campaign) but can be effective in showing growth/declines in performance. Verticals in which seasonality plays a huge role in annual performance (e.g., holiday sales, back to school) would opt for this POV as well because these milestone events may prove complicated for a predictive model to account for. These would require historical data, which normally is not an issue for most brands to source from internal sources. However, major strategy shifts (e.g., redefining funnel groups, changes to targeted audiences) and data-set changes (including new data sources introduced) will require accounting for when establishing these benchmarks.
• Against Competitors.Understanding how performance stacks up against other competitors within the space can provide insights into category share shift and how external factors are affecting other players within the space. This can be particularly valuable during heightened seasons of events shaping economic outlook and population behaviors (e.g., COVID-19, recessions). Competitive data can be difficult to come by but usually can be sourced by third-party vendors who are able to leverage their own curated panel audiences.

In order to enable this sort of multi-view approach toward measurement benchmarks, analytic capabilities will need to be enabled by ready access to integrated data in as streamlined a way as possible. This has allowed both RAPP and UOPX analysts to pull data across a variety of cuts (time frames, audience segments) as well as allowing our RAPP data scientists to build robust predictive models that can produce reliable performance projections.

The primary challenges in standing up the supporting infrastructure has been around the level of data integration possible across varied media sources and the manner and cadence on how quickly data can be refreshed. While an extensive amount of work, generally a well-planned, robust, and sophisticated build on the back end will pay off in flexibility and nimbleness in the front-end reporting.

Such advances in automation and data integration (such as the way RAPP has done for our clients) are enabling quicker, more comprehensive performance reporting and measurement framed through multiple perspectives. By leveraging these multiple POVs, marketers can make better-informed decisions supported beyond just one way of looking at things.