MAX Pro Tips 4: A/B Testing

MAX— AppLovin’s mediation platform — helps publishers enhance and optimize their monetization efforts through the largest selection of network bidders in the industry and a suite of robust features. One of those important feature sets we’re discussing in our fourth installment of the MAX Pro Tips series are those that allow developers to perform A/B testing and measure the effect of changes to their monetization strategies and waterfall configurations before rolling out those changes to an entire audience.

The challenge

A large hyper-casual publisher that works with MAX creates free-to-play games in collaboration with hundreds of studios worldwide. Their apps typically use short, session-based gameplay, with users playing for minutes — or even seconds — at a time.

With such a small window to monetize, they track impression activity down to the user level to maintain as much granular control over revenue growth and user retention as possible. 

Their team wanted tools to test new ad partners and waterfall configurations to understand the impact to their monetization and validate that any changes to their stack yields incremental gains based on their goals. They need to roll out optimizations quickly with confidence to maximize revenue during their games’ short lifecycles.

The approach

In order to learn and make positive optimizations within their waterfall setup, and ensure the best possible returns in their tight monetization windows, they used MAX’s built-in features and functions to A/B test their waterfall configurations and revenue growth strategies.

Here is a breakdown of some of those features and functions.

1. Easy and efficient waterfall management and cloning

A/B testing allowed the publisher to test new waterfall setups for a specific ad unit by dividing the users into a test and control group. Within the test group, they could change a variable, such as adding a new network or network price floor to the waterfall and observe if the change provided incremental lifts in the test group vs. the control.

With MAX’s user-based permissions, studios could be given access to specific games to manage the ad waterfalls within their app portfolio, eliminating the need to expose monetization details to non-relevant studios. 

They also leveraged the Waterfall Cloning feature, allowing them to copy existing waterfall setups to new ad units and customize the setup, saving them the time and effort of manually creating every waterfall from scratch.

Ultimately, the publisher tailored setups to unique needs of the app/format combination that allowed for quick tune-ups and testing of many different waterfall configurations, helping to find which configuration ultimately performed the best based on seasonal changes and other variables that made massive differences to how their collective ad rosters performed.

2. Automating the A/B testing process

MAX’s Ad Unit Management API allows developers to programmatically create and manage their waterfall and ad units’ details, making it easier to automate and optimize their ad strategy.

Using the API, the publisher automated A/B tests to try multiple waterfall optimizations such as different price floors, new bidders, and ad frequency – and promoted tests that drove incremental revenue lifts.

3. User-level data for better UA

When conducting A/B testing with AppLovin MAX, studios can access a variety of reports to help them understand the performance of their campaigns. These reports include user-level data, which provides specific information about the interaction and revenue information associated with users who are exposed to in-app advertising. 

User-level Revenue Data helped the studio identify patterns or trends that weren’t apparent when looking at aggregate data. In conjunction with their other findings from the AB test results, they were able to change and improve their UA campaigns to understand the impact of their monetization on user LTV and churn.

The overall results

Through MAX’s A/B testing capabilities, the popular publisher is testing new partners and waterfall configurations faster than ever before. Additionally, user-level data has enabled them to make quick and highly informed changes to their UA strategy and, ultimately, get the most out of their monetization efforts, even in the smaller interaction increments that their hyper-casual game portfolio affords them.

To get the most out of your own monetization strategies through easy and varied iterations of your waterfall setups and ad configurations, speak to your AppLovin MAX account representative today, or submit a request to our developer support team.