Understanding units in mmm.fit_result

[smusch2]

Looking at the example notebook - https://www.pymc-marketing.io/en/stable/notebooks/mmm/mmm_example.html#model-diagnostics

In my_model_config, there was a "LogNormal" distribution applied to beta_channel .

In some cases, I can see an option to revert results back to "original_scale".

However, I believe that the provided az.summary() code produces results still in the "LogNormal".

1. Is this correct?
2. If so, how do I convert az.summary() back to original units?

[smusch2]

Attempting to solve my own question. Since there was a "LogNormal" transformation applied to betas, the code below is meant to convert beta coefficients back to their original scale.

# "az.summary" provided by pymc, then converted to dataframe
summary_table = pd.DataFrame(az.summary(
    data=mmm.fit_result,
    var_names=[
        "intercept",
        "likelihood_sigma",
        "beta_channel",
        "alpha",    # Carryover
        "lam",      # Saturation
        "gamma_control",
        "gamma_fourier",
    ],)).reset_index()

# Only updating betas
beta_channel_rows = summary_table[summary_table['index'].str.startswith('beta_channel')]

# Inverse of "LogNormal" transformation applied to betas, also accounting for "prior_sigma" defined earlier
def back_transform_dataframe(df, mu_column_name, sd_column_name):
    def original_scale_params(row):
        mu_estimate = row[mu_column_name]
        sigma = row[sd_column_name]
        original_mean = np.exp(mu_estimate + sigma**2 / 2)
        original_var = (np.exp(sigma**2) - 1) * np.exp(2 * mu_estimate + sigma**2)
        original_sd = np.sqrt(original_var)
        return pd.Series([original_mean, original_sd], index=['original_mean', 'original_sd'])
    
    # Creates 2 new columns (1) original mean and (2) original standard deviation
    df[['mean_original_units', 'sd_original_units']] = df.apply(original_scale_params, axis=1)
    return df

back_transform_dataframe(df = beta_channel_rows, mu_column_name = 'mean', sd_column_name = 'sd')

[wd60622]

Hi @smusch2

The use of LogNormal prior shouldn't impact the transformation that is being used back to original units as the prior just reflects the likely values of the parameters.
However, the preprocessing transformations that are being done internally before the model fit will definitely affect the units.

The transformations are described here. In summary:

• y is divided by max value of training
• each channel is divided by max value of training
• each control is standardized with mean and std of training. i.e (x - x_mean) / x_std

The original_scale argument is often used to get the target back to the original space (i.e multiply by the max_value of training target)

As for the beta_channel parameters, they will affected by the scaling of the channel spends

Hope this gives some context. Let me know if you have any other follow ups

[smusch2]

@wd60622 - Still using the example provided by pymc, can you confirm that this is correct?

Values

• y's max value = 8.31
• x1's max value = 0.9967
• x1's summary "mean" (beta_channel[x1]) = 0.361

Equation to undo "MinMax" scaling

• x1_impact_per_dollar = (1 / max of x1) * beta_channel[x1] * max of y
• x1_impact_per_dollar = (1 / 0.9967) * 0.361 * 8.31
• x1_impact_per_dollar = 3.01

Translation: All else equal, a marginal $1 spend in x1 corresponds to a marginal $3.01 in y.

[smusch2]

Hi @wd60622 - I think I figured it out, but it'd be great if you could confirm.

mmm = DelayedSaturatedMMM(
    model_config = my_model_config,
    sampler_config = my_sampler_config,
    date_column="date_week",
    channel_columns=["x1", "x2"],
    control_columns=[
        "event_1",
        "event_2",
        "t",
    ],
    adstock_max_lag=8,
    yearly_seasonality=2,
)

with pm.Model() as model:
    mmm.fit(X=X, y=y, target_accept=0.95, chains=4, random_seed=rng)
    idata_1 = pm.sample(
        target_accept=0.95,
        draws=2000,
        chains=4,
        nuts_sampler="numpyro",
        random_seed=rng
    )

# Inverse of MinMax
beta_channel = np.array([idata_1["posterior"]["beta_channel"].values])  # Beta in scaled form
target_scale = np.array([data['y'].max()])                       # Max value of target variable
channel_scale = np.array([data[["x1", "x2"]].max(axis=0)])                # Max values of media variables

beta_original = beta_channel * (target_scale / channel_scale)

[wd60622]

I think the beta would be multiplied by both target_scale and channel_scale. What is the rationale behind dividing by channel_scale?

[wd60622]

@cetagostini might have a better take on it

[cetagostini]

Hey @smusch2, I'm going to mention a couple of things to make sure we're aligned!

What is beta_channel?
Given our logistic function (the only one currently available natively in PyMC-MMM models) which is expressed as (1 - pt.exp(-lam * x)) / (1 + pt.exp(-lam * x)). Documentation here

Given any value of lam, the function will always have a saturation equal to 1. The parameter λ controls the steepness of this curve, affecting how quickly 𝑓(𝑥) approaches its upper limit of 1 as 𝑥 increases. As 𝑥 → ∞, this part of the function asymptotically approaches 1.

Considering this behavior, multiplying the function by 𝛽 scales the range of the function by β. If 𝛽 > 0, the function's output now ranges from 0 to 𝛽 as 𝑥 goes from 0 to ∞. This transformation does not alter the inherent shape or the approaching behavior at infinity; it merely shifts the upper limit of the function’s output to 𝛽 instead of 1.

What this means?
beta_channel lives on the Y axis because it represents the maximum contribution. Therefore, if you want to add a prior over beta you must scale it based on the maximum of your target.

Let's consider an example involving two channels, X1 and X2. Suppose we know that the maximum contribution value of channel X1 is 100, and that of channel X2 is 450. Based on this information, we can calculate the beta channel prior for each channel. For X1, the beta channel prior would be 100 divided by the absolute maximum value of the target, while for X2, it would be 450 divided by the absolute maximum value of the target. Once we have determined these values, we can use them in the model config to inform our analysis.

If you want to return the model parameter to its original form, multiply beta_channel by target_scale and you will get the beta value in real scale.

Our model is a linear regression in principle but uses non-linear functions. This means that the form beta * logistic(x) cannot be interpreted as beta * X. This also means that beta_channel is not ROAS and cannot be inform by ROAS.

Soon we will have other parameterizations on the saturation functions that can be informed by ROAS. But now, the best thing you can do given a prior ROAS is to evaluate the parameters of the logistic function to determine a beta-lam pair that delivers said ROAS after the contribution is obtained. Ways to do it?

• Introduction of a second likelihood, creates a likelihood that receives a ROAS prior and evaluates the difference of the ROAS given by the model and the prior to be zero. This should guide the model to said ROAS, and therefore, to its parameters as well.
• Use the pm.Potential function and generate a penalty term when your prior ROAS and the ROAS estimated by the model diverge.
• Given the ROAS you want to have, use a function in scipy to evaluate the best combination of lam/beta that generates a saturation function with said Avg. ROAS, you can use minimize from scipy.

Let me know if this helps!