Reframing : Are you solving the right problems ?

ML Design Pattern — Problem representation

“If a problem can’t be solved within the frame it was conceived, the solution lies in reframing the problem.” — Brian McGreevy, Hemlock Grove

In this article, we will explore the reframing machine learning design pattern, what it is, how it works, and why it is important.

We will also give two examples of real-world applications for reframing and go over the best ways to include reframing into your own machine learning projects.

Principle :

The Reframing Design Pattern solves the challenge of posing an intuitive problem with a changed contextual output. Here, we change the representation of the output of the problem.

For example, an intuitive regression problem can be reframed into a classification problem and vice versa.

What problem the Reframing design pattern is meant to solve ?

In most real-world scenarios, the problem is not well-defined, and the data may be noisy or ambiguous and can be difficult to solve using traditional machine learning algorithms. In most real cases we can’t rely on well-defined inputs, outputs, and assumptions about the nature of the problem.

Identifying the underlying structure or patterns in the data: Reframing helps to solve this problem by changing the way we think about the problem. It involves breaking down the problem into smaller, more manageable pieces (See Example).

Identify the key features or signals in the data : Reframing is also particularly useful when dealing with problems that involve a high degree of uncertainty .

How to Implement the reframing design pattern?

Example #1 : Predicting the amount of rainfall in a given location — Google.

We want to predict the amount of rainfall in a given location in a certain timeframe. Now, this appears to be a straightforward time series forecasting problem. Alternatively, this can also be defined as a regression problem as the label (amount of rainfall) is a real number.

The key issue here is that amount of rainfall follows a probabilistic distribution. A regression model is limited to predict only a single number and the chances of getting it right are feeble.

So rather than see the problem as a regression, we can reframe our objective as a classification problem. The model then will return the probability of receiving rainfall in a certain range of amounts as shown below:

This method of modeling a distribution is advantageous since precipitation does not exhibit a normal distribution and instead follows a Tweedie distribution , which allows for a prevalence of points at zero.

The same approach was adopted in a Google research paper in which they documented a classification model to predict the precipitation rate in a given location.

When to Implement the reframing design pattern?

When is it beneficial to model a distribution for a regression task?

It’s beneficial when :

Variable does not exhibit a typical bell curve with a preponderance of points at zero (maybe a tweedie distribution) (See Example) :

When the distribution is bimodal (a distribution with two peaks, 2 modes ) :

Or even when the distribution is normal but with a large variance :

In general, the wider the bell curve, and the more the width of the curve varies at different values of inputs, the more important it is to capture uncertainty and the stronger the case for re-framing the regression problem as a classification one.

Different situations when re-framing a problem would be helpful

Accuracy/Precision trade-off :

For a sharp density function, stick with a regression model else go with classification.

The precision of the regression is indicated by the sharpness of the probability density function for a fixed set of input values.

We can get the same plot on the the left by bucketizing the feature that was in plot on the right.

Large variance → Classification → Increase in Accuracy

Small Variance → Regression → Increase in Precision

Restricting the prediction range :

A lot of times the prediction range is a real-valued variable. Simply using a dense layer with one neuron may throw the prediction output beyond the acceptable/interpretable range.

Reframing can come to the rescue here.

To limit the prediction range, make the activation function of the last-but-one layer a sigmoid function (often associated with classification) so that it is in the range [0, 1] and have the last layer scale these values to the desired range:

MIN_Y = 3
MAX_Y = 20
input_size = 10
inputs = keras.layers.Input(shape=(input_size,))
h1 = keras.layers.Dense(20, 'relu')(inputs)
h2 = keras.layers.Dense(1, 'sigmoid')(h1) # a sigmoid layer
output = keras.layers.Lambda(lambda y: (y*(MAX_Y-MIN_Y) + MIN_Y))(h2) # a custom layer for scaling
model = keras.Model(inputs, output)

Since the output is a sigmoid, the model will never actually hit the minimum or maximum of the range, just get close to it.

Avoid label bias :

If we take video recommendation systems for example, framing the objective as a classification problem (viewer will click or not) can lead to recommendation system prioritizing clickbait.

In this case, it is better to reframe the objective into a regression problem predicting the fraction of the video that will be watched, or the video watch time or even predict the likelihood that a user will watch at least half the video clip.

Conclusion:

In conclusion, the reframing machine learning design pattern offers a fresh perspective on how we can approach the design of machine learning models.

By focusing on the needs of the end-users and their desired outcomes, rather than just optimizing for accuracy, we can create more meaningful and impactful solutions.

The use of this pattern can lead to better decision-making, improved user experiences, and ultimately, more successful machine-learning projects.

Reframing : Are you solving the right problems ?