# Transforming from one scale to another

Converting data from one scale to another is such a common task for data scientists. This blog post is beyond sklearn’s capabilities. I have always lacked one specific scaler, so in this post I am writing it myself – ScoreScaler.

## Consolidating data despite different scales

It is very common to compare scores or product ratings. A typical example is a survey that asks for ratings on different scales:

Question Scale Score
How satisfied are you with the product? Between 1 and 5 stars 3 stars
How likely are to recommend the product? Percantage number between 0 and 100 70%
Overall, what grade would you give the customer service? Grade between 1 (best) and 6 (worst) 2

There is a vague feeling that the scores for all three questions are similar. However, when presenting the data, you do not want to burden the audience by mentally translating ratings from one rating to another. How to scale up the data?

You are using a scaler. The sklearn Python module offers a number of different scalers (see examples here). However, they all scale according to the observed data. Theoretical minimums and maximums of different scales are not included. In surveys like the example above, very low scores are often not observed, which makes the scale seem narrower than it really is.

## A new scaler: `score_scale_fun()`

In order to scale according to the full scales, I wrote a new scaler which follows this formula:

Note that the min() and max() in this case does not refer to the observed minima and maxima but instead to the theoretically possible minima and maxima on the respective scales. Translated into Python code this would look like this:

``def score_scale_fun(X, scores_old_min, scores_old_max, scores_new_min=0, scores_new_max=1):      X = scores_new_max - ((scores_new_max - scores_new_min) * (scores_old_max - X) / (scores_old_max - scores_old_min))      return X  ``

Let’s translate the scores of our example to a common scale from zero to five stars using this function. Note that the German grading scale is reversed, meaning that the worst possible score (6) should be seen as the minimum.

``print(score_scale_fun(3, scores_old_min=1, scores_old_max=5, scores_new_min=0, scores_new_max=5))  print(score_scale_fun(70, scores_old_min=0, scores_old_max=    100, scores_new_min=0, scores_new_max=5))  print(score_scale_fun(2, scores_old_min=6, scores_old_max=1, scores_new_min=0, scores_new_max=5))  ``

The results are now directly comparable:

Question Transformed Scale Score
How satisfied are you with the product? Between 0 and 5 stars 2.5 stars
How likely are to recommend the product? Between 0 and 5 stars 3.5 stars
Overall, what grade would you give the customer service? Between 0 and 5 stars 4.0 stars

Suddenly we see that the customer service was rated much better than the product. This was not obvious prior to rescaling.

## Following the `sklearn` API

Note that `score_scale_fun()` accepts array inputs like `numpy` arrays or `pandas` DataFrames. However, it cannot be used as part of a `sklearn` pipeline. For that the function has to be rewritten as a `sklearn` transformer.

``from sklearn.base import BaseEstimator  from sklearn.base import TransformerMixin  class ScoreScaler(BaseEstimator, TransformerMixin):      """Transforms features by scaling each feature to given scoring scale.        This estimator scales and translates each feature individually such      that it acccords with a given range on the training set, e.g. between      zero and one. Without scale arguments, ScoreScaler acts like MinMaxScaler.        Parameters      ----------      scores_old_min : int, float, or 'auto'; default 'auto'          The smallest/worst score on the original scale. If 'auto', the smallest value of          each feature is assumed to be the smallest possible value.        scores_old_max : int, float, or 'auto'; default 'auto'          The highest/best score on the original scale. If 'auto', the greatest value of          each feature is assumed to be the highest possible value.        scores_new_min : int or float; default 0          The smallest/worst score on the transformed scale.        scores_new_max : int or float; default 1          The highest/best score on the transformed scale.        Notes      -----      NaNs are treated as missing values: disregarded in fit, and maintained in      transform.      """        def __init__(self, scores_old_min='auto', scores_old_max='auto', scores_new_min=0, scores_new_max=1):          self.scores_old_min = scores_old_min          self.scores_old_max = scores_old_max          self.scores_new_min = scores_new_min          self.scores_new_max = scores_new_max        def fit(self, X, y=None):          """Compute the minimum and maximum to be used for later scaling, if no score range is given.            Parameters          ----------          X : array-like, shape [n_samples, n_features]              The data used to compute the per-feature minimum and maximum              used for later scaling along the features axis.          """            if self.scores_old_min == 'auto':              self.scores_old_min_ = X.min()          else:              self.scores_old_min_ = self.scores_old_min            if self.scores_old_max == 'auto':              self.scores_old_max_ = X.max()          else:              self.scores_old_max_ = self.scores_old_max          return self        def transform(self, X):          """Scaling features of X according to scale settings.            Parameters          ----------          X : array-like, shape [n_samples, n_features]              Input data that will be transformed.          """            X = self.scores_new_max - ((self.scores_new_max - self.scores_new_min) *                                     (self.scores_old_max_ - X) / (self.scores_old_max_ - self.scores_old_min_))          return X        def inverse_transform(self, X):          """Undo the scaling of X according to scale settings.            Parameters          ----------          X : array-like, shape [n_samples, n_features]              Input data that will be transformed.          """            X = self.scores_old_max_ - ((self.scores_old_max_ - self.scores_old_min_) *                                     (self.scores_new_max - X) / (self.scores_new_max - self.scores_new_min))          return X  ``

Such a transformer requires at first an initialisation followed by the use of the object’s `scaler.fit_transform()` method.

``scaler = ScoreScaler(scores_old_min=1, scores_old_max=5, scores_new_min=0, scores_new_max=5)  scaler.fit_transform(X)  ``

In case, the old scale’s minima and maxima are not available, the `ScoreScaler` just uses the minima and maxima observed in the data.

## Caution

Using the `ScoreScaler` assumes that the data are on different scales but follow the same distribution. When it comes to different traditions of school grades this assumption is wrong (see previous blog bost).

Still, if you have no idea about the distribution of the data and need to transform it, as in the case of a small survey, then the `ScoreScaler` comes in handy. I for one have used it many times.

The complete code to recreate the analyses and plots of this blog post can, as always, be found on github here.