# getml.pipeline.metrics

Signifies different scoring methods.

##
auc
`module-attribute`

```
auc = _all_metrics[0]
```

Area under the curve - refers to the area under the receiver operating characteristic (ROC) curve.

Used for classification problems.

When handling a classification problem, the ROC curve maps the relationship between two conflicting goals:

On the hand, we want a high *true positive rate*. The true positive
rate, sometimes referred to as *recall*, measures the share of
true positive predictions over all positives:

In other words, we want our classification algorithm to "catch" as many positives as possible.

On the other hand, we also want a low *false positive rate* (FPR). The
false positive rate measures the share of false positives over all
negatives.

In other words, we want as few "false alarms" as possible.

However, unless we have a perfect classifier, these two goals conflict with each other.

The ROC curve maps the TPR against the FPR. We now measure the area under said curve (AUC). A higher AUC implies that the trade-off between TPR and FPR is more beneficial. A perfect model would have an AUC of 1. An AUC of 0.5 implies that the model has no predictive value.

##
accuracy
`module-attribute`

```
accuracy = _all_metrics[1]
```

Accuracy - measures the share of accurate predictions as of total samples in the testing set.

Used for classification problems.

The number of correct predictions depends on the threshold used: For instance, we could interpret all predictions for which the probability is greater than 0.5 as a positive and all others as a negative. But we do not have to use a threshold of 0.5 - we might as well use any other threshold. Which threshold we choose will impact the calculated accuracy.

When calculating the accuracy, the value returned is the
accuracy returned by the *best threshold*.

Even though accuracy is the most intuitive way to measure a classification algorithm, it can also be very misleading when the samples are very skewed. For instance, if only 2% of the samples are positive, a predictor that always predicts negative outcomes will have an accuracy of 98%. This sounds very good to the layman, but the predictor in this example actually has no predictive value.

##
cross_entropy
`module-attribute`

```
cross_entropy = _all_metrics[2]
```

Cross entropy, also referred to as log-loss, is a measure of the likelihood of the classification model.

Used for classification problems.

Mathematically speaking, cross-entropy for a binary classification problem is defined as follows:

where \(p_i\) is the probability of a positive outcome as predicted by the classification algorithm and \(y_i\) is the target value, which is 1 for a positive outcome and 0 otherwise.

There are several ways to justify the use of cross entropy to evaluate
classification algorithms. But the most intuitive way is to think of
it as a measure of *likelihood*. When we have a classification algorithm
that gives us probabilities, we would like to know how likely it is
that we observe a particular state of the world given the probabilities.

We can calculate this likelihood as follows:

(Recall that \(y_i\) can only be 0 or 1.)

If we take the logarithm of the likelihood as defined above, divide by
\(N\) and then multiply by `-1`

(because we want lower to mean
better and 0 to mean perfect), the outcome will be cross entropy.

##
mae
`module-attribute`

```
mae = _all_metrics[3]
```

Mean Absolute Error - measure of distance between two numerical targets.

Used for regression problems.

where \(\mathbf{y}_i\) and \(\mathbf{\hat{y}}_i\) are the target values or prediction respectively for a particular data sample \(i\) (both multidimensional in case of using multiple targets) while \(n\) is the number of samples we consider during the scoring.

##
rmse
`module-attribute`

```
rmse = _all_metrics[4]
```

Root Mean Squared Error - measure of distance between two numerical targets.

Used for regression problems.

where \(\mathbf{y}_i\) and \(\mathbf{\hat{y}}_i\) are the target values or prediction respectively for a particular data sample \(i\) (both multidimensional in case of using multiple targets) while \(n\) is the number of samples we consider during the scoring.

##
rsquared
`module-attribute`

```
rsquared = _all_metrics[5]
```

\(R^{2}\) - squared correlation coefficient between predictions and targets.

Used for regression problems.

\(R^{2}\) is defined as follows:

where \(y_i\) are the true values, \(\hat{y_i}\) are the predictions and \(\bar{...}\) denotes the mean operator.

An \(R^{2}\) of 1 implies perfect correlation between the predictions and the targets and an \(R^{2}\) of 0 implies no correlation at all.