Occupancy - A multivariate time series example¶

Background¶

Our use case is a public domain data set for predicting room occupancy from sensor data. The results achieved using getML outperform all published results on this data set. Note that this is not only a neat use case for machine learning algorithms, but a real-world application with tangible consequences: If room occupancy is known with sufficient certainty, it can be applied to the control systems of a building. Such as system can reduce the energy consumption by up to 50 %.

Introduction to occupancy prediction¶

Usually, getML is considered to be a tool for feature engineering and machine learning on relational data sets. How can we apply it to (multivariate) time series?

The key is a self-join. Instead of creating features by merging and aggregating peripheral tables in a relational data model, for a time-series, we perform the same operations on the population table itself. This results in features like these:

• Aggregations over time, such as the average value of some column for the last 3 days.

• Seasonal effects, such as today is a Wednesday, so let's get the average value for the last four Wednesdays.

• Lag variables, such as get the value of some column from two hours ago.

Using getML's algorithms for relational learning, we can extract all of these features automatically. Having created a flat table of such features, we can then apply state-of-the-art machine learning algorithms, like xgboost. As you will see in this example, this performs better than traditional time series analysis.

The present analysis is based on a public domain time series dataset. It is available in the UC Irvine Machine Learning Repository. The challenge is straightforward: We want to predict whether an office room is occupied at a given moment in time using sensor data. The data is measured about once a minute. Ground-truth occupancy was obtained from time-stamped pictures. The available columns are

• Date, year-month-day hour:minute:second
• Temperature, in Celsius
• Relative Humidity, %
• Light, in Lux
• CO2, in ppm
• Humidity Ratio, Derived quantity from temperature and relative humidity, in kgwater-vapor/kg-air
• Occupancy, 0 or 1, 0 for not occupied, 1 for occupied status

As a reference and benchmark, we use this paper:

Accurate occupancy detection of an office room from light, temperature, humidity and CO2 measurements using statistical learning models. Luis M. Candanedo, Veronique Feldheim. Energy and Buildings. Volume 112, 15 January 2016, Pages 28-39.

The authors apply various artifical neural networks algorithms to the data set at hand and achieved accuracies between 80.324% (batch back algorithm) and 99.061% (limited memory quasi-Newton algorithm).

Analysis¶

Let's get started with the analysis and set-up your session:

1. Loading data¶

The data set can be downloaded directly from GitHub. It is conveniently separated into a training, a validation and a testing set. This allows us to directly benchmark our results against the results of the original paper later.

The training set looks like this:

We also assign roles to each column. To learn more about what roles do and why we need them, check out the official documentation.

2. Predictive modeling¶

We loaded the data, defined the roles, units and the abstract data model. Next, we create a getML pipeline for relational learning.

2.1 getML Pipeline¶

We use a Multirel for generating the features and a simple logistic regression for prediction.

We do not spend much effort on the hyperparameters and largely go with the default values. The only exception is that we add some regularization to the XGBoostClassifiers.

We choose to consider data within the last 15 minutes for creating our features.

.check(...) will be automatically called by .fit(...). But it is always a good idea to call .check(...) separately before fitting, so we still have time for some last-minute fixes.

2.2 Model training¶

We use a routine for automatic hyperparameter optimization to find the best parameters for the predictor:

The logistic regression has one hyperparameter that can be optimized, namely its L2 regularization parameter. We fine-tune this parameter using the validation set. Note that the validation set is different from the testing set on which the final outcome will be evaluated.

2.3 Model evaluation¶

Let's see how well we did by scoring the model on the testing set.

In the original paper, the authors tried several approaches. The best out-of-sample values of all the approaches they tried are the following:

• Accuracy (testing): 0.99061
• AUC (testing): 0.99574

Note that our results outperform the best approach from the original paper, both in terms of accuracy as well as AUC. This demonstrates how powerful getML's relational learning approach is also a powerful tool for time series.

2.4 Studying the features¶

It is always a good idea to study the features the relational learning algorithm has extracted. We can do so in the feature view of the getML monitor. Open the monitor and select the models tab in the sidebar. You will see an overview over the trained pipelines. Select a pipeline to see the most essential summary plots.

If you want to document them inside your notebook, here is how you can do that:

The feature importance is calculated by XGBoost based on the improvement of the optimizing criterium at each split in the decision tree and is normalized to 100%.

We first look at the most important feature. The names returned by feature importances are already sorted, so we can just index them, like this:

Let's check out the second most important feature.

These two features demonstrate the power of the getML feature learning algorithms. It is very unlikely to find these features using manual, trial-and-error based methods. The general structure of features found using such methods might be similar, but you would have had to put in much more effort while getting worse results.

When browsing through the remaining features, you will notice that some are columns directly taken from the original table, such as Light and CO2. But these columns are less correlated and less important than the features generated with the relational model based on self-join and upper time stamps.

Because getML uses a feature learning approach, the concept of feature importances can also be carried over to the individual columns.

As we can see, about 80% of the predictive power comes from the Light column. The second most important column is the Temperature column, we contributes about 7% of the predictive power.

2.5 Productionization¶

It is possible to productionize the pipeline by transpiling the features into production-ready SQL code. Please also refer to getML's sqlite3 and spark modules.

3. Conclusion¶

This tutorial demonstrates that relational learning is a powerful tool for time series. We able to outperform the benchmarks for a scientific paper on a simple public domain time series data set using relatively little effort.

If you want to learn more about getML, check out the official documentation.