AMF Classifier

Aggregated Mondrian Forest classifier for online learning. This implementation is truly online, in the sense that a single pass is performed, and that predictions can be produced anytime.

Each node in a tree predicts according to the distribution of the labels it contains. This distribution is regularized using a Jeffreys prior with parameter dirichlet. For each class with count labels in the node and n_samples samples in it, the prediction of a node is given by

The prediction for a sample is computed as the aggregated predictions of all the subtrees along the path leading to the leaf node containing the sample. The aggregation weights are exponential weights with learning rate step and log-loss when use_aggregation is True.

This computation is performed exactly thanks to a context tree weighting algorithm. More details can be found in the paper cited in the reference¹ below.

The final predictions are the average class probabilities predicted by each of the n_estimators trees in the forest.

Parameters

• n_classes(int) → The number of classes for classification.

• n_estimators(int, Default:10) → The number of trees in the forest.

• step (float, Default:1.0) → Step-size for the aggregation weights. Default is 1 for classification with the log-loss, which is usually the best choice.

• use_aggregation(bool, Default:True) → Controls if aggregation is used in the trees. It is highly recommended to leave it as True.

• dirichlet (float, Default:0.5) → Regularization level of the class frequencies used for predictions in each node. A rule of thumb is to set this to 1 / n_classes, where n_classes is the expected number of classes which might appear. Default is dirichlet = 0.5, which works well for binary classification problems.

• split_pure(bool, Default:False) → Controls if nodes that contains only sample of the same class should be split ("pure" nodes). Default is False, namely pure nodes are not split, but True can be sometimes better.

• seed(int | None, Default:None) → Random seed for reproducibility.

Example Usage

We can simply use the below syntax to invoke the list of algorithms preconfigured in TurboML, here have_labels=True means supervised models.

import turboml as tb
amf_model = tb.AMFClassifier(n_classes=2)