Models

NeuroTreeRegressor

# NeuroTreeModels.NeuroTreeRegressor — Type.

NeuroTreeRegressor(;kwargs...)

A model type for constructing a NeuroTreeRegressor, based on NeuroTreeModels.jl, and implementing both an internal API and the MLJ model interface.

Hyper-parameters

• loss=:mse: Loss to be be minimized during training. One of:

  • :mse

  • :mae

  • :logloss

  • :mlogloss

  • :gaussian_mle

• nrounds=10: Max number of rounds (epochs).

• lr=1.0f-2: Learning rate. Must be > 0. A lower eta results in slower learning, typically requiring a higher nrounds.

• wd=0.f0: Weight decay applied to the gradients by the optimizer.

• batchsize=2048: Batch size.

• actA=:tanh: Activation function applied to each of input variable for determination of split node weight. Can be one of:

  • :tanh

  • :identity

• outsize=1: Number of predictions returned by the model. Typically only used for classification tasks and set to the number of target levels / classes.

• depth=6: Depth of a tree. Must be >= 1. A tree of depth 1 has 2 prediction leaf nodes. A complete tree of depth N contains 2^N terminal leaves and 2^N - 1 split nodes. Compute cost is proportional to 2^depth. Typical optimal values are in the 3 to 5 range.

• ntrees=64: Number of trees (per stack).

• hidden_size=16: Size of hidden layers. Applicable only when stack_size > 1.

• stack_size=1: Number of stacked NeuroTree blocks.

• init_scale=1.0: Scaling factor applied to the predictions weights. Values in the ]0, 1] short result in best performance.

• MLE_tree_split=false: Whether independent models are buillt for each of the 2 parameters (mu, sigma) of the the gaussian_mle loss.

• rng=123: Either an integer used as a seed to the random number generator or an actual random number generator (::Random.AbstractRNG).

• device=:cpu: Device to use. Either :cpu or :gpu (recommended as it improves significantly the training speed).

• gpuID=0: ID of the GPU to use for training.

Internal API

Do config = NeuroTreeRegressor() to construct an instance with default hyper-parameters. Provide keyword arguments to override hyper-parameter defaults, as in NeuroTreeRegressor(loss=...).

Training model

A model is trained using fit:

m = fit(config, dtrain; feature_names, target_name, kwargs...)

Inference

Models act as a functor. returning predictions when called as a function with features as argument:

m(data)

MLJ Interface

From MLJ, the type can be imported using:

NeuroTreeRegressor = @load NeuroTreeRegressor pkg=NeuroTreeModels

Do model = NeuroTreeRegressor() to construct an instance with default hyper-parameters. Provide keyword arguments to override hyper-parameter defaults, as in NeuroTreeRegressor(loss=...).

Training model

In MLJ or MLJBase, bind an instance model to data with mach = machine(model, X, y) where

• X: any table of input features (eg, a DataFrame) whose columns each have one of the following element scitypes: Continuous, Count, or <:OrderedFactor; check column scitypes with schema(X)

• y: is the target, which can be any AbstractVector whose element scitype is <:Continuous; check the scitype with scitype(y)

Train the machine using fit!(mach, rows=...).

Operations

• predict(mach, Xnew): return predictions of the target given features Xnew having the same scitype as X above.

Fitted parameters

The fields of fitted_params(mach) are:

• :fitresult: The NeuroTreeModel object.

Report

The fields of report(mach) are:

• :features: The names of the features encountered in training.

Examples

Internal API

using NeuroTreeModels, DataFrames
config = NeuroTreeRegressor(depth=5, nrounds=10)
nobs, nfeats = 1_000, 5
dtrain = DataFrame(randn(nobs, nfeats), :auto)
dtrain.y = rand(nobs)
feature_names, target_name = names(dtrain, r"x"), "y"
m = fit(config, dtrain; feature_names, target_name)
p = m(dtrain)

MLJ Interface

using MLJBase, NeuroTreeModels
m = NeuroTreeRegressor(depth=5, nrounds=10)
X, y = @load_boston
mach = machine(m, X, y) |> fit!
p = predict(mach, X)

source

NeuroTreeModel

# NeuroTreeModels.NeuroTreeModel — Type.

NeuroTreeModel

A NeuroTreeModel is made of a collection of Tree, either regular NeuroTree or StackTree. Prediction is the sum of all the trees composing a NeuroTreeModel.

source