EvoTrees.jl

Appearance

Public API ​

Fit ​

MLJModelInterface.fit Function
julia
fit(
    params::EvoTypes, 
    dtrain;
    target_name,
    feature_names=nothing,
    weight_name=nothing,
    offset_name=nothing,
    group_name=nothing,
    eval_group_name=group_name,
    deval=nothing,
    print_every_n=9999,
    verbosity=1
    )

Main training function. Performs model fitting given configuration params, dtrain, target_name and other optional kwargs.

Arguments

Keyword arguments

  • target_name: name of the target variable.

  • feature_names = nothing: the names dtrain variables to use as features. If not provided, it deafults to all variables that aren't one of target, weight or `offset``.

  • weight_name = nothing: name of the variable containing weights. If nothing, common weights on one will be used.

  • offset_name = nothing: name of the offset variable.

  • group_name = nothing: name of the variable identifying the group (query) each row belongs to. Rows sharing an id form one group. Ids need not be contiguous, sorted, or numeric. Supplying groups makes rowsample sample whole groups rather than individual rows.

  • eval_group_name = group_name: name of the group variable in deval, defaulting to group_name. A group-aware metric such as :ndcg requires it. Set it on its own to evaluate over groups while training with the usual per-row sampling.

  • deval: A Tables compatible evaluation data containing features and target variables.

  • print_every_n: sets at which frequency logging info should be printed.

  • verbosity: set to 1 to print logging info during training.

source
julia
fit(
    params::EvoTypes{L};
    x_train::AbstractMatrix, 
    y_train::AbstractVecOrMat, 
    w_train=nothing, 
    offset_train=nothing,
    x_eval=nothing, 
    y_eval=nothing, 
    w_eval=nothing, 
    offset_eval=nothing,
    group_train=nothing,
    group_eval=nothing,
    feature_names=nothing,
    early_stopping_rounds=9999,
    print_every_n=9999,
    verbosity=1
    )

Main training function. Performs model fitting given configuration params, x_train, y_train and other optional kwargs.

Arguments

Keyword arguments

  • x_train::Matrix: training data of size [#observations, #features].

  • y_train::VecOrMat: vector or matrix of train targets of length #observations or size (#observations, #targets).

  • w_train::Vector: vector of train weights of length #observations. If nothing, a vector of ones is assumed.

  • offset_train::VecOrMat: offset for the training data. Should match the size of the predictions.

  • x_eval::Matrix: evaluation data of size [#observations, #features].

  • y_eval::VecOrMat: vector or matrix of evaluation targets of length #observations or size (#observations, #targets).

  • w_eval::Vector: vector of evaluation weights of length #observations. Defaults to nothing (assumes a vector of 1s).

  • offset_eval::VecOrMat: evaluation data offset. Should match the size of the predictions.

  • group_train::Vector: group (query) id of each training row, for ranking tasks. Rows sharing an id form one group. Ids need not be contiguous, sorted, or numeric. Supplying groups makes rowsample sample whole groups rather than individual rows.

  • group_eval::Vector: group id of each evaluation row. Required by metric = :ndcg.

  • feature_names = nothing: the names of the x_train features. If provided, should be a vector of string with length(feature_names) = size(x_train, 2).

  • print_every_n: sets at which frequency logging info should be printed.

  • verbosity: set to 1 to print logging info during training.

source

Predict ​

MLJModelInterface.predict Function
julia
predict(m::EvoTree, data; ntree_limit=length(m.trees), device=:cpu)

Predictions from an EvoTree model — the bias plus the sum of boosting trees. Use ntree_limit=N to use the bias plus the first N trees (N = 0 is bias only).

source
EvoTrees.predict_leaf_idx Function
julia
predict_leaf_idx(m::EvoTree, data; ntree_limit=length(m.trees))

Return the index of the leaf into which each observation falls, for each boosting tree.

The result is a Matrix{UInt32} of size (nobs, ntree_limit), where [i, j] is the index of the leaf reached by observation i in tree j. Indices refer to the node numbering of m.trees[j]: the root is 1, and the children of node n are 2n and 2n + 1. Use ntree_limit=N to only use the first N trees. The bias is not a tree, so nrounds = 0 returns an (nobs, 0) matrix.

Leaf indices are a categorical encoding of the partition of the feature space learned by the model, and can be used as features for a downstream model.

julia
leaf_idx = EvoTrees.predict_leaf_idx(m, x)
source

SHAP ​

EvoTrees.Shap.shap Function
julia
shap(m::EvoTree, data; ntree_limit=length(m.trees))

Returns the shap effect as a Matrix of size [nobs, features].

It's based on an implementation of Linear TreeShap by Yu et al. (2022). It computes exact Shapley values for decision trees in O(LD) time. It was originally ported from this repo.

References

Peng Yu, Chao Xu, Albert Bifet, Jesse Read Linear Tree Shap (2022). In Proceedings of 36th Conference on Neural Information Processing Systems.

source

Importance ​

EvoTrees.importance Function
julia
importance(model::EvoTree; feature_names=model.info[:feature_names])

Sorted normalized feature importance based on loss function gain. Feature names associated to the model are stored in model.info[:feature_names] as a string Vector and can be updated at any time. Eg: model.info[:feature_names] = new_feature_names_vec.

source

Plot ​

EvoTrees.treeplot Function
julia
treeplot(model::EvoTree, n=1)
treeplot!(ax, model::EvoTree, n=1)

Plot tree n of a fitted EvoTree. Default n = 1 is the first boosting tree. Requires a Makie backend (CairoMakie or GLMakie). plot(model, n) works once a backend is loaded.

source