ModernNCA

NeuroTabModels.Models.ModernNCA.ModernNCAConfig Type

ModernNCAConfig(; d_embedding=128, n_blocks=2, d_block=256,
                 dropout=0.1, temperature=1.0, sample_rate=0.8,
                 max_candidates=8192, eps=1f-8)

Hyperparameters for ModernNCA. Pass as the arch argument to NeuroTabRegressor or NeuroTabClassifier.

Arguments

• d_embedding: encoder output dimension.

• n_blocks, d_block: number and hidden width of post-encoder MLP blocks.

• dropout: dropout rate inside each block; skipped when <= 0.

• temperature: softmax temperature on negative pairwise distances.

• sample_rate: fraction of the batch complement sampled as candidates per step (Stochastic Neighborhood Sampling). 1.0 uses the full complement.

• max_candidates: cap on sampled training candidates per batch. Set <= 0 to disable the cap.

• eps: numerical floor in sqrt and as a temperature lower bound.

source

NeuroTabModels.Models.ModernNCA.ModernNCALoader Type

ModernNCALoader

Training iterator yielding ((x, cand_x, cand_y, y), y) batches. The corpus is moved to device once at construction; query rows follow a shuffled epoch permutation and candidates are resampled each step.

Fields

• full_x, full_y: device corpus.

• batchsize: query rows per step.

• n_cand: candidate rows sampled per step (0 when batchsize == N).

• rng: RNG for sampling.

• dev: device-transfer callable.

source

NeuroTabModels.Models.ModernNCA.ModernNCAModel Type

ModernNCAModel

Lux wrapper holding the backbone encoder, config, output size, and loss.

source

NeuroTabModels.Models.ModernNCA.ModernNCAModel Method

(m::ModernNCAModel)((x, cand_x, cand_y, y), ps, st)

Training forward: encodes queries and candidates separately, matching the reference ModernNCA BatchNorm behavior. The encoded queries are prepended to the candidate embeddings before NCA attention, then self-neighbors are masked.

source

NeuroTabModels.Models.ModernNCA.ModernNCAModel Method

(m::ModernNCAModel)((x, cx, cy), ps, st)

Inference/eval forward: encodes queries and the raw corpus cx with the current parameters, then attends over the corpus. Encoding is done here (not pre-computed) so the corpus embeddings always match the current model during evaluation.

source

Base.iterate Function

Base.iterate(l::ModernNCALoader, state=nothing)

state = (perm, start) carries the epoch permutation across calls, so query batches are random without depending on dataframe row order.

Candidates are sampled from the complement of the query batch within perm. The index-skip mapping avoids allocating that complement: for j in 1:(n-batchsize), use perm[j] before the batch window and perm[j + batchsize] after it. n_cand == 0 only when batchsize == n; then the forward pass keys on the batch itself with diagonal masking.

source

NeuroTabModels.Models.ModernNCA._backbone Method

_backbone(cfg, ins, embedding_layer)

Build the ModernNCA encoder: embedding, linear, then n_blocks times (BN, Dense(relu), Dropout, Dense), then BN.

source

NeuroTabModels.Models.ModernNCA._encode_corpus Method

_encode_corpus(m, cx, ps, st; chunk=2048) -> Matrix

Encode raw corpus cx (d_ins, N) in chunks of chunk rows to bound peak memory and stay within BatchNorm shape limits. Returns zk (d_embedding, N).

source

NeuroTabModels.Models.ModernNCA._encode_targets Method

_encode_targets(df, target_name, loss, scalers)

Encode targets for each loss:

• MLogLoss: 1-based UInt32 class codes.

• LogLoss: Float32 in {0, 1}.

• MSE / MAE: Float32, standardised when scalers is provided.

Arguments

• df: source data frame.

• target_name: target column.

• loss: target encoding dispatch.

• scalers: optional target scaler (mu, sigma).

source

NeuroTabModels.Models.ModernNCA._mask_diag Method

_mask_diag(d) -> Matrix

Set the diagonal of distance matrix d to typemax, collapsing self-attention weights to zero during training.

source

NeuroTabModels.Models.ModernNCA._nca_logits Method

_nca_logits(m, zq, zk, cy; mask_self=false) -> Matrix

Softmax attention over corpus keys zk for query embeddings zq. When mask_self=true, diagonal entries are masked (training path).

source

NeuroTabModels.Models.ModernNCA._pairwise_dist Method

_pairwise_dist(q, k, ϵ) -> Matrix

(num_keys, batch) Euclidean distance matrix between q (d, batch) and k (d, num_keys). ϵ is added under sqrt for numerical stability.

source

NeuroTabModels.Models.ModernNCA.build_corpus Method

build_corpus(df, feature_names, target_name, loss, scalers)

Return (full_x, full_y): the corpus feature matrix (ins, N) as Float32 and the encoded target vector, ready for ModernNCALoader.

source

NeuroTabModels.Models.eval_dataloader Function

Models.eval_dataloader(m::ModernNCAModel, ...)

Wrap each eval batch as ((x, cx, cy), rest...) with the raw training corpus. The corpus is encoded inside the forward pass so embeddings always use the current model parameters rather than a stale pre-encoded version.

source

NeuroTabModels.Models.infer_dataloader Function

Models.infer_dataloader(m::ModernNCAModel, ...)

Wrap each inference batch as (x, cx, cy) with the raw training corpus. The corpus is encoded inside the forward pass using the current parameters.

source

NeuroTabModels.Models.train_dataloader Method

Models.train_dataloader(cfg::ModernNCAConfig, ...)

Build ModernNCALoader and stash the raw corpus on m.info[:nca_ref]. n_cand = floor(sample_rate × (N − batchsize)), capped by cfg.max_candidates when positive. sample_rate ≥ 1 uses the full complement before the cap; batchsize == N gives n_cand = 0.

Arguments

• cfg: ModernNCA config.

• m: fitted model wrapper.

• df: training data frame.

• feature_names

• target_name

• loss

• scalers

• batchsize

• dev: device

• rng

source