vSLDA

vSDLA (Kobayashi et al., 2026) is a late-interaction probabilistic topic model that models as mixtures of von-Mises distributed embeddings. The model is based on Latent Dirichlet Allocation, except its outcome distribution is a better fit for L2-normalized neural embeddings.

Figure 1: Plate notation of the vSLDA model.
Figure from Kobayashi et al. (2026)

Usage

vSDLA is natively a multi-vector/late-interaction topic model. If you intend to use the model effectively you should make yourself familiar with Turftopic's ragged array API.

The authors originally intend the model to be used on sentence level, but in Turftopic, one can use vSLDA on contextual windows or tokens.

Choose granularity and pooling

Token-level w/o PoolingWindow-level w/o PoolingToken-level w Pooling

from turftopic import vSLDA

documents = [...]

# pooling and window_size are default parameters,
# you don't have to specify that they are None
model = vSLDA(10, window_size=None, pooling=None)

# The doc-topic matrix is a list of token-topic matrixes: (n_windows, n_topics)
# offsets contains the character offsets of each token in (start, end) tuples,
doc_topic_matrix, offsets = model.fit_transform(documents)

model.print_topics()

from turftopic import vSLDA

documents = [...]

model = vSLDA(10, window_size=20, pooling=None)

# The doc-topic matrix is a list of window-topic matrixes shape: (n_windows, n_topics)
# offsets contains the character offsets of each window in (start, end) tuples,
doc_topic_matrix, offsets = model.fit_transform(documents)

model.print_topics()

import numpy as np
from turftopic import vSLDA

documents = [...]

# We will use np.nanmean to pool token-level topic probabilities
# One can also use other pooling techniques that respect the nature of probabilities
# better
model = vSLDA(10, pooling=np.nanmean)

# The doc-topic matrix is a regular matrix, shape: (n_docs, n_topics)
# No offsets are returned when pooling is specify
doc_topic_matrix, offsets = model.fit_transform(documents)

model.print_topics()

Term importance estimation

vSLDA, similar to most other models in Turftopic, estimates term importances post-hoc. Kobayashi et al. (2026) used NPMI for this purpose, which is also the default behaviour in Turftopic. By default, however, Turftopic adds a smoothing term to the probability estimates instead of using frequencies, as very infrequent terms could dominate topics when one uses naive estimates. Additionally, one can use a variety of different term importance estimation methods, some of which are embedding-based.

To read more about the different term importance estimation techniques, read our documentation page on Clustering Topic Models and Topeax.

Choose a term importance estimation method

NPMI (default)Lexical-Semantic (recommended)c-TF-IDFFighting Words

from turftopic import vSLDA

documents = [...]

model = vSLDA(10, feature_importance="npmi")
doc_topic_matrix, offsets = model.fit_transform(documents)
model.print_topics()

Kardos (2026) introduced a mixed method for estimating term importances in probabilistic embedding-based topic models, which combines NPMI, a term based measure, and a cosine distance based semantic measure. In our experience, this term importance method tends to be most robust and produces most interpretable topics, but more research remains to be done in this area.

from turftopic import vSLDA

documents = [...]

model = vSLDA(10, feature_importance="lexical-semantic")
doc_topic_matrix, offsets = model.fit_transform(documents)
model.print_topics()

from turftopic import vSLDA

documents = [...]

model = vSLDA(10, feature_importance="c-tf-idf")
doc_topic_matrix, offsets = model.fit_transform(documents)
model.print_topics()

from turftopic import vSLDA

documents = [...]

model = vSLDA(10, feature_importance="fighting-words")
doc_topic_matrix, offsets = model.fit_transform(documents)
model.print_topics()

Choice of embedding model

The authors used all-MiniLM-L6-v2 for most of their investigations, which is, incidentally, the default encoder model in Turftopic. In addition, they they benchmarked MiniLM against two larger embedding models (all-mpnet-base-v2 and bge-base-en-v1.5) and found no improvement.

It has to be noted, that none of the above models were developed for multi-vector tasks. In Turftopic, vSLDA can be used with any embedding model, including MultiVectorEncoder and SentenceTransformer models.

Choose encoder model

all-MiniLM-L6-v2 (default)jinaai/jina-embeddings-v5-text-smallLateOn (native multi-vector retrieval model)

from turftopic import vSLDA

documents = [...]

model = vSLDA(10, encoder="all-MiniLM-L6-v2")
doc_topic_matrix, offsets = model.fit_transform(documents)
model.print_topics()

import torch
from turftopic import vSLDA

documents = [...]

model = vSLDA(
    10, 
    encoder="jinaai/jina-embeddings-v5-text-small",
    trf_kwargs=dict(
        trust_remote_code=True,
        device="cpu",
        model_kwargs={"dtype": torch.bfloat16}
    ),
    encode_kwargs=dict(task="clustering")
)
doc_topic_matrix, offsets = model.fit_transform(documents)
model.print_topics()

from turftopic import vSLDA

documents = [...]

model = vSLDA(10, encoder="lightonai/LateOn")
doc_topic_matrix, offsets = model.fit_transform(documents)
model.print_topics()

References

Please cite the original authors and Turftopic, when using vSLDA:

@misc{kobayashi2026vmfsentenceldaspherical,
      title={vMF Sentence LDA: A Spherical Topic Model over Sentence Embeddings}, 
      author={Ryotaro Kobayashi and Yuri Murayama and Kiyoshi Izumi},
      year={2026},
      eprint={2610.05095},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2610.05095}, 
}

@article{
  Kardos2025,
  title = {Turftopic: Topic Modelling with Contextual Representations from Sentence Transformers},
  doi = {10.21105/joss.08183},
  url = {https://doi.org/10.21105/joss.08183},
  year = {2025},
  publisher = {The Open Journal},
  volume = {10},
  number = {111},
  pages = {8183},
  author = {Kardos, Márton and Enevoldsen, Kenneth C. and Kostkan, Jan and Kristensen-McLachlan, Ross Deans and Rocca, Roberta},
  journal = {Journal of Open Source Software} 
}

API Reference

turftopic.models.vslda.vslda.vSLDA

Bases: LateModel

von-Mises distributed sentence LDA.

from turftopic import vSLDA
corpus: list[str] = ["some text", "more text", ...]

model = vSLDA(10).fit(corpus)
model.print_topics()

Parameters:

Name	Type	Description	Default
n_components	int	Number of topics.	required
encoder	Union[Encoder, str]	Model to encode documents/terms, all-MiniLM-L6-v2 is the default.	'sentence-transformers/all-MiniLM-L6-v2'
vectorizer	Optional[CountVectorizer]	Vectorizer used for term extraction. Can be used to prune or filter the vocabulary.	None
random_state	Optional[int]	Random state to use so that results are exactly reproducible.	None
n_iter	int	Number of iterations to run with the EM algorithm.	25
window_size	Optional[int]	Size of the sliding window to average tokens over. If None, documents will be represented at a token level.	None
step_size	Optional[int]	Step size of the window. If (step_size == None) or (step_size == window_size), then windows are separate. If step_size < window_size, windows will overlap. If step_size > window_size, there will be gaps between the windows. In this case, we throw a warning, as this is probably unintended behaviour.	None
pooling		Indicates whether and how to pool document-topic matrices. If None, multi-vector topic proportions are returned in a ragged array. If Callable, multiple vectors are averaged with the callable in each document. You could for example take the mean by specifying pooling=np.nanmean.	None
feature_importance	FeatureImportance	Feature importance method to use.	'npmi'
trf_kwargs		Keyword arguments to apply when loading the Encoder model.	None
encode_kwargs		Keyword arguments to apply encoding documents with the encoder.	None

Source code in turftopic/models/vslda/vslda.py

class vSLDA(LateModel):
    """von-Mises distributed sentence LDA.

    ```python
    from turftopic import vSLDA
    corpus: list[str] = ["some text", "more text", ...]

    model = vSLDA(10).fit(corpus)
    model.print_topics()
    ```

    Parameters
    ----------
    n_components: int
        Number of topics.
    encoder: str or SentenceTransformer
        Model to encode documents/terms, all-MiniLM-L6-v2 is the default.
    vectorizer: CountVectorizer, default None
        Vectorizer used for term extraction.
        Can be used to prune or filter the vocabulary.
    random_state: int, default None
        Random state to use so that results are exactly reproducible.
    n_iter: int, default 25
        Number of iterations to run with the EM algorithm.
    window_size: int, default None
        Size of the sliding window to average tokens over.
        If None, documents will be represented at a token level.
    step_size: int, default None
        Step size of the window.
        If (step_size == None) or (step_size == window_size), then windows are separate.
        If step_size < window_size, windows will overlap.
        If step_size > window_size, there will be gaps between the windows.
        In this case, we throw a warning, as this is probably unintended behaviour.
    pooling: Callable, default None
        Indicates whether and how to pool document-topic matrices.
        If None, multi-vector topic proportions are returned in a ragged array.
        If Callable, multiple vectors are averaged with the callable in each document.
        You could for example take the mean by specifying `pooling=np.nanmean`.
    feature_importance: FeatureImportance, default 'npmi'
        Feature importance method to use.
    trf_kwargs: dict, default None
        Keyword arguments to apply when loading the Encoder model.
    encode_kwargs: dict, default None
        Keyword arguments to apply encoding documents with the encoder.

    """

    def __init__(
        self,
        n_components: int,
        encoder: Union[
            Encoder, str
        ] = "sentence-transformers/all-MiniLM-L6-v2",
        vectorizer: Optional[CountVectorizer] = None,
        random_state: Optional[int] = None,
        n_iter: int = 25,
        window_size: Optional[int] = None,
        step_size: Optional[int] = None,
        pooling=None,
        feature_importance: FeatureImportance = "npmi",
        trf_kwargs=None,
        encode_kwargs=None,
    ):
        self.n_components = n_components
        self.encoder = encoder
        self.encode_kwargs = encode_kwargs
        self.trf_kwargs = trf_kwargs
        self.random_state = random_state
        self.window_size = window_size
        self.n_iter = n_iter
        self.step_size = step_size
        self.pooling = pooling
        self.load_encoder()
        if vectorizer is None:
            self.vectorizer = default_vectorizer()
        else:
            self.vectorizer = vectorizer
        if feature_importance not in FEATURE_IMPORTANCE_METHODS:
            valid = list(FEATURE_IMPORTANCE_METHODS.keys())
            raise ValueError(
                f"{feature_importance} not in list of valid feature importance methods: {valid}"
            )
        self.feature_importance = feature_importance

    @property
    def topic_embeddings(self):
        return self.trainer_.topic_means

    def estimate_components(
        self,
        feature_importance: Optional[FeatureImportance] = None,
        doc_topic_matrix=None,
        doc_term_matrix=None,
    ) -> np.ndarray:
        feature_importance = feature_importance or self.feature_importance
        imp_fn = FEATURE_IMPORTANCE_METHODS[feature_importance]
        doc_topic_matrix = (
            doc_topic_matrix
            if doc_topic_matrix is not None
            else self.doc_topic_matrix
        )
        doc_term_matrix = (
            doc_term_matrix
            if doc_term_matrix is not None
            else self.doc_term_matrix
        )
        if feature_importance != "lexical-semantic":
            self.components_ = imp_fn(doc_topic_matrix, doc_term_matrix)
            return self.components_
        vocab = self.get_vocab()
        if getattr(self, "vocab_embeddings", None) is None or (
            self.vocab_embeddings.shape[0] != vocab.shape[0]
        ):
            self.vocab_embeddings = self.encode_vocabulary_items(vocab)
        self.components_ = imp_fn(
            doc_topic_matrix,
            doc_term_matrix,
            self.topic_embeddings,
            self.vocab_embeddings,
        )
        return self.components_

    def fit_transform(
        self,
        raw_documents,
        y=None,
        embeddings: Optional[list[np.ndarray]] = None,
        offsets: Optional[Offsets] = None,
    ) -> np.ndarray:
        console = Console()
        with console.status("Fitting model") as status:
            if (embeddings is None) or (offsets is None):
                status.update("Encoding documents")
                embeddings, offsets = self.encode_late(raw_documents)
                console.log("Documents encoded.")
            self.embeddings = embeddings
            status.update("Extracting terms.")
            ragged_dtm = self.ragged_vectorize(
                raw_documents, offsets=offsets, train_vectorizer=True
            )
            console.log("Term extraction done.")
            status.update("Fitting model using EM.")
            self.trainer_ = VMFLDATrainer(embeddings, self.n_components)
            self.trainer_.sample(embeddings, num_iterations=self.n_iter)
            console.log("Model fitted.")
            status.update("Estimating term importances.")
            flat_dtm, _ = self.flatten_repr(ragged_dtm)
            self.doc_topic_matrix = self.trainer_.infer_topic_proportions(
                embeddings
            )
            flat_topics, _ = self.flatten_repr(self.doc_topic_matrix)
            self.estimate_components(
                doc_topic_matrix=flat_topics, doc_term_matrix=flat_dtm
            )
            console.log("Model fitting done.")
        return self.transform(
            raw_documents, embeddings=embeddings, offsets=offsets
        )

    def transform(
        self,
        raw_documents,
        embeddings: Optional[np.ndarray] = None,
        offsets: Optional[Offsets] = None,
    ) -> np.ndarray:
        if embeddings is None:
            embeddings, offsets = self.encode_late(raw_documents)
        doc_topic_matrix = self.trainer_.infer_topic_proportions(embeddings)
        return self.pool(doc_topic_matrix, offsets)