Model/Topic Merging

In Turftopic, some models allow you to merge information from multiple topic models into one. Currently, this can be done with SensTopic's partial_fit method, which updates a topic model by merging it with an identically initialized model on a new batch of data. More models will be implemented in the future. This guide teaches you about the different methods for merging topics and explanations on why they are useful, and how you should use them.

Topic merging consists of the following steps:

Determine topic matches based on some topic representations (typically a model's components_ attribute) and a similarity threshold
Aggregating the topic representations based on some aggregation regime

See the default named options from Turftopic in the table bellow. We deem these to be reasonable defaults that cover most of the functionality, users might be interested in.

Merge method	Match type	Aggregation	Recommended Use Cases
`symmetric_mean`	Symmetric	`np.average`	Online model fitting on static datasets.
`asymmetric_mean`	Asymmetric	`np.average`	Dynamic modelling, static analyses, where intermediate states are used for analysis.
`keep_first`	Asymmetric	`keep_first`	Dynamic modelling, where intermediate states should be immutable.

Determining matches

When merging topic models, it's important to know the difference between symmetric and asymmetric merges.

Symmetric merge

A symmetric merge treats both models as equal, and finds matching topics in all models at the same time to then merge them into a single topic in the new model.

This is done in the following steps:

We calculate a topic similarity matrix between topics from all models. By default this is based on topic representations' cosine similarity.
We compute a match matrix based on the similarity matrix and a similarity threshold. The default value is 0.7.
The match matrix is used as a match graph between all topics in the models.
To find, which topics should be aggregated to derive the new topics, we find graph components in the match graph. Each component of the match graph is then assumed to be the same topic, and each component will be aggregated into a new topic.

Note

When using symmetric merge, the number of topics from one step to another could go down not just up. This is because sometimes the new model introduces bridges in the match graph between old topics, that then get merged into one larger topic. This is important to take into account when you make assumptions about the way your topics behave.

A symmetric merge is a good fit, if you wish to find all topics in a corpus, and you do not base any of your analyses on intermediate states of the model. Using symmetric merges on a static dataset is a good idea, using it on temporal data (dynamic modelling) is a bad idea.

Asymmetric merge

During an asymmetric merge, earlier models take precedence over new ones. What this means is that newer models' content get merged into older models, while the older models' structure is left unscathed.

The old model's topics might get updated based on the new model, but they will not be removed or merged into other topics. This also means that the number of topics never decreases, only increases over time.

For each pair of old and new model:
1. Calculate the similarity and match matrices between the old and new model.
2. Aggregate the matches from the new model into the old models topics.

You should use an asymmetric merge either if you want to keep your analyses in-tact based on intermediate states, or if you want to make the assumption that the number of topics is non-decreasing over time. Asymmetric merges are perfect for instance for dynamic topic modelling.

Aggregation Regimes

You can technically use any aggregation method to aggregate topics between matches, but there are two defaults used in Turftopic, that probably cover most use cases.

`np.average`

np.average takes the weighted arithmetic mean of topic representations. By default, no weights are provided, but most models, where there is a deliberate implementation included, averages are weighted by the number of documents the models have seen.

This aggregation method is very useful when you want all of your data to influence your topics, and you don't care whether the keywords throughout your analysis change for the topics.

`keep_first`

keep_first aggregation ignores all topics except the first one in a match. This means that topics that are already in the model will be immutable. New topics will be added, but the ones in the model will not change.

This is great when you need to make the assumption that old topics never change. For instance, when you have already based some of your analyses on old topics in your model, new information should not change those analyses.

API Reference

`turftopic.merging.symmetric_merge(component_matrices, weights=None, match_threshold=0.7, agg=np.average, sim_fn=cosine_similarity, allow_within_model_match=False)`

Performs a symmetric merge on a number of topic models.

Parameters:

Name	Type	Description	Default
`component_matrices`	`list[ndarray]`	List of topic representations from all topic models.	required
`weights`		Weight for each of the models in case of a weighted merge.	`None`
`match_threshold`	`float`	Similarity threshold above which two topics will be considered a match.	`0.7`
`agg`		Aggregation method to use for matching topics.	`average`
`sim_fn`		Function to produce the similarity matrix.	`cosine_similarity`
`allow_within_model_match`		Determines whether matches can happen within models.	`False`

Returns:

Type	Description
`ndarray of shape (n_new_topics, n_dims)`	New topic representations merged from the old ones.
`MergeHistory(list[list[int]])`	Indicates which joint topic the original topics were merged into. The data structure is a list of lists, where each list contains the indices of the joint topics each of the original topics were merged into. e.g. [[0,2,1], [3,2]] would indicate that the topics of the first model were merged into topics 0, 2 and 1 in the joint model, while the second model's topics were merged into 3 and 2.

Source code in turftopic/merging.py

def symmetric_merge(
    component_matrices: list[np.ndarray],
    weights=None,
    match_threshold: float = 0.7,
    agg=np.average,
    sim_fn=cosine_similarity,
    allow_within_model_match=False,
) -> tuple[np.ndarray, MergeHistory]:
    """Performs a symmetric merge on a number of topic models.

    Parameters
    ----------
    component_matrices: list[np.ndarray]
        List of topic representations from all topic models.
    weights: Sequence, None
        Weight for each of the models in case of a weighted merge.
    match_threshold: float, default 0.7
        Similarity threshold above which two topics will be considered a match.
    agg: Callable, default np.average
        Aggregation method to use for matching topics.
    sim_fn: Callable, default cosine_similarity
        Function to produce the similarity matrix.
    allow_within_model_match: bool, default False
        Determines whether matches can happen within models.

    Returns
    -------
    ndarray of shape (n_new_topics, n_dims)
        New topic representations merged from the old ones.
    MergeHistory (list[list[int]])
        Indicates which joint topic the original topics were merged into.
        The data structure is a list of lists, where each list contains the indices of the joint topics
        each of the original topics were merged into.
        e.g. [[0,2,1], [3,2]] would indicate that the topics of the first model were merged into
        topics 0, 2 and 1 in the joint model, while the second model's topics were merged into 3 and 2.
    """
    stacked_components, old_labels = stack_components(component_matrices)
    similarity = sim_fn(stacked_components, stacked_components)
    if not allow_within_model_match:
        i_processed = 0
        for comp in component_matrices:
            n_comp = comp.shape[0]
            similarity[
                i_processed : i_processed + n_comp,
                i_processed : i_processed + n_comp,
            ] = np.eye(n_comp).T
            i_processed += n_comp
    matches = spr.csr_array(similarity > match_threshold)
    n_graph_components, labels = spr.csgraph.connected_components(
        matches, directed=False
    )
    merge_history = []
    current_ind = 0
    for i_model, comp in enumerate(component_matrices):
        n_model_components = comp.shape[0]
        merge_history.append(
            labels[current_ind : current_ind + n_model_components]
        )
        current_ind += n_model_components
    new_components = merge_from_history(
        component_matrices, merge_history, weights=weights, agg=agg
    )
    return new_components, merge_history

`turftopic.merging.asymmetric_merge(component_matrices, weights=None, match_threshold=0.7, agg=keep_first, sim_fn=cosine_similarity)`

Performs an asymmetric merge on a number of topic models.

Parameters:

Name	Type	Description	Default
`component_matrices`	`list[ndarray]`	List of topic representations from all topic models.	required
`weights`		Weight for each of the models in case of a weighted merge.	`None`
`match_threshold`	`float`	Similarity threshold above which two topics will be considered a match.	`0.7`
`agg`		Aggregation method to use for matching topics.	`keep_first`
`sim_fn`		Function to produce the similarity matrix.	`cosine_similarity`
`allow_within_model_match`		Determines whether matches can happen within models.	required

Returns:

Type	Description
`ndarray of shape (n_new_topics, n_dims)`	New topic representations merged from the old ones.
`MergeHistory(list[list[int]])`	Indicates which joint topic the original topics were merged into. The data structure is a list of lists, where each list contains the indices of the joint topics each of the original topics were merged into. e.g. [[0,2,1], [3,2]] would indicate that the topics of the first model were merged into topics 0, 2 and 1 in the joint model, while the second model's topics were merged into 3 and 2.

Source code in turftopic/merging.py

def asymmetric_merge(
    component_matrices: list[np.ndarray],
    weights=None,
    match_threshold: float = 0.7,
    agg=keep_first,
    sim_fn=cosine_similarity,
) -> tuple[np.ndarray, MergeHistory]:
    """Performs an asymmetric merge on a number of topic models.

    Parameters
    ----------
    component_matrices: list[np.ndarray]
        List of topic representations from all topic models.
    weights: Sequence, None
        Weight for each of the models in case of a weighted merge.
    match_threshold: float, default 0.7
        Similarity threshold above which two topics will be considered a match.
    agg: Callable, default np.average
        Aggregation method to use for matching topics.
    sim_fn: Callable, default cosine_similarity
        Function to produce the similarity matrix.
    allow_within_model_match: bool, default False
        Determines whether matches can happen within models.

    Returns
    -------
    ndarray of shape (n_new_topics, n_dims)
        New topic representations merged from the old ones.
    MergeHistory (list[list[int]])
        Indicates which joint topic the original topics were merged into.
        The data structure is a list of lists, where each list contains the indices of the joint topics
        each of the original topics were merged into.
        e.g. [[0,2,1], [3,2]] would indicate that the topics of the first model were merged into
        topics 0, 2 and 1 in the joint model, while the second model's topics were merged into 3 and 2.
    """
    merge_history = []
    components = np.copy(component_matrices[0])
    # First components will just be kept
    merge_history.append(list(range(components.shape[0])))
    for incoming_components in component_matrices[1:]:
        n_current = components.shape[0]
        _merge_inst = []
        similarity = sim_fn(components, incoming_components)
        maxsim_ind = np.argmax(similarity.T, axis=1)
        to_add = []
        for i_new_comp, ind_most_similar_old in enumerate(maxsim_ind):
            if similarity[ind_most_similar_old, i_new_comp] > match_threshold:
                components[ind_most_similar_old] = safe_agg(
                    np.concatenate(
                        [
                            components[[ind_most_similar_old], :],
                            incoming_components[[i_new_comp], :],
                        ],
                        axis=0,
                    ),
                    agg=agg,
                    weights=weights,
                    axis=0,
                )
                _merge_inst.append(ind_most_similar_old)
            else:
                _merge_inst.append(n_current + len(to_add))
                to_add.append(i_new_comp)
        components = np.concatenate(
            [components, incoming_components[to_add]], axis=0
        )
        merge_history.append(_merge_inst)
    return components, merge_history

`turftopic.merging.keep_first(a, axis=0)`

Source code in turftopic/merging.py

def keep_first(a, axis=0):
    return np.take(a, 0, axis=axis)