Multiple factor analysis

Table of contents

Resources

• Multiple Factor Analysis by Hervé Abdi
• Multiple Factor Analysis: main features and application to sensory data by Jérôme Pagès
• Wikipedia article

Data

Multiple factor analysis (MFA) is designed for datasets where variables are organized into groups. A common situation is when the same individuals are described by several sets of variables — for example, the same wines rated by different expert panels, the same patients measured with different medical instruments, or the same geographical sites described by both flora and soil characteristics.

The problem with running a plain PCA on all variables at once is that larger groups dominate the analysis simply through sheer numbers, not because they carry more meaningful information. If flora is described by 50 variables and soil by 11, PCA will be influenced mainly by the flora group. MFA solves this by weighting each group so that no single group can dominate the first principal component.

The following dataset contains end of season figures for Premier League football teams. It spans the 2021/22, 2022/23, and 2023/24 seasons. Each season forms a natural group of variables. Only the 14 teams that have been in the Premier League for all three seasons are included.

import prince

dataset = prince.datasets.load_premier_league()
dataset

import pandas as pd

isinstance(dataset.columns, pd.MultiIndex)

True

Fitting

Under the hood, MFA proceeds in two stages:

1. Separate PCAs: a PCA is fitted independently on each group of variables.
2. Weighted global PCA: all variables are concatenated, but each variable in group j is divided by the first singular value of group j’s separate PCA. This ensures that the maximum axial inertia of every group is normalized to 1, so no single group can monopolize the first axis of the global analysis.

The result is a compromise: the global components reflect the common structure shared across groups, rather than the structure of whichever group happens to have the most variables or the highest variance.

The groups are specified by the groups argument when calling fit.

groups = dataset.columns.levels[0].tolist()
groups

['2021-22', '2022-23', '2023-24']

mfa = prince.MFA(
    n_components=3,
    n_iter=3,
    rescale_with_mean=True,
    rescale_with_std=True,
    copy=True,
    check_input=True,
    engine='sklearn',
    random_state=42
)
mfa = mfa.fit(
    dataset,
    groups=groups,
    supplementary_groups=None
)

There are several ways to specify the groups:

• If the columns of the dataframe are a MultiIndex:
  • By default the groups are all the columns in the first level.
  • You can also pass a list with a subset of the columns in the first level.
• You can also pass a dict that maps group names to the desired columns.

The supplementary_groups argument is expected to be a list with one more existing group names.

The rescale_with_mean and rescale_with_std parameters control whether centering and standardization are applied. These are passed to both the partial PCAs (one per group) and the global PCA.

Eigenvalues

As with PCA, eigenvalues indicate how much variance (inertia) each component captures. The first component represents the direction of maximum consensus across all groups.

mfa.eigenvalues_summary

Row coordinates

Row coordinates position each individual (here, each football team) in the space defined by the MFA components. Teams with similar performance profiles across all three seasons will be close together.

The MFA inherits from the PCA class, which means it provides access to the PCA methods and properties. For instance, the row_coordinates method will return the global coordinates of each team.

mfa.row_coordinates(dataset)

Partial row coordinates

A key feature of MFA is the concept of partial individuals. Each team can be viewed from the perspective of a single season: its “partial point” for 2021-22 shows where that team would sit if only the 2021-22 variables mattered. The global coordinate is the barycenter (average) of the partial points across groups.

When a team’s partial points cluster tightly together, it means the team had a consistent profile across all three seasons. When they spread apart, the team’s performance changed significantly from one season to another.

mfa.partial_row_coordinates(dataset)

Visualization

mfa.plot(
    dataset,
    x_component=0,
    y_component=1
)

The show_partial_rows argument allows showing the global row coordinates together with the partial row coordinates. Each team’s partial points are connected to its global point with edges. Teams whose edges are short had a stable profile across seasons; teams with long edges changed substantially.

mfa.plot(
    dataset,
    show_partial_rows=True
)

Separate PCAs

Before building the global analysis, MFA fits a separate PCA on each group. These are stored in the MFA object and can be inspected individually. The first eigenvalue of each group PCA is what determines the weighting: each variable in a group is divided by that group’s first singular value.

For example, here is the 2022-23 season data and its separate PCA eigenvalues:

dataset['2022-23']

mfa['2022-23'].eigenvalues_summary

Column coordinates

Column coordinates show how each variable loads onto the global MFA components. Since MFA normalizes each group’s influence, these loadings reflect the balanced structure across groups rather than being dominated by whichever group has the most variables.

mfa.column_coordinates_

Contributions

Contributions indicate how much each row or column contributes to the variance of each component. They sum to 1 across all rows (or all columns) for each component.

mfa.row_contributions_.style.format('{:.0%}')

mfa.column_contributions_.style.format('{:.0%}')

Cosine similarities

Cosine similarities (cos²) measure the quality of representation of each row or column on each component. A high cos² means that the component faithfully represents that individual or variable; a low cos² means the individual’s position on that component is unreliable.

mfa.row_cosine_similarities(dataset)

mfa.column_cosine_similarities_

Column correlations

Column correlations (also called loadings) measure the Pearson correlation between each original variable and each MFA component. The squared correlations equal the cosine similarities.

mfa.column_correlations

Group results

One of MFA’s most useful features is its ability to summarize results at the group level. Instead of examining 18 individual variable loadings, you can look at 3 group-level summaries — one per season. These correspond to FactoMineR’s result$group outputs.

This is especially valuable when groups are numerous and contain many variables, because a single group-level plot can replace dozens of individual variable plots.

Group coordinates indicate how strongly each group is associated with each MFA component. They are computed by summing the weighted squared variable coordinates within each group. A high value means that the group’s variables are collectively well-aligned with that component.

mfa.group_coordinates_

Group contributions show what percentage of each component’s variance is explained by each group. They sum to 100% across groups for each component. Here, the 2023-24 season contributes more to component 0 than the other seasons, meaning this component captures structure that is particularly prominent in that season.

mfa.group_contributions_.style.format('{:.0%}')

Group cosine similarities (cos²) measure the quality of representation of each group on each component. The 2023-24 season has a very high cos² on component 0 (0.82), meaning this component captures most of that season’s internal structure. In contrast, the same season has a very low cos² on components 1 and 2, meaning those axes tell us little about within-season variation for 2023-24.

mfa.group_cosine_similarities_

Partial axes

Partial axes answer the question: how does each group’s own internal structure relate to the global MFA structure? Each group has its own PCA components (from the separate analyses in step 1). Partial axes show the correlation between these group-level components and the global MFA components.

This is useful for understanding whether all groups share the same dominant pattern (high correlations between each group’s first axis and the global first axis) or whether some groups introduce distinct structure on different global axes.

Partial correlations show the Pearson correlation between each group’s PCA components and the global MFA components. Here, all three seasons’ first component correlates strongly with the global first component (0.84, 0.85, 0.96), meaning the dominant pattern (strong vs. weak teams) is consistent across seasons. The 2021-22 season’s second component correlates with the global third component (-0.79), suggesting that season has a unique secondary pattern that shows up on a different global axis.

mfa.partial_correlations_

This can be visualized with plot_partial. Each arrow represents one group’s PCA axis projected onto the global MFA plane. Arrows close to the unit circle indicate a strong correlation. The plot clearly shows all three seasons’ first axis (dim 0) clustering on the right side of the circle — confirming they share the same dominant structure.

mfa.plot_partial(x_component=0, y_component=1)

Partial contributions show how much each group’s PCA axes contribute to each global MFA component. They sum to 100% per component across all group axes.

mfa.partial_contributions_.style.format('{:.0%}')