ViLLA-MMBench Documentation

MovieLens Module (movielens.py)

Functions:

• downloadMovieLens(url, dest, VERBOSE=True): Downloads MovieLens dataset files
• loadGenres(download_path_prefix: str, DATASET: str) -> pd.DataFrame: Loads genre information for movies
• prepareML(config: dict): Prepares MovieLens dataset with specified configurations, including k-core filtering and train-test splitting

Modalities Module (modalities.py)

Function:

• prepareModalities(config: dict, train_df: pd.DataFrame, test_df: pd.DataFrame)
  • Loads and processes text, visual, and audio embeddings
  • Merges different modalities
  • Handles NaN/Inf values
  • Creates modality-specific image and feature objects
  • Implements dimensionality reduction (PCA, CCA)

Audio Module (audio.py)

Constants:

AUD_FILE_MAP = {
    "mmtf_corr": "fused_llm_mmtf_audio_correlation.csv",
    "mmtf_delta": "fused_llm_mmtf_audio_delta.csv",
    "mmtf_log": "fused_llm_mmtf_audio_log.csv",
    "mmtf_spect": "fused_llm_mmtf_audio_spectral.csv",
    "i_ivec": "i-vector/fused_llm_mmtf_audio_IVec_splitItem_fold_1_gmm_128_tvDim_20.csv"
}

Functions:

• readAudioCsv(url): Reads audio feature CSV files
• loadAudio(config: dict): Loads and processes audio embeddings

Text & Visual Modules

Text Module (text.py):

• loadText(config: dict): Loads and processes textual embeddings

Visual Module (visual.py):

• loadVisual(config: dict): Loads and processes visual embeddings