This is a repo for SenSys 2025 paper: " MMBind: Unleashing the Potential of Distributed and Heterogeneous Data for Multimodal Learning in IoT ".
The program has been tested in the following environment:
- Python 3.9.7
- Pytorch 1.8.1
- torchvision 0.9.1
- sklearn 0.24.2
- opencv-python 4.5.5
- numpy 1.20.3
- Stage 1 of MMbind:
- pairing incomplete data with shared modalities;
- Stage 2 of MMbind:
- weighted contrastive learning with heterogeneous data.
- Cross-subject Evaluation:
- Cross-dataset Evaluation:
- Training on binded MotionSense and Shoaib datasets, testing on RealWorld dataset.