Federated Foundation Model Fine-Tuning

This repository contains the implementations of three works on efficient federated fine-tuning of foundation models using LoRA under heterogeneous client environments.

Papers

1. Fed-PLoRA

Heterogeneous Federated Fine-Tuning with Parallel One-Rank Adaptation
Zikai Zhang, Rui Hu, and Jiahao Xu
ICLR 2026

2. Fed-HeLLo

Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocation
Zikai Zhang, Ping Liu, Jiahao Xu, and Rui Hu
IEEE TNNLS 2025

3. Fed-pilot

Fed-pilot: Optimizing LoRA Allocation for Efficient Federated Fine-Tuning with Heterogeneous Clients
Zikai Zhang, Rui Hu, Ping Liu, and Jiahao Xu
arXiv 2024

Key Features

• Parameter-efficient federated fine-tuning
• Heterogeneous client support
• Communication-efficient training
• Scalable foundation model adaptation
• Robust learning under non-IID data

Citation

@inproceedings{zhang2026plora,
  title={Heterogeneous Federated Fine-Tuning with Parallel One-Rank Adaptation},
  author={Zhang, Zikai and Hu, Rui and Xu, Jiahao},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026}
}

@article{zhang2025fedhello,
  title={Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocation},
  author={Zhang, Zikai and Liu, Ping and Xu, Jiahao and Hu, Rui},
  journal={IEEE Transactions on Neural Networks and Learning Systems},
  year={2025}
}

@article{zhang2024fedpilot,
  title={Fed-pilot: Optimizing LoRA Allocation for Efficient Federated Fine-Tuning with Heterogeneous Clients},
  author={Zhang, Zikai and Hu, Rui and Liu, Ping and Xu, Jiahao},
  journal={arXiv preprint arXiv:2410.10200},
  year={2024}
}