Fed-BLEND — Federated Conformal Prediction for VLMs
Novel federated conformal prediction method that mitigates hallucinations in federated fine-tuned vision-language models via abstention.
Problem
Federated fine-tuned VLMs hallucinate confidently, and naive conformal calibration breaks under non-IID client distributions.
Approach
Bin calibration data by predictive entropy, then shrink local per-bin quantiles toward a global anchor with a sample-size-aware weight (Fed-BLEND).
Architecture
5 non-IID clients · LoRA fine-tuned Qwen2.5-VL-3B · per-client entropy-binned calibration · global anchor aggregation · abstention head.
Results
Hallucination 13.05% → 3.79%, useful-answer rate 76.63%, Precision@Commit 95.29%.
Lessons learned
Entropy binning + anchor shrinkage is more robust than per-client conformal scores under non-IID heterogeneity.
Next
GCTAF — Time-Series Forecasting & Flare Risk Classification
Attention-based forecasting of magnetic field trajectories combined with supervised contrastive learning for solar flare classification.
Skin Lesion Classification via Dual-Weighted Learning
Imbalanced medical image classification using MobileViT v2 with dual-weighted learning — inverse square-root sampling + ENS loss — and Grad-CAM heatmaps for clinical interpretability.