Published Paper Details:

Diabetes mellitus is a complex, chronic condition that requires early and accurate diagnosis to prevent long-term complications. Traditional machine learning models rely on unimodal inputs, such as retinal images or blood test reports, which may not capture the full clinical context of the patient. This research proposes a comprehensive multimodal deep learning framework that integrates three heterogeneous data sources: retinal fundus images, structured blood test reports, and unstructured clinical notes extracted from electronic health records (EHRs). The visual data is processed using Convolutional Neural Networks (CNNs) to extract spatial features from retinal images, while tabular lab data is modeled using feed-forward neural networks. For clinical text, we leverage the Bidirectional Encoder Representations from Transformers (BERT) to capture semantic and contextual representations of patient history. A late fusion strategy is employed to combine modality-specific embeddings and generate a unified prediction for diabetes diagnosis and risk scoring. Our model is trained and evaluated on a multimodal dataset comprising real-world patient records. Preliminary results demonstrate that the proposed approach significantly outperforms unimodal models in terms of accuracy, F1-score, and AUC. The study highlights the potential of multimodal AI systems to provide more robust, explainable, and patient-specific diagnostic support in clinical settings. Future work will focus on improving interpretability and adapting the model for real-time deployment in healthcare systems.