Jeya Maria Jose

I am a Senior Researcher at Microsoft Research. My research currently revolves around i) developing advanced multi-modal foundation models and ii) methods for encoding clinically valuable data representations, tailored for healthcare applications. Broadly, I study Computer Vision, Machine Learning, and Health AI.

Previously, I was a postdoctoral researcher at Stanford University, working with Andrew Ng and Curtis Langlotz. I obtained my Ph.D. and M.S from Johns Hopkins University advised by Vishal M Patel .

I am glad to be recognized as an Amazon fellow . I also had the opportunity to work with research teams at Google, NVIDIA , and Adobe during my Ph.D.

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn

News

  • October, 2024 - Won the Young Scientist Impact Award from MICCAI Society.
  • August, 2024 - 1 paper accepted at WACV 2025.
  • July, 2024 - 1 paper accepted at ECCV 2024.
  • April, 2024 - 4 papers accepted at MIDL 2024.
  • February, 2024 - Serving as Area Chair for MICCAI 2024.
  • October, 2023 - 1 paper accepted at WACV 2024.
  • November, 2023 - Won the DAAD AI-Net Fellowship.
  • August, 2023 - Invited Talk at MedAI Group. Link for Slides, Video.
  • May, 2023 - Invited Talk at Amazon. Link for Slides.
  • April, 2023 - Defended my Ph.D.!! Link for Slides.
  • March, 2023 - 1 paper accepted at MIDL 2023.
  • February, 2023 - 1 paper accepted at CVPR 2023.
  • January, 2023 - 1 paper accepted at ICLR 2023.
  • October, 2022 - Won the Amazon Research Fellowship.
  • September, 2022 - Received Young Scientist Impact Award - Finalist at MICCAI 2022.
  • August, 2022 - 1 paper accepted at WACV 2023.
  • August, 2022 - Received NIH MICCAI Award 2022, which rewards early career scientists.
  • June, 2022 - 3 papers accepted at MICCAI 2022.
  • June, 2022 - Invited Talk at IEEE SPS Summer School on AI in Healthcare.
  • May, 2022 - Awarded Outstanding Automation Paper Finalist (top 3) at ICRA 2022.
  • March, 2022 - Presented a Tutorial at ISBI 2022 on medical image segmentation.
  • March, 2022 - 1 paper accepted at CVPR 2022.
  • January, 2022 - 1 paper accepted at ICRA 2022.
  • November, 2021 - 1 paper accepted in IEEE Transactions on Medical Imaging.
  • June, 2021 - 2 papers accepted at MICCAI 2021.
  • May, 2021 - Joined Adobe as a Research Intern.
  • May, 2021 - 1 paper accepted at ICIP 2021 .
  • November, 2020 - 1 paper accepted in IEEE Journal of Selected Topics in Signal Processing .
  • November, 2020 - 1 paper accepted at WACV 2021 .
  • July, 2020 - Recipient of MICCAI Student Travel Award 2020, which rewards first author students of the highest-quality MICCAI papers.
  • May, 2020 - 1 paper accepted in the IEEE Journal of Selected Topics in Signal Processing .
  • May, 2020 - 1 paper accepted at MICCAI 2020 (Early Acceptance).
  • September, 2019 - Awarded Best Student Paper award at CVIP 2019 .
  • August, 2019 - Joined Johns Hopkins University for my Ph.D with ECE fellowship.



Research

Merlin: A Vision Language Foundation Model for 3D Computed Tomography
Preprint 2024

Stanford AIMI (L.Blankemeier et al.)

Paper
MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models
ECCV 2024

Nithin GK, Jeya Maria Jose , Vishal M. Patel

Paper | Project | Code
Diffscaler: Enhancing the Generative Prowess of Diffusion Transformers
Preprint 2024

Nithin GK, Jeya Maria Jose , Vishal M. Patel

Paper
CheXagent: Towards a Foundation Model for Chest X-Ray Interpretation
Preprint 2024

Z. Chen, M. Varma, JB Delbrouck, M. Paschali, L. Blankemeier, DV Veen Jeya Maria Jose , A. Youssef, JP Cohen, EP Reis, EB Tsai, A. Johnston, C. Olsen, TM Abraham, S. Gatidis, Akshay S. Chaudhari, Curtis Langlotz

Paper | Project | Demo | Code
ReBotNet: Fast Real-time Video Enhancement
WACV 2025

Jeya Maria Jose , Rahul Garg, Andeep Toor, Xin Tong, Weijuan Xi, Andreas Lugmayr, Vishal M. Patel and Anne Menini

Paper | Project

| Code
Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training
MIDL 2024

Jeya Maria Jose , Yucheng Tang, Dong Yang, Ziyue Xu, Can Zhao, Wenqi Li, Vishal M. Patel, Bennett Landman, Daguang Xu, Yufan He and Vishwesh Nath

Paper | Weights | Pre-training Code
Auto-Generating Weak Labels for Real & Synthetic Data to Improve Label-Scarce Medical Image Segmentation
MIDL 2024

Tanvi Deshpande, Eva Prakash, Elsie Gyang Ross, Curtis Langlotz, Andrew Ng, Jeya Maria Jose

Paper | Code

Unlocking Robust Segmentation Across All Age Groups via Continual Learning
MIDL 2024

Chih-Ying Liu, Jeya Maria Jose, Camila Gonzalez, Curtis Langlotz, Andrew Ng, Sergios Gatidis

Paper
Target and Task specific Source-Free Domain Adaptive Image Segmentation
MIDL 2024

Vibashan VS*, Jeya Maria Jose* , and Vishal M. Patel

*equal contribution

Paper | Code

Self-Supervised Denoising Transformer with Gaussian Process
WACV 2024

Rajeev Yasarla, Jeya Maria Jose , Vishwanath Sindagi, Vishal M. Patel

Paper
CLIP Goes 3D: Leveraging Prompt Tuning for Language-Grounded 3D Recognition
OpenSUN3D, ICCV 2023

Deepti Hegde*, Jeya Maria Jose* , and Vishal M. Patel

*equal contribution

Paper | Code | Project

On-the-Fly Test-time Adaptation for Medical Image Segmentation
MIDL 2023

Jeya Maria Jose , Pengfei Guo, Vibashan VS, and Vishal M. Patel

Paper | Code

Ambiguous Medical Image Segmentation using Diffusion Models
CVPR 2023

Aimon Rahman, Jeya Maria Jose, Ilker H, and Vishal M. Patel

Paper | Code | Project

Interactive Portrait Harmonization
ICLR 2023

Jeya Maria Jose, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin , Jose Echevarria, Yinglan Ma, Zijun Wei, Kalyan Sunkavalli, and Vishal M. Patel

Paper | Code | Project

Fine-Context Shadow Detection using Shadow Removal
WACV 2023

Jeya Maria Jose and Vishal M. Patel

Paper

TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions
CVPR 2022

Jeya Maria Jose, Rajeev Yasarla and Vishal M. Patel

Paper | Code | Project

UNeXt: MLP-based Rapid Medical Image Segmentation Network
MICCAI 2022

Jeya Maria Jose and Vishal M. Patel

Paper | Code | Project

Simultaneous Bone and Shadow Segmentation Network using Task Correspondence Consistency
MICCAI 2022

Aimon Rahman, Jeya Maria Jose, Ilker H, and Vishal M. Patel

Paper

Orientation-guided Graph Convolutional Network for Bone Surface Segmentation
MICCAI 2022

Aimon Rahman, Chaminda Bandara, Jeya Maria Jose, Ilker H, and Vishal M. Patel

Paper

SPIN Road Mapper: Extracting Roads from Aerial Images via Spatial and Interaction Space Graph Reasoning for Autonomous Driving
ICRA 2022 (Outstanding Automation Paper Finalist)

Chaminda Bandara, Jeya Maria Jose, and Vishal M. Patel

Paper | Code

Medical Transformer: Gated Axial-Attention for Medical Image Segmentation
MICCAI 2021

Jeya Maria Jose, Poojan Oza, Ilker Hacihaliloglu, Vishal M. Patel

Paper | Code

Over-and-Under Complete Convolutional RNN for MRI Reconstruction
MICCAI 2021

Pengfei Guo, Jeya Maria Jose, Puyang Wang, Jinyuan Zhou,Shanshan Jiang, Vishal M. Patel

Paper | Code

KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation
IEEE Transactions on Medical Imaging


Jeya Maria Jose, Vishwanath Sindagi, Ilker Hacihaliloglu, Vishal M. Patel

Paper | Code | Project

Overcomplete Deep Subspace Clustering Networks
WACV 2021

Jeya Maria Jose and Vishal M. Patel

Paper | Code

Overcomplete Representations against Adversarial Videos
ICIP 2021

Shao-Yuan Lo, Jeya Maria Jose, and Vishal M. Patel

Paper

Exploring Overcomplete Representations for Single Image Deraining using CNNs
IEEE Journal of Selected Topics in Signal Processing

Rajeev Yasarla*, Jeya Maria Jose*, Vishal M. Patel

*equal contribution

Paper | Code

KiU-Net: Towards Accurate Segmentation of Biomedical Images using Over-complete Representations
MICCAI 2020 (Oral)

Jeya Maria Jose, Vishwanath Sindagi, Ilker Hacihaliloglu, Vishal M. Patel

Paper | Code | Project

Learning to Segment Brain Anatomy from 2D Ultrasound with Less Data
IEEE Journal of Selected Topics in Signal Processing

Jeya Maria Jose, Rajeev Yasarla, Puyang Wang, Ilker Hacihaliloglu, Vishal M. Patel

Paper

Tackling Multiple Visual Artifacts: Blind Image Restoration using Conditional Adversarial Networks
CVIP 2019 (Best Student Paper Award)

M Anand, A Ashwin Natraj, Jeya Maria Jose, K Subramanian, S Deivalakshmi

Paper | Code

Brain Tumor Segmentation and Survival Prediction using 3D Attention UNet
BraTS, MICCAI Workshop 2019

Mobarakol Islam, Vibashan VS, Jeya Maria Jose, Navodini Wijethilake, Uppal Utkarsh, Hongliang Ren

Paper

Glioma Prognosis: Segmentation of the Tumor and Survival Prediction using Shape, Geometric and Clinical Information
BraTS, MICCAI Workshop 2018

Mobarakol Islam, Jeya Maria Jose, Hongliang Ren

Paper | Poster

Services

Web Chair: AVSS 2021

Program Committe: DART Workshop, MICCAI 2022

Reviewer at:

Journals :

IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Medical Imaging

Elsevier Medical Image Analysis

IEEE Transactions on Circuits and Systems for Video Technology

IEEE Transactions on Geoscience and Remote Sensing

Springer International Journal on Computer Vision

Elsevier Pattern Recognition

Elsevier Computers in Biology and Medicine

Conferences :

CVPR

ICCV

ECCV

NeurIPS

ICLR

ICML

MICCAI

MIDL

WACV

ICIP

Workshops :

BrainLes, Medical Image Learning with Less Labels and Imperfect Data


source code