About me

Hello, thanks for visiting my webpage. If you want to know a bit more about me as well as my research interests and experience, this is the right place for you.



Introduction and Research Interests


My name is Pramit and I am currently a DPhil (PhD) student in the Department of Engineering Science at University of Oxford. I am fortunate to be supervised by Prof. Alison Noble. I am working at the intersection of medical image analysis/computer vision, and deep learning with a special focus on solving challenges in Federated Learning, Semi-supervised and Self-supervised Learning, Multimodal Learning, Domain adaptation, and Class-imbalanced Learning. My DPhil is funded by the VisualAI project. Here is my university profile.

I am a part of the Oxford Biomedical Image Analysis (BioMedIA) cluster and I also work closely in collaboration with Prof. Konstantinos Kamnitsas. I am also acting as the Biomed Student Representative of the Graduate Joint Consultative Committee (GJCC) that provides a forum for discussion and negotiation between the students and staff concerning a range of matters that arise throughout the year.


My recent work IsoFed on Semi-supervised Federated Learning (shown below) has been published in MICCAI 2023 and recognized as one of the top 20 Highlights of the conference.


Teaching


I am very passionate about teaching. I have taught (and am teaching) many undergraduate courses in the Department of Engineering Science. I am one of the Lead Tutors of the third year undergraduate courses offered by the department. Besides, I spend my weekends as a research mentor for undergraduate and pre-college students, guiding them to get started with and gain deeper understanding of machine learning topics. Some of the very fundamental machine learning works with my students can be found here and here.

Entrepreneurship


I cofounded coooool.space on Cross-modal information search and management in the AI era. We focus on retrieving cross-modal information from different modalities, particularly audio and video, given audio and text query prompts. Please get in touch if you want to talk more about our coooool work.

Masters


Prior to my DPhil, I received my Masters (MASc) in Electrical and Computer Engineering from the University of British Columbia (UBC), Vancouver, Canada. I was a MITACS Globalink Graduate Fellow in the Human Communication Technologies Laboratory (HCT Lab), supervised by Prof. Sidney Fels, where I worked on integrating Human-Computer Interaction (HCI) and Deep Learning (DL) for around 3.5 years.

My Masters research particularly centered around understanding speech and neuromotor control pathway for speech production using deep learning. In the first phase of my MASc, I worked towards decoding imagined speech tokens from EEG signals and published three papers (in INTERSPEECH, ICASSP, and AAAI).

Next, I moved on towards understanding speech from tongue and vocal tract movements in medical videos including Ultrasound and MRI and published two papers (in INTERSPEECH and MICCAI). My research on the development of Ultrasound-based silent speech interfaces to generate synthetic speech for individuals who had their larynx removed, was highly appreciated. Consequently, I was awarded the prestigious Young Scientist Award by MICCAI Society in 2020.

Real-time MRI video-based automatic speech identification

Ultra2Formant Net generates speech formant frequency from ultrasound videos

In order to find a joint latent representation between the articulatory and acoustic domain for vowel sounds, I introduced invertible neural network models, while simultaneously preserving the respective domain-specific features. The architecture of the proposed model is shown below.

The main focus of my Master’s thesis was reducing human motor control effort using deep learning while leveraging categorical perceptual constraints. I introduced a perceptual mapping from speech-like complex multiple degree-of-freedom (DOF) movement of the hand to a controllable formant space, that could leverage categorical perceptual constraints for reducing the difficulty level of hand motor control tasks. The perceptual network was modeled using long short term memory networks (LSTMs) aimed at optimizing a connectionist temporal classification (CTC) loss function. Our motor control mapping network consisted of a graph convolutional neural network (GCNN) combined with LSTM encoder-decoder network, that was regularized with the help of the trained perception model. The mapping allowed the user’s hand to generate continuous kinematic trajectories at a reduced effort by altering the complexity of the task space. This was essentially a human-in-the-loop system where the user played the role of an expert assessor in evaluating the degree to which the network is capable of reducing the complexity of task space.

I also explored different ways of controlling a Vocal Tract Model by developing various interesting physical and virtual interfaces.

During my masters, I also worked in close collaboration with Prof. Bryan Gick, FRSC, the director of UBC Interdisciplinary Speech Research Lab (ISRL) and the-then head of Department of Linguistics, UBC, as well as Prof. Muhammad Abdul-Mageed, the Canada Research Chair in Natural Language Processing and director of UBC Deep Learning and Natural Language Processing (DL-NLP) Lab. Besides being a graduate research assistant, I also acted as the Treasurer and Executive Committee Member of the ECE Graduate Student Association (ECEGSA) for three consecutive terms (2018-2019, 2019-2020, 2020-2021).

I received the UBC Faculty of Applied Science Graduate Award (2018-2019) for outstanding academic and research achievements from the Department of Electrical and Computer Engineering (ECE) of University of British Columbia.


Before joining the HCT Lab, I was a MITACS Globalink Research Intern (2016) in the Faculty of Medicine and Dentistry, Department of Radiology and Diagnostic Imaging, University of Alberta, Edmonton, advised by Term Clin Professor Dr. Lawrence Le where I worked on developing fully automated hard and soft tissue delineation scheme. I proposed a Guided Active Contour Segmentation with level set formulation in Dento-periodontal Ultrasound images.

Undergraduate Study


I obtained my Bachelor of Engineering (B.E) degree in Electrical Engineering from Jadavpur University, Kolkata, India, on Dec, 2016. My final year project was on stereo image matching and depth map generation, supervised by Prof. Amitava Chatterjee. I published a preliminary paper on this at IEEE INDICON 2016.

Besides, during my undergraduate degree, I also worked on image reconstruction, noise estimation, and several engineering design projects. My undergraduate research bagged me best paper awards at IEEE CALCON 2014, IEEE INDICON 2015, and Best White Paper Award 2014 awarded by Schneider Electric Pvt. Ltd.

During my junior year, I interned on the 3D Medical Image Processing, in the Electronics and Electrical Communication Engineering Department at the Indian Institute of Technology (IIT Kgp), jointly supervised by Prof. Sudipta Mukhopadhyay and Prof. Ashis Kumar Dhara. I worked towards developing Differential geometry-based techniques for quantitative characterization of boundary roughness of pulmonary nodules in lung CT images and co-authored a journal paper in IJCARS, Springer.

In my senior year, I worked as a research intern in the Tomography Lab, Department of Electrical Engineering, at the Indian Institute of Science (IISc Bangalore) under Prof. Kasi Rajgopal. I worked towards developing a novel algorithm utilizing bee’s swarm intelligence, and its heuristics based on quality and quantity of food sources (nectars) to generate a variable density sampling (VDS) scheme for compressive sampling (CS)-based fast MRI data acquisition. The MRI Reconstruction paper was published in IEEE TENCON 2016.

During my Undergraduate days, I also acted as the Class Representative (CR) as well as the Member of Anti-Ragging Committee, Electrical Engineering Department, Jadavpur University (2012-2016).

High School


I was awarded the Ministry of Human Resource Development Scholarship (2012 - 2016) for securing 27th rank in Higher Secondary Examination out of 7,00,000 candidates (approx.)