Natural Language Processing (NLP) is one area where Large transformer-based Language Models (LLMs) have achieved remarkable progress in recent years. Also, LLMs are branching out into other fields, like robotics, audio, and medicine. Modern approaches allow LLMs to produce visual data using specialized modules like VQ-VAE and VQ-GAN, which convert continuous visual pixels into discrete…

Foundational models are large deep-learning neural networks that are used as a starting point to develop effective ML models. They rely on large-scale training data and exhibit exceptional zero/few-shot performance in numerous tasks, making them invaluable in the field of natural language processing and computer vision. Foundational models are also used in Monocular Depth Estimation…

Text-to-image (T2I) generation is a rapidly evolving field within computer vision and artificial intelligence. It involves creating visual images from textual descriptions blending natural language processing and graphic visualization domains. This interdisciplinary approach has significant implications for various applications, including digital art, design, and virtual reality. Various methods have been proposed for controllable text-to-image generation,…

Understanding the world from a first-person perspective is essential in Augmented Reality (AR), as it introduces unique challenges and significant visual transformations compared to third-person views. While synthetic data has greatly benefited vision models in third-person views, its utilization in tasks involving embodied egocentric perception still needs to be explored. A major obstacle in this…

Enhancing the receptive field of models is crucial for effective 3D medical image segmentation. Traditional convolutional neural networks (CNNs) often struggle to capture global information from high-resolution 3D medical images. One proposed solution is the utilization of depth-wise convolution with larger kernel sizes to capture a wider range of features. However, CNN-based approaches need help…

Large-scale pre-trained vision-language models, exemplified by CLIP (Radford et al., 2021), exhibit remarkable generalizability across diverse visual domains and real-world tasks. However, their zero-shot in-distribution (ID) performance faces limitations on certain downstream datasets. Additionally, when evaluated in a closed-set manner, these models often struggle with out-of-distribution (OOD) samples from novel classes, posing safety risks in…

Transformers have found widespread application in diverse tasks spanning text classification, map construction, object detection, point cloud analysis, and audio spectrogram recognition. Their versatility extends to multimodal tasks, exemplified by CLIP’s use of image-text pairs for superior image recognition. This underscores transformers’ efficacy in establishing universal sequence-to-sequence modeling, creating embeddings that unify data representation across…

One of the more intriguing developments in the dynamic field of computer vision is the efficient processing of visual data, which is essential for applications ranging from automated image analysis to the development of intelligent systems. A pressing challenge in this area is interpreting complex visual information, particularly in reconstructing detailed images from partial data.…

In the past year, large vision language models (LVLMs) have become a prominent focus in artificial intelligence research. When prompted differently, these models show promising performance across various downstream tasks. However, there’s still significant potential for improvement in LVLMs’ image perception capabilities.  Enhanced perceptual abilities for visual concepts are crucial for advancing model development and…

Artificial Intelligence (AI), particularly through deep learning, has revolutionized many fields, including machine translation, natural language understanding, and computer vision. The field of medical imaging, specifically chest X-ray (CXR) interpretation, is no exception. CXRs, the most frequently performed diagnostic imaging tests, hold immense clinical significance. The advent of vision-language foundation models (FMs) has opened new…