Large Language Models (LLMs) have proven their impressive instruction-following capabilities, and they can be a universal interface for various tasks such as text generation, language translation, etc. These models can be extended to multimodal LLMs to process language and other modalities, such as Image, video, and audio. Several recent works introduce models that specialize in…

The performance of multimodal large Language Models (MLLMs) in visual situations has been exceptional, gaining unmatched attention. However, their ability to solve visual math problems must still be fully assessed and comprehended. For this reason, mathematics often presents challenges in understanding complex concepts and interpreting the visual information crucial for solving problems. In educational contexts…

While humans can easily infer the shape of an object from 2D images, computers struggle to reconstruct accurate 3D models without knowledge of the camera poses. This problem, known as pose inference, is crucial for various applications, like creating 3D models for e-commerce and aiding autonomous vehicle navigation. Existing techniques relying on either gathering the…

VLMs are potent tools for grasping visual and textual data, promising advancements in tasks like image captioning and visual question answering. Limited data availability hampers their performance. Recent strides show that pre-training VLMs on larger image-text datasets improves downstream tasks. Yet, creating such datasets faces challenges: scarcity of paired data, high curation costs, low diversity,…

Deep Neural Networks (DNNs) excel in enhancing surgical precision through semantic segmentation and accurately identifying robotic instruments and tissues. However, they face catastrophic forgetting and a rapid decline in performance on previous tasks when learning new ones, posing challenges in scenarios with limited data. DNNs’ struggle with catastrophic forgetting hampers their proficiency in recognizing previously…

Text-to-image diffusion models are among the best advances in the field of Artificial Intelligence (AI). However, there are constraints associated with personalizing existing text-to-image diffusion models with various concepts. The current personalization methods are not able to extend to numerous ideas consistently, and it attributes this problem to a possible mismatch between the simple text…

The pursuit of high-fidelity 3D representations from sparse images has seen considerable advancements, yet the challenge of accurately determining camera poses remains a significant hurdle. Traditional structure-from-motion methods often falter when faced with limited views, prompting a shift towards learning-based strategies that aim to predict camera poses from a sparse image set. These innovative approaches…

In the ever-evolving domain of remote identification technologies, gait recognition stands out for its unique capacity to identify individuals from a certain distance without requiring direct engagement. This cutting-edge approach leverages the distinctive walking patterns of each person, offering a seamless integration into surveillance and security systems. Its non-intrusive nature distinguishes it from more conventional…

Image Quality Assessment (IQA) is a method that standardizes the evaluation criteria for analyzing different aspects of images, including structural information, visual content, etc. To improve this method, various subjective studies have adopted comparative settings. In recent studies, researchers have explored large multimodal models (LMMs) to expand IQA from giving a scalar score to open-ended…

Almost all forms of biological perception are multimodal by design, allowing agents to integrate and synthesize data from several sources. Linking modalities, including vision, language, audio, temperature, and robot behaviors, have been the focus of recent research in artificial multimodal representation learning. Nevertheless, the tactile modality is still mostly unexplored when it comes to multimodal…