Academic Articles & Research Papers

An Explainable Multi-Modal Deep Learning Framework for Investigating the Influences of Dynamic Environmental Exposure on Momentary Stress.

Multi-label Contextual Operator Inference in Knowledge Acquisition for Building Generalization: A Multi-modal Learning Approach.

Pixel-Accurate Registration of Photogrammetric Images and LiDAR in a Hybrid Airborne Oblique Imaging System

Hybrid airborne imaging systems integrating oblique photogrammetric cameras and LiDAR sensors provide complementary geometric and radiometric information for high-fidelity 3D reconstruction. While LiDAR offers stable global geometric accura...

Fusing content and social relationships: a multi-modal heterogeneous graph transformer approach for social bot detection

Abstract Social bots pose a significant threat to online platforms, demanding robust methods to detect their increasingly complex behaviors. This paper introduces MM-HGT-Bot, a multi-modal framework that advances the field by operationalizi...

Hypergraph Neural Networks

In this paper, we present a hypergraph neural networks (HGNN) framework for data representation learning, which can encode high-order data correlation in a hypergraph structure. Confronting the challenges of learning representation for comp...

Expanding the galactose oxidase substrate repertoire via a multi-modal computational strategy.

An empirical study of LLaMA3 quantization: from LLMs to MLLMs

Abstract The LLaMA family, a collection of foundation language models ranging from 7B to 65B parameters, has become one of the most powerful open-source large language models (LLMs) and the popular LLM backbone of multi-modal large language...

A Complete System for Automated Semantic–Geometric Mapping of Corrosion in Industrial Environments

Corrosion, a naturally occurring process leading to the deterioration of metallic materials, demands diligent detection for quality control and the preservation of metal-based objects, especially within industrial contexts. Traditional tech...

Attention is All Large Language Model Need

With the advent of the Transformer, the attention mechanism has been applied to Large Language Model (LLM), evolving from initial single- modal large models to today's multi-modal large models. This has greatly propelled the development of ...

Multi-scale attention and collaborative fusion for enhanced RGB-D salient object detection in weld quality inspection

Abstract Salient object detection (SOD) plays a crucial role in computer vision by mimicking human visual attention to identify and segment the most prominent objects in a scene. The advent of depth sensing technologies has propelled RGB-D ...

Multi-Modal Image Fusion Based on Matrix Product State of Tensor

Multi-modal image fusion integrates different images of the same scene collected by different sensors into one image, making the fused image recognizable by the computer and perceived by human vision easily. The traditional tensor decomposi...

Deep Learning for Sensor-based Human Activity Recognition

The vast proliferation of sensor devices and Internet of Things enables the applications of sensor-based activity recognition. However, there exist substantial challenges that could influence the performance of the recognition system in pra...

Machine learning for precision medicine

Precision medicine is an emerging approach to clinical research and patient care that focuses on understanding and treating disease by integrating multi-modal or multi-omics data from an individual to make patient-tailored decisions. With t...

A unified multi-modal foundation model for end-to-end emergency care.