KosmoBots

KosmoBots

Hefei University of Technology

Our laboratory, as part of the Engineering Management and Intelligent Manufacturing (EMIM) Lab, focuses on cutting-edge research and development in embodied intelligent robotics and intelligent systems. We concentrate on realizing “task intent understanding, dynamic resource organization, and autonomous operation execution” to build intelligent robotic systems with advanced autonomous cognition and decision-making capabilities. Leveraging multimodal large models and embodied intelligence technologies, we promote the transformation of robots from single-agent execution to multi-agent collaboration and adaptive operation, creating a flexible, efficient, and intelligent robotic system architecture.

Within EMIM, we work with a diverse range of heterogeneous intelligent agents, including humanoid robots and quadruped robots, and conduct in-depth research on robot decision-making and control technologies. Building on this core technology platform, we actively explore diverse application scenarios such as intelligent manufacturing and extraterrestrial exploration, driving deep integration and broad application of robotic technologies in product manufacturing, space missions, and beyond.

🎓 Research Direction

  • Multi-Agent System
  • Embodied Intelligent System
  • Reinforcement Learning Control
  • Tactile Perception and Reconstruction

🎓 Team Members

  • Qiang Zhang (Supervisor)
  • Yanjiu Zhong (Co-supervisor)
  • Jun Li
  • Liang Peng
  • Jian Sun
  • Yang Wang
  • Shuai Chen
  • Lingfeng Li
  • Xianpeng Li
  • Xincheng Han
  • Shirong Luo
  • Jinyang Zhao
  • Yuanjun Yang
  • Chengzheng Sun
  • Chunxiao Wu
  • Bo Ouyang

📝 Research Progress

Data Collection

sym
  • To address the challenges of multi-modal, multi-scenario, and multi-task data acquisition, our lab has developed a VR-based heterogeneous robot integrated data collection system. This system combines various types of robots and sensors to efficiently collect and manage data across diverse environments and tasks.
  • It also supports training of Vision-Language-Action (VLA) models using the collected data, enhancing the agents’ perception, understanding, and decision-making capabilities in complex tasks.

Multimodal Perception-Driven Robotic Arm Decision Making

sym
  • To address complex demands in high-level cognitive decision-making, we developed a Vision-Language-Action (VLA) model that integrates visual and tactile feedback. A fine-grained multimodal semantic alignment mechanism enables effective fusion of visual and haptic information, improving the model’s understanding of environmental and task-specific details. This empowers robotic arms with intelligent decision-making and precise control across diverse scenarios, significantly enhancing execution accuracy and adaptability in complex operations.

Reinforcement Learning-Based Human-Like Motion Transfer

sym
  • To tackle challenges in low-level motion control, we designed a reinforcement learning-based Sim2Real framework for fast and robust transfer of human-like motions. By bridging simulation and real-world environments, the framework improves motion transfer efficiency and robustness. A hierarchical multi-agent tolerance reinforcement learning method was also proposed, enabling collaborative learning and fault tolerance across different agent levels. This approach accelerates learning and enhances stability and adaptability during transfer.

Flexible Skin-Enabled Tactile Sensing for Dexterous Hands

sym
  • To enhance perception and control in dexterous robotic hands, we developed an efficient tactile sensing system by integrating flexible electronic skin with dexterous hand hardware. For different application scenarios, custom tactile sensing and reconstruction algorithms were implemented, improving the accuracy of tactile perception and interpretation. By fusing tactile and visual information, the system significantly strengthens the multimodal perception and autonomous control capabilities of dexterous hands in complex environments.

Intelligent Navigation for Heterogeneous Robot Swarms

sym
  • To address navigation challenges in industrial multi-scenario environments, we designed a reinforcement learning-based algorithm for dynamic scene recognition and path planning. The system enables real-time perception and understanding of complex and changing environments. By considering differences between individual robots, it optimizes path planning strategies to achieve efficient collaborative navigation and swarm control for heterogeneous multi-agent systems.

🛠️ Ongoing Projects

Research on Lifelong Growth Mechanisms of Space Embodied Intelligence Robots Driven by Cognitive Reinforcement and System Evolution

With the advancement of deep space exploration, intelligent capabilities of space robots have become critical for on-site lunar construction. This project focuses on intelligent evolution mechanisms of space embodied robots in complex dynamic environments.

  • Building a high-fidelity lunar surface world model
  • Designing a decoupled perception-control heterogeneous computing framework for adaptability in microgravity and lunar dust environments
  • Integrating large models with expert knowledge to enhance cognitive and manipulation capabilities
  • Developing a co-evolution mechanism across structure, control, and knowledge
  • A simulation platform combining virtual and physical systems will be used for closed-loop optimization from perception to execution

Multi-Agent Collaborative Scheduling System for Self-Organizing Robotic Workcells

Traditional centralized control systems face difficulties in efficiently scheduling diverse robotic arms in increasingly complex smart manufacturing settings. This project proposes a distributed multi-agent collaboration framework tailored for "multi-product on one line" production modes.

  • High-level agents decompose production goals into sub-tasks suitable for different robotic arms
  • Each heterogeneous agent uses a digital twin to model its own state and adaptively complete tasks
  • Integration of visual perception and process knowledge graphs to support task decomposition
  • Use of graph neural networks for real-time collision prediction and motion trajectory optimization

Multimodal Large Model-Based Intelligent Motor Assembly Assistance System

In manufacturing and equipment maintenance, motor assembly is challenged by diverse parts, complex assembly relationships, and reliance on manual expertise. This project develops an intelligent assistant system combining image understanding, structural knowledge modeling, and natural language processing.

  • Automatic identification of motor diagrams
  • Extraction of key assembly triplets
  • Construction of a structured knowledge graph
  • Generation and evaluation of executable assembly/disassembly sequences
  • Robotic execution of optimized procedures

Collaborative Decision-Making for Heterogeneous Robots in Urban Warfare via Diffusion Models

Robotic teamwork is becoming vital in modern warfare scenarios. This project explores collaborative decision-making for heterogeneous robots in urban combat settings.

  • Dynamic scene understanding through diffusion models
  • Diffusion-based path planning in complex environments
  • Vision-Language-Action (VLA) models for autonomous decision-making
  • A scalable system architecture for state awareness and multi-function integration

VLA Model-Based Mobile Robotic Arm Grasping System for Aerospace Maintenance

Aircraft maintenance often involves challenges such as low lighting and confined spaces, which hinder manual cable connection tasks. This project aims to develop a mobile robotic arm system based on VLA models for intelligent cable assembly in harsh aerospace environments.

  • Multimodal perception
  • High-precision motion control
  • Integrated mobility and manipulation

🧪 Experimental Facilities

实验室设备图

🏛️ Collaborators

🏫 Academic Collaborations

  • Tsinghua University
  • Shanghai Jiao Tong University
  • Nanjing University
  • University of Alberta

🤝 Industry Partners

  • China Academy of Space Technology (CAST)
  • Chengdu Aircraft Industry Group (AVIC Chengfei)
  • Aero Engine Corporation of China (AECC)
  • China Baowu Steel Group Corporation
  • China North Industries Group Corporation Limited (Norinco Group)
  • Anhui Sanheyiyi Information Technology Co., Ltd.
  • Chery Automobile Co., Ltd.