Yixin Zhu

Assistant Professor

PKU Institute for AI

Interests

Computer Vision
Artificial Intelligence
Human-Robot Teaming

Education

Ph.D. in statistics, 2018
UCLA
M.S. in computer science, 2013
UCLA
B.Eng. in software engineering, 2012
Xi'an Jiaotong University

Latest

[SIGGRAPHAsia24] Autonomous Character-Scene Interaction Synthesis from Text Instruction
[IROS24] Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations
[IROS24] PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environments
[ECCV24] Zero-Shot Image Feature Consensus with Deep Functional Maps
[CogSci24] Evaluating and Modeling Social Intelligence: A Comparative Study of Human and AI Capabilities
[ScienceAdvances24] Human-level few-shot concept induction through minimax entropy learning
[CVPR24] Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance
[CVPR24] AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents
[CVPR24] Scaling Up Dynamic Human-Scene Interaction Modeling
[RA-L24] Grasp Multiple Objects with One Hand
[3DV24] Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture
[ICLR24] I-PHYRE: Interactive Physical Reasoning
[ICLR24] SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation
[ICLR24] Neural-Symbolic Recursive Machine for Systematic Generalization
[NeurIPS23] ProBio: A Protocol-guided Multimodal Dataset for Molecular Biology Lab
[NeurIPS23] Active Reasoning in an Open-World Environment
[NeurIPS23] ChimpACT: A Longitudinal Dataset for Understanding Chimpanzee Behaviors
[NeurIPS23] Evaluating and Inducing Personality in Pre-trained Language Models
[NeurIPS23] Interactive Visual Reasoning under Uncertainty
[ICCV23] Full-Body Articulated Human-Object Interaction
[ICCV23] X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events
[IROS23] Learning a Causal Transition Model for Object Cutting
[IROS23] Part-level Scene Reconstruction Affords Robot Interaction
[IROS23] Sequential Manipulation Planning for Over-actuated Unmanned Aerial Manipulators
[ACL23Demo] PersLEARN: Research Training through the Lens of Perspective Cultivation
[ICML23] MEWL: Few-shot multimodal word learning with referential uncertainty
[ICML23] On the Complexity of Bayesian Generalization
[AIR22] Artificial Social Intelligence: A Comparative and Holistic View
[CVPR23] Diffusion-based Generation, Optimization, and Planning in 3D Scenes
[ICLR23] Understanding Embodied Reference with Touch-Line Transformer
[ICLR23] A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics
[ICRA23] GenDexGrasp: Generalizable Dexterous Grasping
[ICRA23] Rearrange Indoor Scenes for Human-Robot Co-Activity
[Engineering23] A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps
[NeurIPS22] On the Learning Mechanisms in Physical Reasoning
[NeurIPS22] Emergent Graphical Conventions in a Visual Communication Game
[NeurIPS22] HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes
[IJCV22] Scene Reconstruction with Functional Objects for Robot Autonomy
[ScienceRobotics22] In situ bidirectional human-robot value alignment
[ECCV22] Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning
[ECCV22Workshop] PartAfford: Part-level Affordance Discovery from 3D Objects
[IROS22] Sequential Manipulation Planning on Scene Graph
[IROS22] Downwash-aware Control Allocation for Over-actuated UAV Platforms
[RA-L/IROS22] Understanding Physical Effects for Effective Tool-use
[ICML22] Latent Diffusion Energy-Based Model for Interpretable Text Modeling
[JCB2022] Sharing Rewards Undermines Coordinated Hunting
[CogSci22] What Is the Point? A Theory of Mind Model of Relevance
[RA-L/ICRA22] Object Gathering with a Tethered Robot Duo
[AAIL21] Patching interpretable And-Or-Graph knowledge representation using augmented reality
[NeurIPS21] Unsupervised Foreground Extraction via Deep Region Competition
[RA-L21] Synthesizing Diverse and Physically Stable Grasps with Arbitrary Hand Structures using Differentiable Force Closure Estimator
[ICCV21] YouRefIt: Embodied Reference Understanding with Language and Gesture
[ICCV21] Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds
[IROS21] Consolidating Kinematic Models to Promote Coordinated Mobile Manipulations
[IROS21] Efficient Task Planning for Mobile Manipulation: a Virtual Kinematic Chain Perspective
[IROS21] Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene
[CogSci21] Individual vs. Joint Perception: a Pragmatic Model of Pointing as Communicative Smithian Helping
[ICLR21Workshop] HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving
[ACL-Findings21] GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning
[CVPR21] Learning Triadic Belief Dynamics in Nonverbal Communication from Videos
[CVPR21] ACRE: Abstract Causal Reasoning Beyond Covariation
[CVPR21] Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution
[ICRA21] Reconstructing Interactive 3D Scene by Panoptic Mapping and CAD Model Alignments
[ICRA21] Congestion-aware Multi-agent Trajectory Prediction for Collision Avoidance
[IJNME21] Lagrangian‐Eulerian Multi‐Density Topology Optimization with the Material Point Method
[ECCV20] LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities
[IROS20] Human-Robot Interaction in a Shared Augmented Reality Workspace
[IROS20] Graph-based Hierarchical Knowledge Representation for Robot Task Transfer from Virtual to Physical World
[Engineering20] Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
[SIGGRAPH20] A Massively Parallel and Scalable Multi-GPU Material Point Method
[SIGGRAPH20] IQ-MPM: An Interface Quadrature Material Point Method for Non-sticky Strongly Two-Way Coupled Nonlinear Solids and Fluids
[ICRA20] Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs
[ICRA20] Congestion-aware Evacuation Routing using Augmented Reality Devices
[ScienceRobotics19] A tale of two explanations: Enhancing human trust by explaining robot behavior
[AAAI20] Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning
[AAAI20] Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning
[NeurIPS19] Learning Perceptual Inference by Contrasting
[NeurIPS19] PerspectiveNet: 3D Object Detection from a Single RGB Image via Perspective Points
[ICCV19] Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense
[IROS19] Learning Virtual Grasp with Failed Demonstrations via Bayesian Inverse Reinforcement Learning
[CogSci19] Decomposing Human Causal Learning: Bottom-up Associative Learning and Top-down Schema Reasoning
[CVPR19] RAVEN: A Dataset for Relational and Analogical Visual Reasoning
[TURC19] VRGym: A Virtual Testbed for Physical and Interactive AI
[ICRA19] High-Fidelity Grasping in Virtual Reality using a Glove-based System
[ICRA19] Self-Supervised Incremental Learning for Sound Source Localization in Complex Indoor Environment
[AAAI19] MetaStyle: Three-Way Trade-Off Among Speed, Flexibility and Quality in Neural Style Transfer
[AAAI19] Mirroring without Overimitation: Learning Functionally Equivalent Manipulation Actions
[NeurIPS18] Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation
[ECCV18] Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image
[IJCV18] Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars
[CogSci18] Human Causal Transfer: Challenges for Deep Reinforcement Learning
[CVPR18] Human-centric Indoor Scene Synthesis Using Stochastic Grammar
[SIGGRAPH18] A Moving Least Squares Material Point Method with Displacement Discontinuity and Two-Way Rigid Body Coupling
[ICRA18] Interactive Robot Knowledge Patching using Augmented Reality
[ICRA18] Unsupervised Learning using Hierarchical Models for Hand-Object Interactions
[AAAI18] Tracking Occluded Objects and Recovering Incomplete Trajectories by Reasoning about Containment Relations and Human Actions
[IROS17] A Glove-based System for Studying Hand-Object Manipulation via Joint Pose and Force Sensing
[IROS17] Feeling the Force: Integrating Force and Pose for Fluent Discovery through Imitation Learning to Open Medicine Bottles
[CogSci17] Consistent Probabilistic Simulation Underlying Human Judgment in Substance Dynamics
[TVCG16] The Martian: Examining Human Physical Judgments Across Virtual Gravity Fields
[SIGGRAPHAsia16Workshop] A Virtual Reality Platform for Dynamic Human-Scene Interaction
[IJCAI16] What is Where: Inferring Containment Relations from Videos
[CVPR16] Inferring Forces and Learning Human Utilities From Videos
[CogSci16] Probabilistic Simulation Predicts Human Performance on Viscous Fluid-Pouring Problem
[CVPR15] Understanding Tools: Task-Oriented Object Modeling, Learning and Recognition
[CogSci15] Evaluating Human Cognition of Containing Relations with Physical Simulation