Brief Description of Research

My research interests cover intelligent systems in general and span robotics, haptics, computer vision, and artificial intelligence. From spatial/geometric reasoning, planning, and computation in robotics, physical simulation, and haptics, to learning (Bayesian and neural networks), and to optimization with evolutionary computation and other heuristic methods.

Research Areas

: Contact State Space: Automatic Generation, Search, and Motion Planning
: Haptic Simulation and Display of Contact States and Compliant Motion
: Automatic Contact Recognition in the Presence of Uncertainties
: Contact-related Computation in Virtual Environments
: Real-time Adaptive Motion Planning in Dynamic Environments with Unknowns and Unforseen Changes
: Real-time Adaptive Motion Planning for Continuum Manipulation
: Replanning for Robot Motion in the Presence of Uncertainties
: Semantic Video Classification, Indexing, and Retrieval
: Cooperative Haptic Manipulation of Virtual Objects
: Adaptive Evolutionary Planner/Navigator (EP/N)
: Structure Study of Neural Networks for Continuous Function Approximation
: Sequencing Optimization

Contact State Space: Automatic Generation, Search, and Motion Planning

Information of contact state topology, geometry, and physics among objects and motion constrained by contacts is fundamental to many applications, ranging from the real world (such as robotic operations), the interaction between the real and virtual world (such as haptics interaction), to the virtual world (such as CAD/CAM, virtual prototyping, physically-based simulation/animation). Often such information is manually input to a system by human operators, which is tedious at best and rather impossible for complex situations. Automatic generation of contact state information from information of objects (such as CAD models) is thus highly desirable.

From the perspective of robotics, many robotic manipulation tasks, especially automatic assembly tasks, require motion of objects in contact in order to reduce uncertainties. Automatic planning and executing such motion is necessary. We are interested in both the problem of automatic generation of contact states and the problem of contact motion planning. The two problems are intertwined. Our approach reduces the complexity by studying the topological and geometrical characteristics of contact states in the physical space of objects. With this approach, we focus on the following related subproblems: (1) properly represent discrete contact states and automatically generate the states, state transitions to obtain the contact state space; (2) investigate criteria and techniques for automatic planning of contact motions in terms of contact state transitions and contact motion within a contact state. We have achieved significant results regarding both problems for polyhedral objects. See related publications. The research has been funded by National Science Foundation (NSF) under grant IIS-9700412 (with Co-PI Robert Sturges).

We have also extended the research to contact states between general objects, including objects with curved surfaces, articulated objects, and certain deformable objects. This important extension has been funded by NSF under grant IIS-0328782. See related publications for results.

For robotics applications, theoretical concepts will be validated by experimental implementations and executions. We have collaborated with Prof. Robert Sturges of Virginia Tech on experimental verification of principal contacts and contact formations. More recently, we have started collaboration with Prof. Joris De Schutter and Prof. Herman Bruyninckx of KU Leuven of Belgium on execution of compliant motion plans and on-line (re)planning of such motions. Some results have been achieved. See related publications.

We are starting a collaborative project with General Motors for robotic surface assembly, extending the above work on contact states and compliant motion.

Haptic Display of Contact States and Compliant Motion

The word "haptic" means "of or relating to or proceeding from the sense of touch" (http://www.dict.org/). Effective haptic interaction is increasingly found useful in a wide range of potential applications, from universal access (to computers and computer-assisted learning by people of disability), to virtual training of surgeons, to virtual assembly, virtual prototyping, virtual and remote collaboration, telerobotics, and to computer assisted art and entertainment. Various aspects of haptic interaction have been studied by researchers, of which one major area is haptic rendering: to simulate and render the force or force and moment exerted to or felt by the human operator via a haptic device. This essentially requires the simulation of contact force and moment between the virtual object/tool ``held" by the operator via the haptic device and other objects in the virtual environment.

Our research is focused on haptic rendering involving complex contact states and compliant motion, not only between rigid objects but between one rigid and one deformable object. We have developed general methodologies for real-time (kHz) and realistic force/torque and graphic modeling and rendering. This research has been funded by NSF under grant IIS-0328782.

We have also investigated applications in assembly operations, with specific focus on optical fiber assembly, which involves assembly of deformable cable and micro/nano assembly of bare fiber.

See related publications.

Automatic Contact Recognition in the Presence of Uncertainties

Automatic contact recognition is crucial for part-mating or assembly tasks where both task descriptions and operations are contact-based, especially because of uncertainties. First, due to uncertainties, unintended contact between the part held (by the manipulator) and other parts in the environment can occur. Thus it is necessary to be able to recognize such a contact and to distinguish unintended contacts from intended ones, such as the contacts which define the goal state of an assembly. Secondly, the recognition task itself is also much complicated by the presence of uncertainties, even if the environment can be well controlled and structured in the sense that the models of all objects and fixtures are known, and their locations are either fixed or can be sensed. Such recognition task in the presence of uncertainties is especially difficult if the objects in contact are nonconvex.

Using different sensors to compensate for the uncertainties of one another is essential to accurate contact recognition, which in turn, is essential for task state recognition and for devising error recovery strategies as different contact situations may require different recovery motions (see replanning). A few researchers focused on using force sensing to verify contact states. I did some preliminary work on using vision sensing to verify contact states. These methods are all based on hypotheses-and-tests. Clearly, a key problem is how to obtain effectively the initial contact hypotheses in the first place. My approach is to generate the possible contact states based on the sensed location of contacting objects and the location uncertainties. See related publications.

This research brings out many interesting theoretical and practical issues, a lot of them still wait to be explored.

Contact-related Computation in Virtual Environment

In the process of simulating assembly motions, we have developed a useful algorithm for deriving contact state manifolds for polyhedra from distance computation based on Gilbert-Johnson-Keerthi algorithm and a useful algorithm for computing rotational distances for objects already in contact. We have also developed an algorithm for identifying geometrically valid contact formations based on the information of contact formation graphs.

More recently, we have developed real-time algorithms for computing exact minimum distances and multiple contacts between general non-convex objects with curved surfaces of parametric representations. Our novel hybrid approach can achieve the update rate of 1KHz. It is well suited for high-fidelity interactive haptic applications, dynamic simulations, and contact motion planning. See related publications.

Real-time Adaptive Motion Planning in Dynamically Unknown Environments

This line of research includes the following major work: (1) A novel planning approach, RAMP, to conduct real-time, optimized, simultaneous path/trajectory for robots of high degrees of freedom (DOFs), such as mobile manipulators, in dynamic environments with moving obstables of unknown motions. The approach is applicable not only to a single mobile manipulator but also to multiple mobile manipulators sharing the same task environment. (2) A novel approach to perceiving guaranteed collision-free continuous trajectories for high-DOF robots in dynamic environments with moving obstacles of unknown motion or even unknown obstacles. (3) A hybrid stochestic and deterministic appraoch to multi-robot pursuit and evasion in unknown environments. See related publications. This research has been funded by NSF.

Real-time Adaptive Motion Planning for Continuum Manipulation

Recently we have started a collaborative research with Prof. Ian Walker of Clemson University on manipulation with a continuum manipulator, such as an elephant trunk robot, investigating real-time adaptive motion planning and control for autonomous grasping using a continuum robot in unstructured environments. See related publications. The research is being funded by NSF.

Replanning for Robot Motion in the Presence of Uncertainties

Various uncertainties, such as sensing, modeling, and motion uncertainties, can be crucial enough to make a low-tolerance robotic assembly task fail. Thus, an important problem is how to enable a robot to accomplish an assembly task successfully in spite of the inevitable uncertainties.

There are generally three types of approaches to tackle the problem. One is to model the effect of uncertainties in the off-line planning process. In particular, Lozano-Perez, Mason and Taylor proposed the concept of preimages in configuration space to compute motion strategies that would not fail in the presence of uncertainties.

Another approach is to use task-dependent knowledge to obtain efficient strategies for specific tasks rather than focusing on generic strategies independent of tasks. Many strategies dealing with different types of insertion tasks fall into this category. The task-dependent knowledge can be learned explicitly by knowledge-based systems or implicitly via neural networks. The uncertainties are dealt with by either active control of forces and positions or passive reaction to errors.

Since I started working on a different approach since I was a Ph.D. student at the University of Michigan (Ann Arbor), called the replanning approach . The approach is to rely on on-line sensing to identify errors caused by uncertainties in a motion process and to replan the motion in real-time based on sensed information. In particular, I proposed a general replanning scheme, which consists of patch-planning to repair error situations caused by uncertainties and motion strategy planning to regulate motions in ways to reduce the effect of uncertainties. See the diagram below.

The replanning scheme works as follows. Given a nominal motion plan for a part held by a manipulator (for a part-mating task) which assumes no uncertainty, the motion strategy planner will generate a control strategy to execute such a plan. If the plan is stopped by an unplanned collision between the held part and the environment due to uncertainties, the patch-planner will be actived to generate a patch-plan which is to guide the held part back to its nominally planned course. A patch-plan is generated based on the analysis of the contact situation at the collision and designed to incorporate contact motions to reduce the effect of uncertainties. Once a patch-plan is generated, the motion strategy planner will generate a control strategy to execute the patch-plan, until either another unintended contact occurs, in which case the patch-planner will be activated again, or a goal state of the nominal plan is achieved (which is most likely also a contact state).

The replanning scheme is a compromise between purely model-based off-line planning (such as the preimage scheme) and rather "unconscious" low-level online control/reacting (such as many strategies for specialized insertion tasks). It simplifies the planning problem by decomposing planning into different levels and scopes (i.e., global and off-line nominal planning, local patch-planning based on sensing, and low-level motion strategy planning based on uncertainty models), treating the relatively independent issues of the problem separately. In doing so, it aims to achieve (1) efficiency , (2) generality , i.e., task-independence, by being highly modular and integrating both dynamic and static knowledge of relatively general nature, (3) effectiveness , i.e., guaranteeing or almost guaranteeing success, by being systematic and incorporating uncertainty models, and (4) flexibility , by being sensing-based and by being highly modular to facilitate the incorporation of different sensors and task-dependent knowledge upon specific applications.

A simulation package SimRep implements the replanner and simulates the assembly motion in the presence of uncertaities under the guidence of the replanner.

See related publications.

Adaptive Evolutionary Planner/Navigator (EP/N)

This was a project done ten years ago. This project has triggered our discovery of many interesting research issues in their own rights, and the potential of the approach is further explored. See related publications for more detailed descriptions.

Semantic Video Classification, Indexing, and Retrieval

This project is in collaboration with Dr. Jianping Fan. See project description. See related publications.

Cooperative Haptic Manipulation of Virtual Objects

This is a new project in collaboration with Dr. William Tolone. We will investigate the architectural challenges to the proper integration of cooperative haptic environments within an ongoing research project in component-based collaborative systems. We will also study interaction fidelity for the cooperative manipulation of complex objects where an object’s anatomy (e.g., geometry, center of gravity) may not be easily observed.

Structure Study of Neural Networks for Continuous Function Approximation

The objective of this project is to gain general understanding and heuristic knowledge of the optimal neural network (NN) structures for approximating real-valued functions and to explore the feasibility of automatic mechanisms for finding such a structure with the aid of evolutionary computation (EC) techniques. We are particularly motivated by the industrial problem of mapping engine control variables to performance parameters using neural networks. The project was funded by the Ford Research Lab of the Ford Motor Company. See related publications.

Vehicle Sequencing Optimization

During my two-month sabbatical leave at Ford in 1997, I created and implemented an algorithm to optimize vehicle sequence for assembly, which was further developed and tested in a Ford assembly plant. The results showed a 36% reduction in total cost due to soft constraint violations. The algorithm had been made available to all North American plants of Ford. See related publications.

xiao@uncc.edu