Knowledge 4 CoTeSys
Knowledge Processing for Cognitive Technical Systems
|
The project targets at building knowledge representation and processing systems for mobile robots by combining description logics knowledge bases with data mining, (self-) observation modules and imported knowledge from the World Wide Web.
|
|
Research Topics
- Knowledge Representation
- Learning Grounded Models
- Import of WWW Information Into the Knowledge Base
- Logical Models
- Ontological Representation and Reasoning
- Data Mining / Knowledge Discovery
Project Details
| Knowledge Processing / Programming Language - PROLOG
Our knowledge processing system is an extension of the logical programming language PROLOG, which we have extended with integrated datamining routines, object-oriented knowledge modeling in description logics, grounded action and environment models and an interface to probabilistic first-order reasoning. The knowledge processing system operates on data structures produced by activity observation systems, on 3D semantic object models of indoor environments, and log data from the robots' behavior while performing their tasks. These data structures are represented using sets of computable relations that can be queried symbolically and return symbolic answers by retrieving the respective information from the data structures and translating them into the symbolic language on demand. |
 |
| Processing of Robot Data Structures
The data for the knowledge processing stem from three main sources: semantic environment maps, robot self-observation routines and a full-body human pose tracking system. Objects detected during the map building process get their meaning by associating them with the respective concept in the robot's encyclopedic knowledge base. We represent the objects as instances of the respective classes: Each cupboard, for example, is an instance of the class Cupboard. This representation allows for example to query for objects based on their function or properties. The log data of robot actions are provided by RoLL (Robot Learning Language) which was developed as part of the Cogito project. RoLL allows to define conceptual models of problem-solving behavior in form of hybrid automata that are anchored to the control program execution. RoLL filters the stream of observations and records data that are accepted by the automata as structured "experiences". From the detailed logs of action execution, we learn grounded action models that represent objects, places and other concepts by their roles in actions. We started with the analysis and interpretation of observed human manipulation actions using data from the full-body human pose estimation. Overall, we are able to perform reasoning on the environment model, on actions performed by the robot and on observed human activities. |
 |
| Integration of data mining, learning and reasoning mechanisms
Having access to data mining and learning tools is crucial for interpreting, understanding and learning from collected data. We embedded open-source Java libraries into the PROLOG system. The Weka machine learning library provides a large set of classifiers and clustering routines, Mallet which contains implementations of Cconditional Random Fields (CRFs) we use for analyzing and labelling sequences. We started with embedding MATLAB for sophisticated data analysis and easy manipulation of mathematical data structures. |
 |
| Graphical Interface A graphical user interface (GUI) to the knowledge processing system was developed in order to be able to demonstrate the system's capabilities. The GUI allows for the manual exploration of data before automating the process using the integrated data mining modules, custom features for different calculations, visualisations, links to data sources like MySQL databases and the interface to the PROLOG knowledge processing system. During this project, we created several visualisations demonstrating the robot's knowledge about the world, for instance about objects, trajectories, human poses or observed positions. We also integrated modules from the Protege ontology editor for visualizing the encyclopedic knowledge the robot has. |
 |
| Integration into Robot Control
One of the main tasks of the knowledge processing system is to support the planning and decision making process. We created the infrastructure for linking the plan-based robot control developed in the Cogito project to the knowledge processing system. The planning system can send queries via a YARP connection that provides an interface for remote procedure calls. Custom extensions to the RPL plan language integrate the queries to the knowledge base, e.g. for determining the pose for performing an action. This pose was previously calculated based on heuristics and is now provided by the knowledge processing system that learns it from observation. Knowledge Sources In this project, we aim at re-using existing knowledge whenever possible. We derived a large encyclopedic knowledge base of objects, actions, spatial and temporal concepts specific for the kitchen environment from the reserarchCyc upper ontology. By using reserarchCyc identifiers throughout the whole system we ensure interoperability and guarantee that different modules refer to the same thing when using the same name. In addition to the encyclopedic knowledge from researchCyc, we imported large amounts of common-sense knowledge from the OMICS database, a project that targets on the collection of indoor common-sense knowledge represented in natural language. For understanding the statements in natural language and mapping them to ontological concepts, we used WordNet and existing mappings between WordNet and researchCyc. For obtaining procedural knowledge in form of task descriptions, we successfully imported step-by-step instructions in natural language from websites like ehow.com into the knowledge base and exported them to robot plans written in the RPL language. |
 |
Videos:
