Neural Network Models for Learning Representations of 3D Objects via Tactile Exploration

Abstract

How does the brain represent the geometry of 3D objects? Most researchers considering this question focus on vision. However, infants first learn about 3D objects in the haptic system -- that is, by tactile exploration of objects. In this project, we develop a neural network model that learns something about the structure of a 3D cuboid, using input from the motor system that controls a simulated hand navigating on its surfaces. It does this with a simple unsupervised network, that learns to represent frequently-experienced sequences of motor movements. The network learns an approximate mapping from agent-centred (i.e., egocentric) movements to object-centred (i.e., allocentric) locations on the cuboid's surfaces. We also show how this mapping can be improved by the addition of tactile landmarks, by the presence of asymmetries in the cuboid and by the supplement of agent's configurations. We then investigate how the learned geometry of the cuboid can support a reinforcement scheme, that enables the agent to learn simple paths to goal locations on the cuboid.

Personnel

Tags

Computational models of cognition, Reinforcement Learning, Learning Theory

Related Publications

X. Yan. Neural network models for learning representations of 3D objects via tactile exploration. PhD thesis, University of Otago, 2020. Copy bibtex to clipboard
Bibtex has been copied to clipboard.
X. Yan, A. Knott and S. Mills. A neural network model for learning to represent 3D objects via tactile exploration. In CogSci, 2018. Copy bibtex to clipboard
Bibtex has been copied to clipboard.
X. Yan, A. Knott and S. Mills. A Model for Learning Representations of 3D Objects Through Tactile Exploration: Effects of Object Asymmetries and Landmarks. In Australasian Joint Conference on Artificial Intelligence, pp. 271-283, 2018. Copy bibtex to clipboard
Bibtex has been copied to clipboard.
X. Yan, A. Knott and S. Mills. A neural network model for learning to represent 3D objects via tactile exploration: technical appendix. Department of Computer Science, University of Otago, 2018. Copy bibtex to clipboard
Bibtex has been copied to clipboard.