AAAI 2021

https://aaai.org/Conferences/AAAI-21/aaai21tutorials/#ah4

Slides

Download slides

Video

Content

The contents of this tutorial is split into three parts. First, an introduction to the StarAI field is given. Next, we show how the same concepts carry over NeSy and how these concepts can help categorizing neural symbolic approaches. Finally, we look at challenges and future work.

Introduction

We start with an introduction to the field of StarAI.
We provide basic concepts of statistical relational learning and (probabilistic) logic programming, which will lay the basis for the whole tutorial. In doing so, we will use key concepts and techniques outlined in a number of textbooks and tutorials such as Russel et al 2015, De Raedt et al 2016. This will help us to introduce the field of Neural Symbolic Artificial Intelligence, because it turns out that the same issues and techniques that arise in StarAI apply to a large class of NeSy systems as well.

Dimensions

The main characteristic of this tutorial is that we identify seven dimensions that these fields have in common and that can be used to categorize both StarAI and NeSy approaches. These seven dimensions are concerned with:

• directed vs undirected models, where we look at the differences between directed vs undirected models with respect to inference and how these models are applied. For example, the nature of directed models lends itselfx more to inference, whereas the undirected model naturally lends itself more to a constraint based approach.
• model- vs proof-based inference, where we distinguish the model theoretic perspective, where one first grounds out the clauses in the theory and then calls a SAT solver, and the proof-theoretic perspective, where one performs a sequence of inference steps in order to obtain a proof;
• integrating logic with probability and/or neural computation, where we analyse to which extent hybrid systems retain properties of the base paradigms. We will also discuss the difference of methods originating from neural methods versus those originating from logic-based methods;
• logical semantics, where we distinguish between Boolean logic, probabilistic logic and fuzzy logic, and how this impact learning and inference;
• learning parameters or structure, where we analyse the capabilities of a system to learn its parameters, structure or both;
• representing entities as symbols or sub-symbols, where we distinguish between systems representing elements of the logic theory, like constants or atoms, as symbols and those representing them as sub-symbols (e.g. embeddings);
• the type of logic used, where we categorise systems in terms of the expressiveness of their logical theory, being it propositional, relational, FOL or a logic program.

We position a wide variety of StarAI and NeSy systems along these dimensions and point out similarities between them. This provides not only new insights into the relationship between StarAI and NeSy, but also allows one to carry over and adapt techniques from one field to another. The insights provided in this tutorial can thus be used to create new opportunities for cross-fertilization between StarAI and NeSy, by focusing on those dimensions that have not been fully exploited yet. We will also take care of underlining a few important differences between StarAI and NeSy the most important one being that the former operates more at the symbolic level, lending itself naturally to explainable AI, while the latter operates more at the sub-symbolic level, lending itself more naturally for computer vision and natural language processing.

Before moving to the last part, we will sketch an overview of other research directions in the field of NeSy. This is particularly important since it is not yet clearly and globally defined what the key ingredients of a good integration are. This has led to an extremely heterogeneous range of approaches, including paradigms that are not based on logic (e.g. Graph Neural Networks, Modular Networks).

Challenges and future work

In the last part, we will describe the implications of making different choices along the seven dimensions and which challenges and research questions are still open. We will focus on the problem of how to combine different semantics. We will describe time, memory and data efficiency issues of the current approaches. Finally, we will conclude by showing few applications of these methods and emphasizing a urgent need for real-life applications.