Noisy-or

The interaction rule used for ProbLog is the noisy-or construct. If you define multiple rules with the same head, they will be combined as a noisy-or (a determistic or if all heads are deterministic). The strength of the noisy-or parents can be learned using LFI (the inhibition strengths).

Example 1

Given a set of words that appear in documents, we can train a Noisy-OR rule to determine how strongly a word indicates that the document has a certain topic. Make sure to include leak probabilities such that all possible evidence can be linked to a possible world (otherwise ProbLog will return an “Inconsistent Evidence” error).

0.0001::word(_). t(_)::topic(t1). % leak probability t(_)::topic(t1) :- word(w1). t(_)::topic(t1) :- word(w2). t(_)::topic(t2). % leak probability t(_)::topic(t2) :- word(w1). t(_)::topic(t2) :- word(w2).
evidence(topic(t1),true). evidence(topic(t2),false). evidence(word(w1), true). evidence(word(w2), false). ----- evidence(topic(t1),true). evidence(topic(t2),false). evidence(word(w1), true). evidence(word(w2), false). ----- evidence(topic(t1),false). evidence(topic(t2),true). evidence(word(w1), false). evidence(word(w2), true). ----- evidence(topic(t1),false). evidence(topic(t2),true). evidence(word(w1), true). evidence(word(w2), true). -----

When using the command line interface of ProbLog we can write the learned program immediately to a file that can be interpreted by problog to perform classification:

$ problog lfi model.pl evidence.pl -O learned_model.pl

Example 2

The absence of a word can also be used as a feature.

0.0001::word(_). t(_)::topic(t1). % leak probability t(_)::topic(t1) :- word(w1). t(_)::topic(t1) :- \+word(w1). t(_)::topic(t1) :- word(w2). t(_)::topic(t1) :- \+word(w2). t(_)::topic(t2). % leak probability t(_)::topic(t2) :- word(w1). t(_)::topic(t2) :- \+word(w1). t(_)::topic(t2) :- word(w2). t(_)::topic(t2) :- \+word(w2).
evidence(topic(t1),true). evidence(topic(t2),false). evidence(word(w1), true). evidence(word(w2), false). ----- evidence(topic(t1),true). evidence(topic(t2),false). evidence(word(w1), true). evidence(word(w2), false). ----- evidence(topic(t1),false). evidence(topic(t2),true). evidence(word(w1), false). evidence(word(w2), true). ----- evidence(topic(t1),false). evidence(topic(t2),true). evidence(word(w1), true). evidence(word(w2), true). -----

Example 3

In case the word predicate includes terms that are not relevant (e.g. the count of the occurences of the word in the document):

t(_)::topic(t1). % leak probability t(_)::topic(t1) :- word(w1,_). t(_)::topic(t1) :- word(w2,_). t(_)::topic(t2). % leak probability t(_)::topic(t2) :- word(w1,_). t(_)::topic(t2) :- word(w2,_). % Leak probabilities 0.0001::word(w1,0). 0.0001::word(w1,1). 0.0001::word(w1,2). 0.0001::word(w1,4). 0.0001::word(w2,0). 0.0001::word(w2,4). 0.0001::word(w2,5).
evidence(topic(t1),true). evidence(topic(t2),false). evidence(word(w1,2), true). evidence(word(w2,0), false). ----- evidence(topic(t1),true). evidence(topic(t2),false). evidence(word(w1,1), true). evidence(word(w2,0), false). ----- evidence(topic(t1),false). evidence(topic(t2),true). evidence(word(w1,0), false). evidence(word(w2,4), true). ----- evidence(topic(t1),false). evidence(topic(t2),true). evidence(word(w1,4), true). evidence(word(w2,5), true). -----

The leak probabilities are required such that ProbLog finds ground clauses (evidence is not included as facts in the program). Including these leak probabilities can be done automatically by ProbLog using the --leak-probabilities flag. This option will include the true evidence as probabilistic facts with the given probability.

$ problog lfi model.pl evidence.pl -O learned_model.pl --leak-probability=0.0001

An alternative solution is to explicitly include the domain for variables in the probabilistic facts. This only works for finite domains, though.

counts(C) :- between(0,5,C). 0.0001::word(_,_). t(_)::topic(t1). % leak probability t(_)::topic(t1) :- counts(C), word(w1,C). t(_)::topic(t1) :- counts(C), word(w2,C). t(_)::topic(t2). % leak probability t(_)::topic(t2) :- counts(C), word(w1,C). t(_)::topic(t2) :- counts(C), word(w2,C).
evidence(topic(t1),true). evidence(topic(t2),false). evidence(word(w1,2), true). evidence(word(w2,0), false). ----- evidence(topic(t1),true). evidence(topic(t2),false). evidence(word(w1,1), true). evidence(word(w2,0), false). ----- evidence(topic(t1),false). evidence(topic(t2),true). evidence(word(w1,0), false). evidence(word(w2,4), true). ----- evidence(topic(t1),false). evidence(topic(t2),true). evidence(word(w1,4), true). evidence(word(w2,5), true). -----