Ideas as Rules of Inference
I
Inferential reasoning is generally understood as involving at least two kinds of entities: statements (aka "sentences", "propositions") and rules of inference. Rules of inference tell us how we can derive (or "infer") new statements from existing statements. One of the core rules of inference used in classical logic, for instance, is called modus ponens, and allows us to derive, from two statements like "A" and "If A, then B", a new statement "B".
Inferential systems like classical logic operate on statements expressed in specialized, formal languages, not statements in a conventional language like English. If you want to use a formal inferential system to analyze a line of reasoning expressed in an informal language, you first have to translate the informal sentences into formal statements that the inferential system can operate on.
Consider a simple, everyday example of inferential reasoning. Say I have the following two beliefs: "I should carry an umbrella when it's raining outside" and "It's raining outside". From these statements, I can reasonably conclude "I should carry an umbrella with me". How can we understand this inference in the context of a formal inferential system?
If we wanted to use classical logic, we could rephrase the first sentence, "I should carry an umbrella with me when it's raining outside", as an if-then statement: "if it's raining outside, then I should carry an umbrella". We can then use the aforementioned inference rule modus ponens to derive, from this modified statement and the other statement "it's raining outside", the conclusion "I should carry an umbrella".
But classical logic isn't the only inferential system we might want to use, and this strategy of reformulating the sentence "I should carry an umbrella with me when it's raining outside" as an if-then statement isn't the only conceivable way of formalizing it. Instead, we may choose to treat "I should carry an umbrella when it's raining outside" not as a statement at all, but as a rule of inference - specifically, a rule that allows us to infer the statement "I should carry an umbrella" from the statement "it's raining outside".
As another example, consider Newton’s law of gravity, which says that the gravitational force between two objects will be G*m1*m2/r², where G is the gravitational constant, m1 and m2 are the masses of the two objects, and r is the distance between the two objects. This law, just like the previous idea about umbrellas, can be understood as a rule of inference: a rule that takes in three statements like “Object A has mass m1”, “Object B has mass m2”, and “The distance between objects A and B is r”, and returns a new statement like “The force between objects A and B is G*m1*m2/r²”.
This way of formalizing ideas is relatively unusual. Most formal accounts of inferential reasoning involve a small, unchanging set of inference rules, and the contents of novel ideas are expressed as statements. But clearly we could, if we wanted, express the contents of some ideas as inference rules, rather than as statements.
So what? What advantages, if any, does this way of understanding inferential reasoning provide?
II
In Realism and the Aim of Science, Karl Popper discussed the proposal of treating ideas (specifically, scientific theories, or universal laws) as rules of inference.
We certainly may look upon universal statements as validating certain inferences, or as equivalent to certain rules of inference: it is a simple fact of logic that, whenever we have a valid inference which proceeds from more premises than one, we may interpret any of the premises (if true) as a (valid) rule permitting us to draw the conclusion from the other premises. Thus we can say that ‘Socrates is a man’, if true, validates the inference from ‘All men are bipeds’ to ‘Socrates is a biped’; and similarly, that ‘All men are bipeds’, if true, validates the inference from ‘x is a man’ to ‘x is a biped’
Popper acknowledges that we could, if we wanted, treat our scientific ideas as rules of inference, rather than as universal statements. He goes on to argue that the two perspectives - of scientific theories as universal statements, or as rules of inference - are formally equivalent. However, Popper points out, there are some important caveats that we must keep in mind if we choose to use rules of inference this way:
It is customary to use only logical, or analytic, rules of inference. By the suggestion that all universal statements should be considered as rules of inference, we are thus implicitly encouraged to take all universal statements for analytic... The custom of using only logical, or analytic, rules of inference is connected with the fact that in ordinary life as well as in mathematics or in science we are hardly ever conscious of the rules of inference we use. We use them unconsciously, implicitly, relying on their validity without question: there can hardly be such a thing as a problematical rule of inference. As a consequence, we rarely question the validity of rules of inference, and we never think of putting them to experimental tests. Yet if we were, as proposed, to interpret all universal theories as rules of inference, then we should have to treat these rules of inference like universal theories: we should have to test them, to try to falsify them—unless we give up the critical method of science.
Ultimately, Popper rejects this proposal. Because treating theories as inference rules is equivalent to treating them as universal statements, and because it would force us to think of rules of inference in an unconventional, potentially confusing way, Popper saw no point in adopting this rule-focused perspective over the more conventional statement-focused perspective.
Given the problem-situation that Popper was in, I think his rejection of the proposal makes sense. Popper saw it as an ill-fated attempt to solve some problems with positivist epistemology, like the problems of verification and induction. I agree with Popper that the rule-focused perspective doesn't solve any of the problems its positivist/inductivist proponents might have hoped, but I think that it may prove to be a very valuable in solving a different problem: creating an Artificial General Intelligence.
III
A central question in AGI is: What is the structure of an intelligent mind? What components does it consist of, and how do these components interact? I suggest that the rules of inference are, in fact, the basic components out of which minds are built.
A mind is essentially a collection of conjectured inference rules. The inference rules within a mind are not fixed, or unchanging, or adopted uncritically, as is typical in conventional theories of reasoning. Instead, the collection of inference rules that a mind uses is constantly changing, as the mind conjectures new rules and rejects old ones. The inference rules that a mind uses are not, in any sense, justified, but instead simply represent the mind's current best guess about the nature of reality.
A mind constantly criticizes and attempts to falsify its inference rules. When the mind discovers that one of its inference rules is problematic - say, because its consequences contradict some empirical evidence - it may choose to reject the rule. Or it may choose to replace the old rule with some newly-conjectured variant rule that slightly modifies the old one in a way that resolves the problem. Learning is essentially a process of searching, via this iterative process of conjecture and criticism, for a set of inference rules that help the mind understand and explain reality.
But why use inference rules as the basic building blocks of a mind, rather than, say, universal statements, as Popper preferred? Mathematically, an inference rule can be thought of as a function which takes some statements as inputs ("premises"), and returns some other statements as outputs ("conclusions"). More specifically, any inference rule that we hope to use in practice must be a computable function, otherwise we wouldn't be able to determine what conclusions the rule entails. This means that every worthwhile inference rule can be represented as a program. So if its true that a mind can be understood as a collection of conjectured inference rules, then a mind can be represented on a computer as a collection of programs, each of which represents an individual rule of inference. This perspective also suggests a convenient computational interpretation of the notion of conjecture: conjecturing a new inference rule is equivalent to simply generating a novel program.
David Deutsch has argued that general intelligences must be universal explainers, meaning they must be capable of explaining anything that can possibly be explained. I think that treating ideas as inference rules is an important step in formalizing Deutsch's notion of explanatory universality. If a mind is a collection of programs which represent inference rules, and if the language in which these programs are expressed in is Turing-complete (in the sense that it can express any computable function), then the mind can potentially represent any inference rule, or in other words, any conceivable way of inferring conclusions from premises.
IV
Popperian epistemology provides powerful criticisms of inductivist theories of epistemology. These criticisms are highly relevant to the field of AGI, as they suggest that models of intelligence based on inductivist epistemologies, such as the various Bayesian models, are doomed to fail. However, there have not, so far, been any detailed, formal proposals for a Popperian model of AGI (at least, not that I'm aware of).
I believe that the perspective I've described in this post, of treating ideas as rules of inference represented by programs, is a key step in formalizing Popperian epistemology in way that can inform research on AGI. In future posts, I'll elaborate on the details of this formalism, and discuss some of the problems with it that still need to be solved.