Classicism versus Connectionism
Isabel GóisIn this paper, I attempt to give a brief account of the current debate in Artificial Intelligence between the so-called classical computationalist view and the more recent trend of connectionist views concerning what to count as an appropriate explanation of intelligent behaviour. In particular, I will focus on the question of whether connectionist models of cognition constitute a genuine alternative to more traditional views of mental processing, or if, as some have argued, they have nothing to offer but a story about the implementation of symbolic structures at the level of neural functioning. The conclusion I wish to support is that connectionism does offer an alternative account of cognition, although the debate as to which is the best approach - the classical or the connectionist - is far from settled.
It is an article of faith among Artificial Intelligence practitioners that cognition, or intelligent behaviour, is best understood as a complex of information processing transformations carried over representations. However, and despite significant progress in devising a computational view of the mind, the past two decades have seen the field divide in two views on the nature of information processing, and it is fair to say there is no shared agreement on what to count as the proper theoretical approach to cognition. Accordingly, on one side of the fence there are those who maintain the traditional view in AI that intelligent behaviour can only be accounted for by appeal to syntactically structured symbols that are manipulated according to certain formal rules. On the other, there are those who defend that cognition is better understood in terms of sub-symbolic patterns of activation in a group of highly interconnected units. As can be expected, discord over who holds the key to the enigmas of cognition runs deep, with the symbolic AI people accusing connectionist models of having nothing new to offer to the central issues of intelligence, and the connectionist camp arguing that they do.
Now, before moving further, it is important to notice that both the classic view and the connectionist share the goal of understanding and modeling (that is, implementing) cognitive processes in artificial devices. Also, it worth emphasizing that both traditional and connectionist models have inputs and outputs and these inputs and outputs can be given (in the right circumstances) intentional interpretations. What makes them different is that, unlike the traditional computational model, connectionism claims that the output is not produced as a result of the rule governed manipulation of functionally discrete symbols. That is, proponents of connectionism argue that the whole system can be interpreted as representing some state of affairs without it having to be supposed that this has to be achieved by the manipulation of token symbols or representations. The accusation that connectionism routinely meets is that its view of cognition fails to capture the essential properties of (human) information processing, and therefore cannot serve has an appropriate account of intelligent behaviour. According to the defenders of GOFAI (Good Old Fashioned Artificial Intelligence) the most that connectionism can offer is an account of the implementation details of traditional models at a very low level, but as far as cognition is concerned, connectionism remains an irrelevant appendix.
In order to evaluate whether or not the pretensions of connectionism to establish itself as a competing model of cognition are valid, we need to have a clear idea of what both views are claiming and what exactly are the distinguishing features of each model. Let us start by considering the classical view. What does it say about cognition? Roughly put, it says that it is rule-based manipulation of syntactically structured symbols. One of the important characteristics of traditional models is that they operate though rule-governed transformations of discrete functional elements (the symbols), and have access only to the form of the symbols (their syntax), not their meaning (that is, their semantics). This means that, given a task (meaning, a problem to solve) a symbolic system will proceed by a step-by-step transformation of the original representation until it reaches its end-goal (solves the problem it was given.)
Why take this to be a good model of cognition? According to Fodor and Pylyshyn the advantage of classic AI, and also the reason why it stands without qualified competitors, is its ability to accommodate the folk psychological view that mental processes involve real, discrete, causally significant, syntactically structured mental entities. Note that what is important in these authors argument is not, strictly speaking, the idea that GOFAI saves propositional attitudes from greedy reductionists (e.g., those who would dispense with them in favour of talk in terms of C-fibers and P-3000 waves), but their claim that a model of cognition that fails to capture certain features of our common sense conception of the mind simply cannot be a model of (human) cognitive capacities.
Thus, what traditional models can do, and connectionist ones can't, is to emulate the two most important features of our reasoning processes. To be clear, these two features are a compositional syntax and a combinatorial semantics. That is, according to Fodor, thoughts have combinatorial structure which allows atomic units to be combined in more complex structures to create new thoughts (e.g., if you can think 'honest' and you can think 'friend', then you can also think 'honest friend'.) By compositional structure, Fodor means that cognitive processes are sensitive to the configuration of the thoughts they operate on. That is, the computational processes that drive cognition are syntactically governed transformations defined exclusively over the structure and place occupied by the symbols recruited in current tasks. In this sense, a classical model of intelligent behaviour (basically) consists in a store of information (data structures) and a set of search algorithms (rules of inference) that allow it to arrive at a satisfactory solution based only on transformations guided by the shape of the symbols available to it. As classicists claim, this basic picture of a rule-governed system provides the formal architecture of all cognition.
In opposition to this view, connectionists have long argued that symbolic systems have only limited success in accounting for all types of cognition, are unable to deal with incomplete or partial information, and provide a rather implausible picture of our brains due the relatively slow speed at which they perform the tasks they are assigned. What is more, as Smolensky has pointed out (1988), there is the danger that traditional models may be illegitimately projecting certain elements of our conscious, linguistically and culturally informed thought processes onto the brain. In other words, it may well be the case that the Language of Thought hypothesis, so dear to Fodor, is nothing more than a misguided view of what our cognitive processes are like, and thus one that is more likely to delay progress in psychological research than be of help to it.
But, then, how do connectionist propose to account for cognitive performance? While friends of connectionism come with many different flags (some being more extremist about the implications of connectionism (Churchland 1988), some more cautious (Clark, 1989)), they all share the idea that representation of information in a system is the emergence of stable patterns of activation in a network of simple components. Roughly put, a connectionist network consists in a set of units connected to each other, each of which has a specific level of activation at any given time. The connections between units allow the activation level of one unit to influence another's activation level, and this can be done either through excitatory or inhibitory input. That is, the activation level of a given unit may either increase or decrease the level of activation of those units to which it is linked. Moreover, it is a fundamental characteristic of connectionist models that the connections between units can be modified; that is, how the activation of one unit bears on the activation of another is not a matter settle once and for all, and the network can learn new 'routes' of activation.
This said, how exactly do connectionist models represent, and how do they work? As said before, representation in a connectionist network is a function of the whole system. That is, typically a connectionist representation is an emergent global state into which the network stabilizes in response to its initial input. The way these global states are achieved is through the combined activity of multiple units working in parallel to achieve a stable and coherent pattern of activation. This, in turn, is made possible due to local rules for the individual activation of units and rules for changes in the strength ('weight') of the connections between units. However, it is important to notice that these rules, contrary to what happens in traditional models, are not hardwired in the system. They are embedded in the connections between units and, as mentioned before, can be modified or adjusted. Put in other words, the representational capacities of connectionist networks depend on the plasticity of the connections between units, and the rules that operate to transform a current state of activation into a subsequent one are mathematical rules for updating the activation of a unit or change the strength of a connection. Accordingly, 'rules' in a connectionist system should be understood as 'vectors of activation', and not as 'if-then' clauses like in classical models. Also, it is worth emphasizing that representation in a connectionist network is distributed across activated units, and not a function of a discrete symbol being tokened within the system as in traditional models.
What, then, are the advantages of connectionist models? To begin with, connectionist networks have been able to model cognitive capacities so far deemed intractable by classical models. Examples include visual pattern recognition, the production of speech, and decision-making in real-time. Other advantages include flexibility in processing, the ability to learn, and the capacity to perform on the basis of partial or incomplete information (that is, connectionist networks exhibit what is otherwise known as graceful degradation). On the side of disadvantages, connectionist models are often said to be unable to capture the rational, or inferential character of thought. As seen before, this amounts to the accusation that connectionist networks fail to capture the compositionality of thought, and thus cannot gives us an account of the essential architecture of cognition.
Now, as seen before, Fodor and Pylyshyn have often championed the thesis that connectionism can at most provide a low-level implementation of symbolic structures and processes, but as regards the principles of (human) information processing they have nothing new to offer. In fact, if we were to take connectionism as an explanatory model of cognition we would be heading for a dead-end, since cognition as the connectionists portray it simply makes no sense. In other words, according to Fodor and Pylyshyn, either a model of cognition entails rules of composition, or it isn't a model of cognition at all. Connectionist models operate under no such rules unless they are simulated in symbolic systems. But this, as these authors see it, can only be proof that connectionism is merely a theory of implementation of symbolic structures, and not a model of cognition on its own right.
Is this true? That is, Are connectionist networks mere implementation of symbolic AI models? Paul Smolensky has on a number of occasions argued against this accusation, and tried to show that the structural and systematic aspects of reasoning can arise out of processes that, in fact, do not have such a structure. Put otherwise, Smolensky thesis is that compositionality can be accounted for in connectionist models by looking at the holistic patterns of activation in the whole network. However, as Smolensky also notes, while connectionist networks involve a notion of compositionality, this notion is significantly distinct from the one offered by traditional models. To be more specific, the connectionist understanding of compositionality rests on the use of distributed representation across patterns of activation, and it is the sum of unit activations that properly constitute the composit elements of representation. Plus, as Smolensky also notes, connectionist models are sensitive to the structure of the inputs manipulated, the difference being that connectionist information processing operate via statistical-sensitive processes, and not via processes sensitive to the syntax of discrete symbols composed in a Language of Thought. For the connectionist, there is no language of thought at the level of mental structure and processing of representation.
As can be seen, there are significant differences in the story each side has to tell about the symbolic level of cognition. Classicists insist that knowledge is encoded in discrete symbolic structures, and that thought proceeds through rules of inference. What is more, according to the classicist model, the level of semantic interpretation pertains to the level of symbols being manipulated according to the rules that define the system. In this sense, representation in a classic model is a discrete function of symbols being tokened. For Connectionists, on the contrary, representations are highly distributed patterns of activation in a network of individuals units, and there is a radical separation between the level at which formal rules of transformation are applied and the level at which semantic interpretation can be ascribed. That is, in a connectionist model the semantic interpretable level pertains to whole patterns of activation, and the level of rule-manipulation to the sub-symbolic domain of individual activation of single units.
Now, as I see it, these differences in mode of representation and information processing (or rule-governed manipulations) are enough to set the two models as competing theories of cognition. In other words, connectionism is not merely a theory of how a language of thought could be implemented in physical structures. It offers an alternative account of the fundamental principles of intelligent behaviour, as well as a different view of the architecture of cognition. Moreover, it should be noticed that the mode of explanation in connectionist models is significantly different from that offered by traditional models. These remain fairly close to folk psychological explanations of behaviour, in that they cite the tokening of beliefs and desires (symbols) to explain the causal structure of thinking processes. Connectionist models, on the other hand, appeal to statistical rules of inference and inductive processes to account for the pattern transitions (that is, sequence of representations) that constitute the order and coherence of a system's behaviour.
In brief, then, I consider it to be a mistake to say that the most that connectionism can offer is a story about the implementation level of symbolic systems. This, however, does not mean that the debate between classicism and connectionism has been solve. Both models of cognition face difficulties and, so far, none can claim to be in a better position than the other to solve all the problem in AI. Still, it seems to be undeniable that connectionism provides an alternative view of our processes of thinking, and should therefore bee evaluated for its merits and disadvantages in accounting for these, instead of judged for its ability or inability of showing how our brains could be speaking the language of thought.
This paper was written with the support of Ministerio da Ciência e Tecnologia (Grant BM/10514/97). I would also like to thank Dr. Ian Ravenscroft for comments and suggestions on earlier drafts.
isabel.gois@kcl.ac.ukReferences
BETCHEL, W.: (1990) 'Connectionism and the Philosophy of Mind: an overview' in Lycan (ed.) Mind and Cognition; Oxford; Blackwell, pp.252-276.
BETCHEL, W. and ABRAHAMSEN, A.: (1991) Connectionism and the Mind; Oxford; Blackwell.
BODEN, M.: (1991) 'Horses of a Different Colour?' in Ramsey, Stich and Rumelhart (eds.) Philosophy and Connectionist Theory; Hillsdale; Lawrence Earlbeum Associates; pp.3-20.
CHURCHLAND, P.: (1988) Matter and Consciousness (revised edition); Ch. 7; Cambridge-Mass.; The MIT Press; pp.123- 166.
CHURCHLAND, P. M. and SEJNOWSKI, T.: (1990) 'Neural Representation and Neural Computation' in Lycan (ed.) Mind and Cognition; Oxford; Blackwell; pp.224-252.
CLARK, A.: (1989) 'Beyond Eliminativism' in Mind and Language; vol.4, Nº4, pp. 251-279.
DAVIS, M.: (1991) 'Concepts, Connectionism, and the Language of Thought' in Ramsey, Stich and Rumelhart (eds.) Philosophy and Connectionist Theory; Hillsdale; Lawrence Earlbaum Associates; pp. 229-258.
DENNETT, D.: (1984) 'Cognitive Wheels: the frame problem in AI' in Hookway (ed) Minds, Machines and Evolution; Cambridge; Cambridge University Press; pp.190-213.
FODOR, J.: (1983) The Modularity of Mind; Cambridge-Mass.; The MIT Press.
FODOR and PYLYSHYN: (1988) 'Connectionism and cognitive architecture: a critical analysis'; Cognition 28, pp. 3-71.
FODOR (1990) ' Why There Still Has To Be A Language of Thought' in Lycan (ed) Mind and Cognition; Oxford; Blackwell; pp. 282-300.
RAMSEY, STICH and RUMELHART: (1991) 'Connectionism, Eliminativism and the Future of Folk Psychology' in Greenwood (ed) The Future of Folk Psychology; Cambridge; Cambridge University Press; pp. 93-119.
SMOLENSKY, P.: (1987) 'The constituent structure of connectionist mental states: a reply to Fodor and Pylyshyn' in Southern Journal of Philosophy 26 (supplement) pp. 137-161.
SMOLENSKY, P.: (1988) 'On the Proper Treatment of Connectionism' in Behavioural and Brain Sciences 11, pp.1-23.
SMOLENSKY, P.: (1990) 'Connectionism and the foundations of AI' in Partridge and Wilks (eds.) The Foundations of Artificial Intelligence; Cambridge; Cambridge University Press; pp-306-326.
STERNELY, K. (1990) 'Connectionism' in The Representational Theory of the Mind; Ch 8; Oxford, Blackwell.