Binding and the effects of traumatic brain injury on the organization of thought: A pilot study Thomas E. Gladwin Studentnumber 0944882 13-6-2000 University of Groningen Supervisors: dr. P. H. de Vries dr. W. H. Brouwer Contents Abstract 1. Introduction 1.1 Models used to explain the effects of traumatic brain injury 1.2 Criticism of disturbances of the CES and SAS as explanations for the effects of TBI 1.3 ACT-R: production systems as control structures 1.3.1 Criticism of the ACT-R model 1.4 Levels of description 1.5 Connectionist dual-weight architectures 1.5.1 Cell assemblies 1.5.2 Binding 1.5.3 The conceptual network as a model for TBI 1.6 The function of prefrontal cortex 1.7 Dissociation of the neural correlates of implicit and explicit memory 1.8 The effects of TBI on cognition 1.9 The tasks 1.9.1 The lists tasks 1.9.2 The cued modality lists task 1.9.3 The set-switching task 2. Method 2.1 Subjects 2.2 Tasks 2.2.1 The lists tasks 2.2.2 The cued modality lists task 2.2.3 The set-switch task 3. Results 3.1 The letter lists task 3.2 The tone lists task 3.3 The mixed lists task 3.4 The cued lists task 3.5 The set-switch task 4. Discussion 4.1 Interpretation of the results 4.1.1 The letter lists 4.1.2 The tone lists 4.1.3 The mixed lists 4.1.4 The cued lists 4.1.5 The set-switch task 4.2 The tokenizing hypothesis: confirmed? 4.3 Relating the tokenizing hypothesis to Timerman & Brouwer's spread of activation theory and ACT-R 4.4 Relation to CE-type models 4.5 Further research References Abstract A theoretical framework, merging a network model incorporating binding with a neuropsychological model of the prefrontal cortex, for looking at the effects of traumatic brain injury (TBI) is proposed. Within this model, task-set networks implementing rules which direct how the brain reacts to stimuli are located separately from the long term representations of the stimuli. Traumatic brain injury is hypothesized to disrupt the binding of posterior long term representations to anterior task-set nodes. In this way, TBI could be said to leave central-executive (see below) mechanisms intact, but to damage their contact with the representations of the environment their function it is to control. The central-executive concept is discussed, and an attempt is made to understand the functions bundled in the central-executive concept in terms of an extended control system consisting of interactions between memory, environment and task-set. The results provided some support for the hypothesis and suggested a more specific disturbance of the synchronization of posterior and anterior cell assemblies in TBI. 1. Introduction There is practically no aspect of theorizing about neuropsychological phenomena that is not subject to criticism or re-evaluation; every assumption, on a variety of levels right up to the meta-theoretical, influences what is looked for, what is found, and how findings are interpreted. So how real can theories be, especially in historically controversial areas such as attention and disorders of attention? That is, are they somehow consistent with each other, and grounded in the empirical world? Or is there just a city of ghostly constructions, floating in a loose cloud around the very real effects of brain damage, a breeding ground for ever-multiplying swarms of buzzwords? Part of this paper – subsections 1.1 through 1.4 - presents some considerations about traumatic brain injury (TBI) at this level of applied science philosophy. One meta-theoretical question is whether the ghostly city should be demolished, leaving only one single, solid skyscraper, or whether the city should remain and our view of it enhanced, and if so, how? Another question is what approach should be chosen in the present situation, in response to existing theoretical problems and (in principle, though this aspect is not a focus of this paper) practical demands. The stance taken in tackling these questions is based on the idea that science is fundamentally pragmatic: the total context in which a scientific problem occurs will, and should, lead to scientists choosing the methods perceived to be “best” (most ethical, most falsifiable, etc) according to the norms of that specific situation. Thus, “the” scientific method may change depending on the situation. Not only that, but the criteria for what counts as the best method, and what is accepted as a solution for a problem depends on the context in which the problem occurs. In accordance with this stance, no principled answers will be supplied in this paper, but arguments as to why, in the current situation surrounding TBI, certain theoretical choices and approaches might be expected to be useful. This methodological pragmatism can be seen as an application of Dehue’s theory of local rationality. Readers with a general interest in the history and theory of psychological methodology are referred to her book, Changing The Rules (Dehue, 1995). Another part of the paper, subsection 1.5 and onwards, is concerned with a specific hypothesis about TBI, couched in theoretical terms following from the preceding discussion. The hypothesis is that the physical damage caused by TBI results in a disruption of the binding of representations of objects and concepts to transient patterns of activity which subserve the control of behaviour. This hypothesis is based on a general model for the control of behaviour, which combines Dalenoort & de Vries’ (1998) conceptual network with Petrides’ (1996) model of the prefrontal cortex. The following subsections will describe some effects of TBI and models in terms of which these effects have been explained. Criticism of these models, together with some meta-theoretical considerations allowing the models to be related to each other, will lead to the model used here, and the hypothesis concerning TBI. 1.1 Models used to explain the effects of traumatic brain injury The complaint of TBI patients which correlates most strongly with the duration of their post-traumatic amnesia is the inability to do two things at once (Van Zomeren & Van den Burg, 1985). In accordance with this complaint, patients show poorer performance than controls on the Paced Auditory Serial Addition Task, PASAT (Gronwall & Sampson, 1974). In this task, digits are presented at a fixed rate, and subjects have to state the sum of the latest and previous digit. The PASAT can be seen as a test attempting to measure divided attention. There are, however, doubts concerning the interpretation of the PASAT (Ward, 1997). In any case, the test, whatever label is given to what it actually measures, is sensitive to TBI and thus captures something about its effects. Van Zomeren (1995, in Eling & Brouwer, 1995) names two possible explanations for the complaint and PASAT data, or, more generally, divided attention disorders. Firstly, slowed information processing could serve as an explanation. This, of course, begs the question “In what way?”. Furthermore, the data supplied by Timmerman & Brouwer (1998) do not support this undifferentiated view (see subsection 1.3). Van Zomeren's second possible explanation is a lowered quality of processes necessary for coordination and integration. He uses Norman & Shallice's (1986) model of controlled and automatic processing as a framework for this explanation, with the Supervisory Attentional System (SAS) being the locus of effect of TBI. This approach fits well with a selective disturbance of the integration of information by TBI, the performance of single or dual but separate tasks being unaffected (Verzendaal, 2000). McDowell, Whyte & d'Esposito (1997) also point to the disruption by frontal brain damage, as occurs in TBI (see below, in subsections 1.6 and 1.7), of the performance of complex tasks, and the lack of strong effects on component functions, such as selective filtering (Gronwall & Sampson, 1974). A related finding is that performance on the Stroop test is slowed no more in the interference condition for patients than controls (Thomas, 1977; Stuss et al., 1985; Van Zomeren & Brouwer, 1987). McDowell, Whyte & d'Esposito argue for the use of a theoretical model to describe cognitive deficits, instead of using empirically defined terms based on neuropsychological tests, such as selective or divided attention. They write that “[...] a theoretical model could suggest which cognitive processes are involved and define ways to measure these deficits, thus offering significant advantages.” The theoretical model McDowell, Whyte & d’Esposito propose is Baddeley's working memory model. They conclude from their double task experiments that TBI disrupts the function of the Central-executive. This of course lies close to Van Zomeren's proposal concerning the SAS. We will now look more critically at these models. 1.2 Criticism of disturbances of the CES and SAS as explanations for the effects of TBI Newell (1973) wrote in “You can’t play twenty questions with nature and win” the following about the short term memory in Atkinson & Shiffrin’s (1968) model, which Baddeley developed into his working memory model. “Much of the new progress in the experimental analysis of the information processing of humans has eschewed attention to the control structure. The present papers of this symposium are no exception. However, my best example (my canonical one, so to speak) is the deservedly well-known paper by Atkinson and Shiffrin (1968) entitled: “Human Memory: A proposed system and its control processes.” The model of memory is there all right, and is applied to a number of tasks with quantitative precision. However, the control structure is completely absent and is used as a deus ex machina to concoct separate models for each task. Criticism is not directed at that justly influential piece of work. But it does illustrate well the current state of the theoretical art. As long as the control structure – the glue – is missing, so long will it be possible to suggest an indefinite sequence of alternative possibilities for how a given task was performed, hence to keep theoretical issues from becoming settled.” Styles (1997) uses the term homunculus instead of deus ex machina when discussing Baddeley’s CES and Norman & Shallice’s SAS, but her objection is the same as Newell’s. The model uses a “something” that contains all kinds of, perhaps well-specified, monitoring powers, but whose inner workings are not at all specified. Even worse, to explain a subject’s task performance the “something” must be equated to a somewhat slimmed-down version of the subject himself. Newell’s main concern was the fragmentary nature of theorizing without a unifying and binding control structure, but the absence of explicit control structures also leads to what is known as the homunculus problem: explaining what goes on inside a subject by invoking a “little man in the head” explains nothing. Styles (1997) writes the following about Baddeley’s approach to the homunculus problem. “Recently the SAS has been equated with the central-executive in Baddeley’s (1986) model of working memory. Unlike Broadbent (1984) who tried to avoid the homunculus problem in his Maltese Cross model of memory by proposing control resulted from the operation of productions, Baddeley posits control by the SAS. Is this, however, a homunculus by a different name? By giving control over to the SAS, Norman and Shallice (1986) and Baddeley (1986, 1990) seem to have done little more than re-name “the subject” in Atkinson and Shiffrin’s (1986) model of short-term memory as “the supervisory attentional system”. However, Baddeley (1996, p.26) believes that the homunculus can serve a useful purpose provided that we remember it is a way of labelling a problem rather than explaining it and that we continue to work at “stripping away the various functions we previously attributed to our homunculus until eventually it can be declared redundant”. Baddeley point out that whether the central-executive will prove to be a single unitary system or a number of autonomous control processes is yet to be discovered. Certainly there is good evidence that people act as if they have an SAS and can behave in goal-directed ways, initiating and changing behaviours, apparently at will.” Baddeley’s proposed approach will be discussed in the final chapter. In any case, it seems clear that the homunculus / CE / SAS is at best a bundling and labelling of functions, not an explanation of them. So, stating that these functions are disturbed by TBI narrows the field of enquiry, but is still only a starting point for relating TBI’s physical damage to specifically these effects. Even so, some kind of distinction between controlled and automatic processing seems valid, even if it may be gradual (Neuman, 1984). This is especially so since the bundle of executive functions does seem to be the locus of effect of frontal damage. So what kind of specified control structure would satisfy Newell? 1.3 ACT-R: production systems as control structures Newell’s solution to the problem of control structures was production systems. These are cognitive architectures containing productions, entities linking certain conditions to certain actions. In this way, the mechanisms underlying the control of cognition are made explicit. ACT-R is a well-known production system and a previous version of the model has been used by Timmerman & Brouwer (1998, see below) as a framework for explaining the effects of traumatic brain injury, so it will be used here to examine this approach in more detail. ACT-R is built up from three memories: declarative memory, procedural memory and goal memory (the goal stack). Declarative memory contains independent chunks of information. A chunk is an information pattern consisting of slots and values for the slots. Procedural memory contains the productions mentioned above. The conditions contain a goal and usually a partially unspecified chunk which must be matched to a chunk in declarative memory to fill in missing slot values. The matching of a declarative chunk to the type and slot values specified in the condition of a production is called retrieval, and the filling-in of empty slots in the production by this retrieved information is called instantiation. The actions primarily specify changes to the goal state. Finally, goal memory contains the active goal and any deeper goals not yet fulfilled. The active goal specifies the set of productions which will be evaluated for attempted instantiation. Evaluation refers to the calculation of the expected net gain of choosing that production, using a formula derived from the rational analysis theory underlying ACT-R. This theory states that the cognitive system will make that information available which is likely to be needed in a given situation, based on its learning history. The lacking information must, as described, be retrieved from a chunk matching the type specified in the production’s condition. ACT-R is a hybrid model in that it works by symbolic and subsymbolic mechanisms. Symbolic mechanisms are deterministic, based purely on some high-level representation of information. The selection of the set of productions which may fire on the basis of the type of the goal is such a symbolic mechanism. Subsymbolic mechanisms may contain noise, and influence selection in a neural-network-like manner, using base-level and contextual activation to arrive at parameters used in selection formulae. As described, certain aspects of declarative memory are subsymbolically respresented, as is the case with procedural memory. For example, fluctuations of the activation of chunks can influence whether they will be available for retrieval (ACT-R uses an activation threshold below which chunks will not be retrieved) and a random factor in the evaluation of productions allows the system to try out new solutions to problems once in a while. Timmerman & Brouwer (1998) used the older ACT-* model as a theoretical framework to explain the effects of TBI. They used the performance rates and learning rates of certain tasks as indications of the functioning of declarative and procedural memory, and concluded that it was specifically declarative memory which was affected. For example, performance on variations of a categorizing task, with manipulations thought to be related to association strength between categories and examplars, was taken to be a measure for declarative memory, and changes in difficulty in a mental rotation task were expected to differentially tax procedural memory. It was found that the impact of TBI on performance interacted with the variations in the declarative, and not on the procedural manipulations. Timmerman & Brouwer’s conclusion was that the links between chunks in declarative memory were weakened, slowing the spread of activation and thus the activation of chunks which might be needed to instantiate a production. As ACT-R no longer contains this network-like representation of declarative memory (chunks now receive activation only from the active goal), this theory may need to be re-evaluated. Searching for category information is modelled by extensive subgoaling in ACT-R, and this would suggest that the categorisation task used by Timmerman & Brouwer would be interpreted quite differently. For example, consider two chunks in declarative memory, encoding “a trout is a fish” and “a fish is alive”. To decide whether “a trout is alive” ACT-R would have to retrieve the immediate category (i.e. “fish”) of “trout”, the subgoal to check if fish are alive (retrieving the second chunk, with latencies depending on the association strengths between the (sub-)goal chunks and the declarative memory chunks), and return the confirmation to the goal containing the trout. There is no direct linkage between the two chunks; activation in declarative memory spreads in a controlled fashion, guided by subgoals. In the discussion a variation of their ACT-R-based theory will be presented, which will be analogous to the binding hypothesis to be discussed later, and which explains their results in terms available in the newer version of ACT. 1.3.1 Criticism of the ACT-R model Anderson & Lebiere say the following about the goal stack in The Atomic Components of Thought (1998), p. 40: “A third question concerns the perfect-memory assumptions behind such a goal stack. In ACT-R, all of the goals on the stack are retained perfectly in order. If the current goal is popped, the next one is always ready to take over. We have to confess to not being particularly sanguine about this feature. This assumption has not been stressed much because most cognitive tasks tend to have rather shallow goal structures (perhaps an indication that deep goal structures are problematical). One of the cognitive tasks that has the deepest goal structure is the Tower of Hanoi task - at least under certain strategies. As a later subsection displays, ACT-R has considerable success in modelling the available data from this task. Perhaps future research in this domain will expose problems with ACT-R's assumption of perfect goal memory.” In individual differences and brain damage it seems quite possible that variations in the possible size of the goal stack, and the efficiency of manipulations of the active goal might be important. Though ACT-R supplies a framework for thinking about these differences and effects, they cannot actually be modelled due to the lack of a subsymbolic level in these aspects of ACT-R. There is however no reason for a new version of ACT-R not to incorporate a subsymbolic level for the activities surrounding the goal stack. A more fundamental criticism concerns the implementation of the model in the brain. Anderson & Lebiere (p. 30) have this to say about complaints that “there are no variables in the brain”: “In reading the production syntax, the reader should keep in mind that there is nothing implied beyond the goal transformations that these productions specify. It is easy to think that there is something more implied by the syntax. For instance, we have had people say to us, "Production rules have variables and neuroscientists have never found a variable in the brain." However, just as an integral sign in mathematics means nothing more than integrate and the dx means nothing more than the dimension being integrated over, so the ACT-R formalisms mean nothing more than the operations they help specify. In the case of variables, they just specify the transfer of information from goals to constrain retrievals in declarative memory and from results retrieved back to the goal structure. This transfer of information, which is specified by variables, most likely corresponds to neural pathways in the brain between goal areas and declarative areas (see Chapter 12).” While their counterargument seems valid, in modelling neuropsychological phenomena it still seems preferable that a model should be directly and specifically related to the brain, or that at least there are explicit translations of the model to the brain. That is to say, a further model must be supplied (which is actually what happens in Anderson & Lebiere’s Chapter 12, where they introduce their connectionist ACT-RN) which explains exactly how neural pathways work to transfer information, and can model how brain damage might disrupt this transfer and thus the cognitive system. The relationship of such a model to ACT-R, or the CES or SAS, can be placed in the meta-theoretical framework of levels of description, the subject of the next section. 1.4 Levels of description The models described so far contain what could be called black boxes. We cannot, within the models, look inside the CES or the SAS or the mechanisms relating a goal’s type to the set of competing productions. This would only be a highly theoretical problem, with little practical interest and little need to research further, if there was no interesting variation to be expected within the boxes. However, as these boxes are part of the brain and the brain’s function, brain damage seems unlikely to ignore their contents. To understand the neuropsychology of brain damage thus demands a change in the kind of models used, if only to rule out the possibility of changes within the boxes. So it seems that it could be worthwhile to open black boxes. The theories and models with which that is achieved do not suddenly become unrelated to the original model. Their relationship can be described as being on different levels of description. In this framework, models can be described with the relative adjectives “structural” and “functional” (cf. Dalenoort & de Vries, 1998). When a model containing a black box is restated in such a way as to open the black box, the original model is functional as regards to the opening model; the opening model is structural as regards to the original model. So, a model on its own cannot be labelled structural of functional in the meaning of the terms used. An important aspect of this meta-theoretical framework is that hierarchical structures of models can be built, providing descriptions and explanations of a subject of interest by way of interrelated models on different levels. In (neuro)psychology this would mean that models on a neural level could be systematically related to the behavioral, emotional or social level at which relevance for lower-level models is found. Though structural and functional models describe the same part of reality, they do not make each other obsolete. The functional model is efficient, precisely because of its black boxes. But it does not specify underlying processes, a failing argued here to be potentially important in modelling neuropsychological phenomena. Structural models can pick up where the functional models’ range of explanatory power stops. However, there is a downside to focussing on structural models. Firstly, structural models may lack inherent restrictions. Various structural models can be created for a certain functional model and the question becomes which, if any, is the right one? For example, every information processing system can be modelled as a Turing machine(Deutsch, 1997; McCullough & Pitts, 1943, proved this mathematically for neural networks), but obviously not every information processing system is a Turing machine structurally. In the extreme case, the structural model predicts nothing about the actual implementation in real life; picking one of a large range of possible implementations would then reveal little more than the personal inclinations of the modeller. This problem is not as relevant if the physical, knowable structures of the brain make up the elements of a model as at least part of the implementation is then known already, but the second point illustrates that the importance of a meaningfully related set of models is still present when models based on physical structures become available: an explanation of emergent behaviour may be inefficient or even unintelligible precisely because of its structurality. To take a very extreme example, it is undoubtedly true that all our behaviour, including cognition, is a consequence of processes at the sub-atomic level; but a quite different kind of cognition could exist given exactly the same sub-atomic processes, and there seems to be no way to class the exact configurations of the constituent elements and processes which would, in principle, define the emergent behaviour of a given high-level implementation (such as the brain). A model at this level, for all its structurality, becomes vague and actually loses explanatory power as, in the absence of the possibility of describing efficiently and in detail the interactions resulting in the behaviour of interest, it seems likely to degenerate into a general statement of belief in sub-atomic, or some other low-level processes. In the case of (neuro)psychology, functional models can home in on the right level of description of causes of variations in complex human behaviour where the structure of the brain itself makes no predictions; either because that structure itself is described at too high a level to register the right variations (differences at the level of neurotransmitters will not show up in a model of neuroanatomy), or because the differences at a lower neural level map onto the differences in emergent behaviour by way of a function practically too complex, or principally not systematic enough, to provide more than the afore-mentioned “article of faith” explanation. Finally, it seems unlikely that a good structural model of some class of behaviour will be achieved if the functional characteristics of that behaviour are badly defined. Take, for example, an fMRI study designed to detect systematic activation patterns related to certain task manipulations. Without some functional model, the interpretation of brain imaging studies is circular. That the hippocampus is active in memory tasks means very little if memory is defined as hippocampal activity! Furthermore, in higher level cognition, simple neural structures which work through interactions that are intelligibly describable and lead to functionally insightful outcomes are unlikely to be found (of course, there are processes with simple enough substrates for the problems sketched here not to be relevant; however, such processes seem likely to be more relevant to methodological considerations than to psychological theories concerned with complex behaviour). A functional model at a more abstract level than the neural is then necessary to turn the correlations between tasks conditions and activations in the brain into psychologically meaningful information and, eventually, into an integrated model of mind (the term "mind" could be defined here simply as how we view the higher functions, that is, those which mediate behaviours with many degrees of freedom, of the brain; this is a definition that fits with our primary concept of mind - the higher functions of the brain as seen from the first person perspective). Thus both levels are heavily dependent on each other, and on a healthy interaction providing deep explanations for the functional, and objective restrictions, meaning and integration for the structural level. Using more a broader range of functional models with a higher level of detail provides one source of restrictions for structural models of which the implementation is unknown (that is, abstracted models of the brain) – fewer implementations of a structural model will predict subtle effects or work for a broader range of situations. Use of physiological and neuropsychological evidence provides another restriction for such models, if the modularity of the brain is used to guide the form of models. Empirical data should be able to test certain aspects of models; such data could be provided by experiments or simulations (“empirical” in the sense that when patterns in, for example, connectionist models form over time, something is discovered about reality; in this case about the subsection of reality sufficiently covered by the modelled connectionist learning mechanisms). Finally, and speculatively, the integration of models close to the mechanisms of the brain itself and brain imaging could provide both restrictions for structural models and a meaningful framework for brain imaging studies. In a following subsection the dual-weight connectionist network (the term binding network will be used for the real-world objects that can be modelled by a dual-weight connectionist network) will be presented as a possible class of structural model for the models discussed above. The problems associated with the CES, the SAS and ACT-R as explanations for the effects of TBI could be solved by the right structuralisation; in effect, the whole of this paper is, and the experimental data collected are, concerned with the question of whether dual-weight networks (see below), with the chosen physiological restrictions yet to be discussed, are the right kind of structuralisation for the job. 1.5 Connectionist dual-weight architectures Connectionist dual-weight architectures form a class of network in which connections between nodes have both a long- and a short-term weight (Levy & Bairaktaris, 1995). The relation between the two weights has been modelled as additive (Hinton & Plaut, 1987; Cleeremans & McClelland, 1991), multiplicative (Gardner-Medwin, 1989) and independent (Levy & Bairaktaris, 1991; 1993). Conceptual network (Dalenoort & de Vries, 1998) simulations can be classed as a dual-weight architecture with independent weights. These networks are set up by the experimenter to perform a trial of a certain task. A short-term (called “temporary” in conceptual networks) connection is laid if the participating nodes are simultaneously active and have been labelled as part of a context which in a real-life binding network would provide for synchronization between the nodes (see subsection 1.5.2, Binding). These connections determine the flow of activation during the rest of the simulation. Nodes and groups of nodes in the conceptual network correspond to cell assemblies, and a short-term weight changing from 0 to the temporary weight corresponds to the binding of two cell assemblies. Cell assemblies and binding will be discussed below, together with a number of arguments for the use of networks in studying TBI. 1.5.1 Cell assemblies Hebb's (1949) idea of cell assemblies follows from the Tanzi-Hebb learning rule (1893 / 1949). This rule states that synapses will form between simultaneously active neurons, or that existing connections will become stronger. A cell assembly is the result of such learning: a group of highly interconnected (groups of) neurons, which can be part of multiple assemblies and whose significance depends on the active assembly they are currently part of. Such a cell assembly fits well with distributed representation of features in the brain. Instead of objects having to be represented by a certain neuron or a certain group of neurons, they can be represented by an assembly. Roelfsema (1998) gives the following three reasons that cell assemblies would be a good method of representation. Firstly, cell assemblies are economical: neurons representing a certain aspect common to many objects can be re-used for each object. Secondly, similar objects evoke similar representational states. Finally, new objects can be represented as a new pattern of activity over existing modules. Due to these adaptive advantages of cell assemblies, they might be expected to be used in the physical basis of human cognition. The role of cell assemblies could be extended from object representation to task-set representation, for analogous reasons. However, the cell assemblies forming a task-set are less well defined than those for the representation of a visual object. Since a system operating only by the Tanzi-Hebb learning rule would lead to ever growing cell assemblies, and even weak activation would lead to the total activation of the assembly, the existence of inhibitory connections must be assumed, since there must be separate assemblies and controlled activation for the brain to work (Dalenoort & de Vries, 1998). These connections imply the existence of a critical threshold, a level of activation under which activation dies out if not fed by further input and above which a cell assembly reaches maximum activation on its own power. 1.5.2 Binding Binding is best known from two related versions of the binding problem. The location - identity version (Treisman, 1996) is concerned with the question of how separate representations for locations and identities can be correctly integrated to form the right object. Consider a visual display with an O and an X presented next to each other. Given that the identity and location are represented separately in the brain, how does the brain “know” to integrate the position “left” with the identity “O”, and “right” with “X”? Treisman's illusory conjunctions show, in a broader sense than location and identity, that the brain can get this binding of features wrong: the right features can be activated, but then be put together incorrectly. The other version of the problem looks at the question from the following perspective (Simon, 1994): how can controlled behaviour emerge from a brain in which there is constant, widespread activity and everything is connected with everything else? How can a given brain area distinguish relevant from irrelevant information without an all-knowing and neurally omnipotent homunculus supervising its activity? At one moment, for example, certain motor areas must be controlled by one kind of information, at another moment, by another. We may, in some task, first want to press a button if the colour of an object is green, and then again if it is presented to the left; or more ecologically, we may want to throw a ball and then hit it with a bat; we don't usually want to throw the bat and hit it with the ball, but we could choose to do so. What transiently links actions to objects? Evidently, there are temporary connections made between brain areas concerned with responses and areas representing objects in the outside world, just as at a lower level temporary connections are made between features to create an object representation. Binding is the laying of these temporary connections. From Simon’s version of the binding problem it becomes clear that simultaneous activation cannot in itself be a sufficient criterion for binding to occur. The problem in the first place was that many areas may be active at the same time. Von der Malsburg (1995) proposes that temporal synchrony may be how groups of cells are bound to each other; similarly, Roelfsema (1998) names neuronal synchrony as a possible tag for labelling neuronal assemblies (see also von Neumann, 1958; Abeles, 1991). Why should synchronization make a difference in binding? The Tanzi-Hebb learning rule implies the importance of simultaneous activation, but in what way could discharge frequency moderate the effects of simultaneous activation? One possibility is that binding is moderated by a third assembly. Given that any two assemblies are linked together by (at least) one third assembly, we can look at the effects of synchronization on this third assembly and its connections. If the two assemblies are simultaneously active but are not synchronized, the third assembly will at only some moments receive double activation. Synchronization leads to the third assembly being doubly activated each time the assemblies discharge. In this way a transient connection is made. The third assembly, which already connected the two bound assemblies (and perhaps had role in their occilations), simply becomes temporarily more ready to fire, and so the link is made without the need to slowly create new synapses, a process far too slow for the bindings necessary in task performance or perceptual integration. Thus, the chance that two assemblies will be bound is a question of the extent to which activation in the one can be predicted by activation in the other. Exactly which assembly is the third is unimportant, as is, in principle, where it is; binding can be achieved solely through the synchronization of two active assemblies, without the need for some homunculus to have knowledge of the position of the third assembly and the ability to specifically bias its threshold or activate it. The term “third assembly”, just like the “third variable” in interpreting correlations, does not even have to refer to a third assembly; maybe there are many more involved, but the principle of the third assembly mechanism of synchronization stays the same. The third assembly mechanism could function as a special method of learning in a dual-weight network, and explains how, after activity, synchronized or not, has died down, activation of one assembly results in the activation of a previously bound assembly. This attribute of binding, applied to the controlled re-entry (Posner & Rothbart, 1998) of posterior areas bound in a certain way within a task-context supplying network, is necessary for the models given for the processes underlying task performance given later to work. A question that arises here is in what way cell assemblies become synchronized. The discussion will explore this question in relation to TBI a little further, using some unexpected results. For now, the function of synchronization will be taken to be an implementation of predictablity, and thus symbolization as, if one cell assembly fires when another fires, it symbolizes (Crick & Koch, 1994) the same aspect of reality (see below). The two could be said to effectively be the same assembly. Initially unsynchronized assemblies must be able to become synchronized under certain conditions, for the flexibility which is the raison d’être of binding in the first place to be conserved. The theoretically most economical condition for synchronization seems to be continued simultaneous activation. Without speculating on the mechanisms by which simultaneous activation could enhance synchronization, the predictive function could be related to the statistical alpha (chance of an error of the first kind) in hypothesis testing. An adaptive mechanism would simply cause synchronization to be faster for larger alpha’s; such a mechanism would open possibilities for effective biasing (see 1.9.2, The cued lists task). Singer (1994) does go into the mechanisms for synchronization and writes the following: “This hypothesis [the assembly hypothesis that interactions between distributed cell groups should be variable and influenced by the constellation of features in the visual stimulus] requires that synchronization probability depends, to a substantial degree, on interactions between the neurons whose responses actually represent the features that need to be bound together. As cells in subcortical centers possess only very limited feature selectivity, one is led to postulate that corticocortical connections should also contribute to the synchronization process. This postulate is supported by the finding that synchronization between cells located in different hemispheres is abolished when the corpus callosum is cut (Engel et al., 1991a; Munk et al., 1992), which is direct proof that (1) corticocortical connections contribute to response synchronization and (2) synchronization with zero phase lag can be brought about by reciprocal interactions between spatially distributed neurons depsite considerable conduction delays in the coupling connections.” Perhaps the mechanism of synchronizaton has to do with the fluctuation of firing rate, where the faster assembly sets the slower off within its possible range, but somewhat before it would have without its influence, and in this way is “coaxed” to a synchronized rhythm. But what do these low-level processes have to do with the attentional control which is said to be damaged by TBI? Binding can be used as a control mechanism through the closure of circuits in networks, allowing assemblies to receive activation from only one, or multiple sources, depending on the state of the network in terms of which temporary connections have been laid (cf. Dalenoort & de Vries, 1998). The models for the tasks will illustrate how this control structure works, and how powerful it can be, especially in the sense of avoiding combinatorial explosions, in modelling complex control. 1.5.3 The conceptual network as a model for TBI Given the nature of the problems of the other models used for TBI, a network approach seems to be a natural choice. Once the functionality subserved by black boxes is shown to emerge from causal interactions between cell assemblies, the black boxes are destroyed, and the neuropsychological aim of relating brain to behaviour should be further aided by the closeness of cell assemblies to the physical level at which brain damage is best known. Importantly, there is nothing to stop an implemented conceptual network behaving like a model that does incorporate black boxes, or to preclude its modelling empirically or functionally defined dichotomies (e.g. controlled - automatic, or declarative - procedural) through different kinds of networks or interactions within a network. These models even hold the possibility of directly relating model activity to brain damage and brain imaging, if knowledge of the brain is used to the greatest extent possible to determine the modules and restrictions of conceptual network implementations. In subsection 1.9 conceptual network models will be described which link nodes together, suggesting a spatial analogue of what is present and what happens in the brain during task performance. This may be misleading, as expecting the assemblies represented by the nodes to all translate into seperate groups of neurons, spatially seperated in the brain, the one being activated by the other through some facilitatory or inhibitory connection, would assume an isomorphism which might not be justified. Perhaps nodes which are separate in the model overlap in the cortex; some nodes in the global network might represent a global state themselves, and the connections between them should then be seen as representing transformations between these states. Such state changes might map more easily to a dynamic systems model, but since the hypothesis of this study has to do with the relation of input representations to the task-set structure, whatever their exact implementations, the question of the true structure and dynamics of the task-set in the brain is not the primary concern. Task-sets will be taken to be adequately represented as spatially, not temporally, separable cell assemblies with a certain connectivity. The points which will be made later concerning the way executive processes can be modelled explicitly as the outcome of causal mechanisms are not affected by these considerations. The aspect of merging knowledge of the brain with a network model incorporating binding will form the basis of the approach to TBI taken here. But first some considerations about and a model for prefrontal cortex will be presented. 1.6 The function of prefrontal cortex Prefrontal cortex (PFC) has been proposed to be the seat of working memory. According to d' Esposito & Grossman (1996), two errors in the study of the physiological basis of the CES component of working memory have hampered the study of higher cognitive functions: the equation of “executive function” to frontal lobe function, and the view of the frontal lobes as an undifferentiated cortical mass. Responding to the first assumption, d' Esposito & Grossman state that it is likely that not all measures claiming to measure executive function tax only the frontal lobes, and that the frontal lobes are in fact not the only brain areas contributing to executive function. Primate (Wilson, Scalaidhe & Goldman-Rakiz, 1993) and human (Courtney, Ungerleider, Keil & Haxby, 1996) research suggest a functional differentiation of PFC, in contrast to the second assumption, that of PFC being an undifferentiated mass. Primate studies have shown a ventral - dorsal segregation for what - where information (Smith, Jonides & Koeppe, 1995) and a left - right segregation for non-spatial and spatial working memory (Smith, Jonides & Koeppe, 1996). A third functional segregation of PFC, proposed by Petrides, has also received some supporting evidence (d' Esposito, Detre & Incledon, 1995). Petrides' (Owen, Evans & Petrides, 1996) two-stage model proposes the existence of two executive processing systems in dorsal and ventral PFC. The ventral region is the site of initial reception of information from posterior association areas and active comparisons of information in working memory. Dorsal PFC is concerned with monitoring and manipulaton of working memory. d' Esposito & Grossman end their article "The Physiological Basis of Executive Function and Working Memory" with the following words: “Together, nonhuman and human physiological studies support the notion that working memory is not a unitary system but is subserved in part by a complex network within dorsolateral prefrontal cortex where subdivisions are organized in a dedicated fashion to support the temporary retention and manipulation of a particular type of material. Further work is clearly necessary to determine the nature of these functional subdivisions within prefrontal cortex.” Together with the importance of finding restrictions for structural models, such as the conceptual network, this shows the interdependence of functional and structural models as discussed earlier. Petrides' model will play an important part in creating a meaningful conceptual network implementation for the processes that may be damaged in TBI. Another piece of physiological knowledge which will be used in this regard is the dissociation of the neural correlates of implicit and explicit memory. 1.7 Dissociation of the neural correlates of implicit and explicit memory Rugg, Mark, Walla, Schloerscheidt, Birch & Allan (1998) compared the ERPs evoked by studied and new words correctly and incorrectly classified as such in a word recognition test. The important findings for the purposes of this paper are, firstly, that studied words not consciously recognized evoke different patterns of activity at parietal electrodes than new words; secondly, that the activity at parietal electrodes was the same in magnitude for recognized and unrecognized studied words; and thirdly, that recognition of words correlated with activity at frontal electrode sites. This physiological pattern is central to the hypothesis about the effects of TBI and supplies a further restriction to the models. 1.8 The effects of TBI on cognition A criticism of neural networks by Brown & Hagoort (1989) is the following (translation by the author): “Ultimately, a neural model simulating a cognitive function gives us no insight into the human mind as long as this simulation cannot be characterized in terms of a well specified (mathematical) abstraction of the actual simulation. […] If connectionism wishes to design adequate models of human cognition by taking its heuristics from the organisational principles of the brain, then at some moment the leap from mathematically uninsightful simulations in a neural jungle of neurons (nodes) and their relations (connections) to an abstracted, functionally insightful description of the mental software must be made.” The abstracted merged model of cognition used here has the following components: a set of long term representations (LTRs) of stimuli and concepts, implemented as cell-assemblies located in posterior areas of the brain which symbolize by selective response certain aspects of reality (Crick & Koch, 1994); transitory neural contexts connecting LTRs to actions in a task-relevant manner located in ventral PFC; and structures located in dorsal PFC which react to certain conditions by changing the ventral neural contexts. This general model will be implemented in the models for the five experiments used here. In all cases, for the system to perform the tasks, posterior LTRs must be bound to ventral contexts subserving task-sets. LTRs can be seen as types, which become tokens when they become part of the ventral context by being bound to an assembly there. This process of bringing an LTR into a neural context by binding will be labelled tokenizing, and it is this process which is hypothesized to be the locus of effect of TBI. The reason for this hypothesis is the nature of the physical damage caused by TBI: frontal "coup / contre-coup" damage occurs, as well as shearing damage to axons (Adams, 1982). This disrupts dopaminergic input to PFC (Adams, 1982) and neural connections between PFC and other brain areas such as posterior parietal cortex and subcortical areas activated in working memory tasks (McDowell, Whyte & d' Esposito, 1997), and so would be expected to disrupt the tokenizing of LTRs. This tokenizing hypothesis will be tested in various ways through five experimental tasks. The following subsection will describe these tasks and show how they will be used to attempt to draw conclusions from the collected data. The hypothesis can be formalized by way of an example, in which a function describing a very simple conceptual network is given and the locus of TBI in that function is given. Given the following network, Dashed lines represent temporary connections, continuous lines permanent connections. The circles are nodes which fire (that is, activate connected nodes) if they receive activation above a critical threshold. The labels of the nodes do not imply differences in their characteristics. the activation of an output node corresponding to one of the LTR’s is given by Activation (output, time 1) = weight (X, output) * Activation (X, time 1) + weight (input, output) * f[Activation(input, time 0) / (Activation (input, time 0) – Mean (Activation (other inputs, time 0)))] * Activation (X, time 1). So the output node does not only receive activation from the directly connected organizing node X (the term organizing will become clearer in later, more complicated models; note that it refers to the function of node X as determined by its place in the network, not (necessarily) any special qualities it might have), but also by the temporary connection between X and one of the nodes connected to a certain input, via the permanent connection of that node to an output node. The effect of TBI is hypothesized to be on the function in italics substituted for weight (X, input). In this simple case, this weight depends on the selective, or relative, activation, at the time of binding, of one of the LTR nodes and node X: if there is a strong enough selective activation in the period available for binding, the temporary connection will be correctly set. In the list models, the selectivity of the activation of the relevant organizing, or task-set node (that is, the relative activation of the most activated one in comparison to the others) is also a factor. An interesting aspect of the formalization is that effects of synchronization could be seen as the normal effects of selectivity, where the function is applied to a set of short time intervals. 1.9 The tasks Three classes of tasks were used: list tasks, cued modality list tasks, and set-switching tasks. Each of these tasks was designed to test the hypothesis that tokenizing is damaged in TBI. Tokenizing has been defined here as the binding of posterior LTRs to a ventral prefrontal task context, and it is this process the list tasks were designed to test. The cued modality lists task and the set-switching task tested the integration of task relevant information and the quality of what, in terms of ACT-R, could be called goal memory, respectively. Though performance on these latter tasks would be expected to be affected by TBI under a CES or SAS model, the "tokenizing hypothesis" proposed here would predict that effects would not be found to be primarily located in the CES / SAS or goal memory itself, but in the binding mechanism subserving the implementation of the “instructions” for control subserved by these systems / this memory. The tasks will now be described in short, informal conceptual network models for them will be given and the rationale for their interpretation will be discussed. 1.9.1 The lists tasks In these tasks stimuli were presented, one at a time in a multiframe display, in rapid succession. Each stimulus was expected to be presented long enough to activate its LTR, based on a forced-choice test where subjects had to identity a masked letter. The SOA of the mask was shortened until the subject could not correctly identify the letter, and the presentation time of the stimuli was set 10 msec above this level. In the general model incorporating Rugg et al' s dissociation of implicit and explicit memory it is possible that the binding occuring between the task context and the LTRs may be slower than the presentation tempo of the stimuli. In that case, multiple LTRs will be simultaneously active before binding occurs, and the binding that does eventually occur may be in wrong order. In this way, the influence of tempo on the amount of perturbations in the reproduction of a list can be seen as a measure for the efficiency of binding between LTRs and the task context, i.e., for tokenizing. The basic model for the lists tasks is given in figure 1. The method used for holding presented stimuli in the right order, in such a way that the order can be used later in processing, and the functional modules in the models are based on conceptual network models (Dalenoort & de Vries, 1998). The models given for the lists tasks are intended primarily as aids for understanding the rationale underlying the chosen tasks and analyses. For the later tasks, the models illustrate a way in which executive processes can be viewed as causal processes in a network. The structure of this model allows for good control of the spread of activation through the network, and can (and will) be generalized to other implementations. Though only lists of three stimuli were used in this experiment, the changes necessary - both in modelling and in the brain - for reproducing lists of different lengths are very slight. Using mixed modality lists makes more information available. If only simultaneous activation was the condition for binding, the performance on mixed lists should be somewhere between performance on the simple lists. However, if rhythm is a factor in binding, different predictions must be made. Figure 1. The basic model for the list tasks Activation is held in resonance between a node and a subnode. Active subnodes convey less activation than necessary to raise a connected, unactivated node above its critical threshold so that convergence of activation from subnodes is necessary to further activation. Nodes raised above their critical threshold by a resonating subnode inhibit the activating node. Nodes marked with different letters can bind if simultaneously active. If an assembly with an activation level below its critical threshold is bound, its activation level is raised above its critical threshold. The presentation of stimuli causes activation of the input nodes and node A is active at the start of a trial. Using these rules, the reader can determine the way the connectivity of the network is influenced by the temporally distributed stimuli and how the choice of a response is based on this stimulus-dependent connectivity. For more details, see Dalenoort & de Vries (1998). From everyday experience we know that people don' t make mistakes about the modality-category of visual and auditory information – the synesthesia of poets and psychotics excepted. Even so, if we look within the brain, assemblies in visual and auditory areas must still be bound to higher areas to be integrated, as having the processing going on in one modality directly moderate the processing in the other, as would be the case if integration of multimodal information concerned the binding of sensory-specific areas, seems extremely unadaptive and intuitively unappealing. It would, at first glance, be expected that to these higher order areas, one assembly of simultaneous activity would be quite like another, whatever sensory organs originally evoked them. Yet, we effortlessly avoid mixing things up. If spike resonance is accepted as an additional dimension influencing binding (Dalenoort, 1998), different frequencies of discharge frequencies for the visual and auditory modalities would separate the types of information, and higher order networks could confidently use this differentiation in further processing. If this is the case, multimodal lists will be harder to prepare for. If a subject knows a simple list is coming, a dorsal PFC network may bias (biasing has been suggested as a control mechanism by Singer (1994)) the assemblies to a single frequencies (how this could be modelled without any need for a homunculus, based on the effects of instruction on the pattern of connectivity, can be seen in the models for the cued-lists and set-switching task). In this way, synchronization between assemblies will be achieved more quickly in the simple list tasks. If the subject must prepare for a mixed list, the bandwidth cannot be narrowed down to that of one or the other modality. A broader bandwidth is not the only reason a disadvantage would be expected for mixed modality lists. For example, simply opening up, by whatever mechanisms, a pathway to a second area of the brain introduces a new source of noise. The question which reason is more important cannot be answered in this study. However, both effects can be described as reducing the selectivity in the binding part of the formalization in subsection 1.8. Differences in the speed with which visual and auditory information is processed - the time between stimulus presentation and activation of the LTR - would not be such a problem, as the expectation would then be that the error curve would lie somewhere between the higher and lower error curves for the simple tasks; not above the higher curve, as introducing faster stimuli would sometimes be relatively advantageous. The possibility of modality-specific list structures, whose contained order is integrated into a higher-level network, is tested by comparison of intra-modal perturbations and overall perturbations at equivalent separation times. Finally, the possibility of some unknown factor causing one modality to be systematically reported before the other would confound a simple interpretation of lowered performance in terms of sychronization. The validity of this alternative explanation can and will be tested (see Results). 1.9.2 The cued-lists task The aspect of biasing in multimodality opens up the first of two ways to explore the way dorsal PFC / CES / SAS manipulates information. In the cued lists tasks subjects are given information on the order of modalities in which the stimuli will probably be presented. Three effects of this information could be imagined. Perhaps a dorsal PFC network would bias the various nodes in the task context, and the cognitive system would from then on work just as the uncued tasks, except with the advantage of ordered biases. Then again, perhaps first the stimuli would be registered with or without them being ordered, and then the information given in the cue would be used to straighten out the order on the basis of intramodal position of stimuli and the cue. Finally, both effects could occur. On the basis of a pilot study, the second mechanism is taken as a hypothetical restriction for the cued lists task model. The main test for the validity of this model concerns the response latency in comparison to the mixed lists task. If integration of the cue information has to wait until after the stimuli have been presented, it will take longer for a response to be produced. Certain effects of separation time which are consistent with the second mechanism were found and will be discussed in the last chapter, Discussion. The model for the cued lists task is given below. In this model, the dorsal PFC is set up by the cue and a very simple task-set to endogenously create the conditions for binding on the basis of the cue (for modality) and the activated LTRs (for identity). Figure 2. The cued-lists task The cue is represented in the connectivity of the subnetwork to the left of the figure, which is set up by binding caused by reading the cue from the left to right, activating the elements one by one (that is, seperated in time). The model assumes that the intramodal order is preserved at a lower level than the to be reported intermodal order (the test of this assumption is given in Results). 1.9.3 The set-switching task In the fifth task, subjects had to remember one, two or three targets consisting of combinations of the letters d and q. The task was not to search for these targets, but to compare one of them at a time with a single stimulus which was also a d-q combination. So in one trial, up to three stimuli were presented, on different displays, each requiring a separate response and using a different target. The model of this task in figure 4 illustrates how the manipulation of working memory - changing the target - can be represented by reconfiguring a task network. One aspect of the set-switch task is to test the effects of TBI on the goal stack. Will patients have more trouble than controls in holding the targets in memory? Will their target-switching be less efficient? CES / SAS models would predict these effects. The tokenizing hypothesis, however, only predicts slower responses for later targets, due to the slowed binding interacting with the weaker activation of the later target. A second test of this effect is made possible by a variation in the delay between stimuli, effecting the time the activation levels of later targets have decayed. Figure 3. The model for the set-switching task The (order of) the targets is represented by the connectivity of the string of (context manipulating) node pairs connected to LTR's and the choice network, which is laid by reading the targets. The active target causes two instances of binding at the context embodying level, between the LTR and choice nodes bound by each of the target nodes in the active pair. An interesting aspect of this model is that, for the switch to be controlled, a double-activation bottleneck (at node B) had to be used. It has been found (Monsell & Rogers, 1995; Allport, 1994) that switching a task-set depends on exogenous triggering. In the light of this model, this need is adaptive: it prevents irrelevant but prepared goals from interfering with the presently relevant one. To get around the need for exogenous control, a subject could generate his or her own trigger. The ability to create and maintain the extra architecture necessary for this might be what underlies how likely subjects are to “fail to engage” (de Jong, 1995) preparation for a trial in the intertrial interval. If subjects do use the delay to prepare for the next target, and this effect is stronger than the effect of activation level decay, then reaction times should be shorter for longer delays. A more detailed description of these tasks will be given in the next chapter: Method. 2. Method 2.1 Subjects Four subjects, three male, one female, who had suffered TBI and had been selected for clear axonal damage formed the TBI group. They varied in age from 20 to 50. A further four subjects formed the student group. These subjects, three male and one female, were all healthy, in their early twenties, and all studied or had studied psychology at the Rijksuniversiteit Groningen. 2.2 Tasks The tasks were programmed and presented using MEL 2 and a PC. Output and input were given and received by monitor, computer tone generator and keyboard. Subjects performed the tasks in the order letter lists, tone lists, mixed lists, cued lists, set-switch task. A break was suggested between the list tasks and cued list task. The experiment took about an hour and a half, including breaks, to complete. 2.2.1 The lists tasks There were three lists tasks, which the subject performed in the order letter lists, tone lists, and mixed lists. Before the letter list task proper, subjects underwent a calibration task intended to set the presentation time long enough for backwards masking at rapid presentation rates not to effect performance. Two letters were presented, the second masking the first with no interstimulus interval (ISI), so that its SOA was the presentation time of the first letter, and the subject typed in the first letter. As long as the subject typed the correct letter, the presentation time of the first / SOA of the second stimulus was reduced by 10 msec. The second stimulus stayed on screen till the response was given to avoid perturbations. After the first incorrect response, the SOA was not reduced. At the second incorrect response, the SOA of the second stimulus was registered, 10 msec was added and this time (the stimulus presentation time, spt) was used as the presentation time for the stimuli in the rest of the task. The range of possible presentation times was 20 to 160 msec. The subject was presented with three letters randomly chosen from the set o, k, l and p, which appeared one after the other at the same position. The time between the stimuli (separation time) was varied, from 0 to 400 msec. There were ten separation times used in all the list tasks. After presentation, the subject was to re-type the list, taking as much time as needed. There were 70 trials, as in the rest of the list tasks. The tone lists task did not have a calibration phase, as the perception of the category of the tones was not an issue, as was the case with the letter identities. Only two tones were used, high and low. Subjects reproduced the tones using the keys o and k, and were attended to the relative high – low positions of o and k on the keyboard. In other respects the task was the same as the letter lists task. The mixed lists were preceded by another calibration, equal to that of the letter lists task. The lists in this task could consist of the letters a and s, and the high and low tones used in the tone lists. Response to the letters was with the corresponding keys, response to the tones was as in the tone lists task. Again, the rest of the task was equal to the simple lists tasks. In all list tasks the subjects were instructed not to try and respond quickly, but to take their time to find the right keys. Errors and perturbations were used as dependent variables. An error was logged for every response which did not correspond to the stimulus presented at the corresponding position in the lists of stimuli of responses. An error on a given position was also logged as a perturbation if there was another position for which the response would have been correct, and which had not already been used to log an earlier error as a perturbaton. For example, if the stimulus list was o l p and the response was l l l, the first position of the response would have been both an error and a perturbation, and the second and third positions only errors. The time taken between the offset of the last stimulus of the list and the first keypress of the response was also registered. 2.2.2 The cued modality lists task The cued modality lists task, or simply the cued task, was largely the same as the mixed lists task. However, on each trial before presentation of the list, a cue appeared indicating the probable sequence of modalities which would occur. The cue took the form of a list of words, such as “letter tone letter,” and was presented for either one, two or five seconds. Due to the longer time per trial, only two onofts were used, 30 and 75 msec. The cue was valid on 80% of the trials. Subjects were instructed to use the cue, but also knew of the 20% chance of an invalid cue. There were 84 trials, and subjects were given two breaks for as long as they wanted. As in the list task, the emphasis was strongly laid on accuracy, not speed. Errors and perturbations were used as dependent variables. As in the lists tasks, the time taken before responding was recorded. 2.2.3 The set-switch task In the first block of this task subjects were given a target to remember, and subsequently saw a stimulus to be compared with the target. If target and stimulus were equal, subjects were to press the z-, otherwise the /-key, which came down to left for yes, right for no (the Dutch for right as “correct” is not equal to the word for right as in “to the right”). There were three practice runs and then fourteen trials. The targets and stimuli consisted of two letters from the set d, q. In the practice runs subjects could get used to the task and the positions in which targets would appear. In the practice runs the target was presented below the stimulus. The target was changed each trial. In the second block, a trial consisted of two responses. Subjects had to remember two targets. Then a stimulus was presented, which was to be compared to the first target; after the response was made, another stimulus was presented, which was to be compared with the second stimulus. There were, again, three practice runs and then fourteen trials. In the practice runs, the active target was given so that subjects could get used to the new task-set. Again, the target set was varied over trials. The third block was equal to the second, except now three targets / responses were used. The delay between response and an eventual next stimulus was varied, and could take 50 or 2000 msec. Subjects were told to react both as accurately and as quickly as possible in this task. Reaction time and accuracy were recorded for each response. 4. Discussion 4.1 Interpretation of the results The only comparisons available at the time of writing were those between students and TBI patients. The confounds of intelligence and age are obvious, and further research using better matching seems the only way to ascribe the tokenizing difference specifically to the effects of TBI. This discussion intends primarily to set out how the task-data might relate to tokenizing, that is, the binding of LTR’s to task set nodes, as an underlying process / cause for differences between TBI patients and other groups in general, but the best guess at the reason for group differences with the present, unmatched groups does seem to be a difference in the connectivity between anterior and posterior brain areas, which is at least in part a TBI effect. 4.1.1 The letter-lists task The letter-lists task did not strongly support the tokenizing hypothesis, as the only difference between groups on errors and perturbations was at 60 msec (and 400 msec for perturbations; however, this result, given non-differing results for shorter separation times, does not support the hypothesis). The difference between groups could be better characterized, within the task model, by a slowed spread of activation within the non-temporary connections in the task set network, as responses for the first position were more accurate than for the second and third in the TBI group. This result also supports the interpretation of differences in terms of central processes, and not backwards masking. The significant difference at 60 msec was interesting, as this separation time was part of an unexpected “bump” in the error / perturbation curves. Both groups showed a strong decrease of errors and perturbations at 45 msec separation time followed by an increase at 75 and 60 msec for the student and TBI groups respectively, before dropping to baseline. Both effects can be tentatively explained a posteriori within the binding network framework. The local accuracy optimum separation time (LAOS) around 60 msec could reflect a natural firing rate of cell assemblies in the task set network. Stimuli presented at the right rhythm would slot in; stimuli presented somewhat slower would experience more hinder from the disharmonics than help from the longer separation time, depending on how quickly synchronization through continued simultaneous activation could be achieved. The LAOS covers a broader range for student group (t-testing for the increase in errors and perturbations from 45 to 60 visible in the curves gave alpha’s of .179 and .266 for the TBI group, whereas the student group showed a slight, non-significant (alpha = .772, alpha = .657 respectively) further decrease in errors and perturbations). Speculatively, this could reflect a slower synchronization of rhythms of task set node and LTR. With the same distance in rhythm from natural firing rate to presentation rhythm, the translation from that distance to time needed for synchronization (and thus binding) would be larger for the TBI group, reducing the time separating the binding of the LTR of stimulus n and the activation of the LTR of stimulus n+1 (the effective separation times) more for that group than for the students. 4.1.2 The tone-lists task While visual inspection of the curves for this task showed a consistently better performance for the student group, the overall differences did not reach significance, and only one out of twenty comparisons of errors and perturbations reached significance. Both the lack of significance and the non-significantly but visible upwards shift of errors and pertubations on the auditory task for the TBI group may reflect a difficulty in encoding responses, which were less compatible than the “press ‘o’ for ‘o’” type of response mappings in the letter lists task, in the TBI group. This may introduce a source of variation unrelated to separation time. 4.1.3 The mixed lists task The mixed lists showed strongly non-parallel curves for the two groups, the students reaching an apparent baseline around 75 msec (60 – 75 msec shows the only significant difference (alpha = .018, alpha = .022 for errors and perturbations respectively) in consecutive errors and perturbations from 60 msec on), while the TBI group made no significant improvement from 60 to 400 msec separation time (figures 3.6 and 3.7). However, the significant differences in this range, at 100 and 150 msec, seemed to be primarily due to a response bias related to visual dominance (Colavita, 1982). Visual dominance refers to the phenomenon that if a visual and auditory stimulus are presented together, the response tends to be to the visual stimulus. The correlation of errors and perturbations with visual dominance was significant and very high (around .7 at 100 msec, and around .9 at 150 msec separation time, implying that if a perturbation was made, it usually concerned switching an earlier auditive stimulus with a later visual one), which was not predicted by the broadened bandwidth hypothesis. Egeth & Sager (1977) found that visual dominance is affected by the necessity of responding to the light signal, the probability of the light, tone and dual trials, and instructional set, and thus “is not a phenomenon that depends on the consequences of “hard-wiring” of the nervous system. Instead, these findings implicate a system that is under at least partial cognitive control.” If task set nodes are set to the visual frequency in response to expectations of stimuli of both visual and auditor modality, and not to an average, the response bias can be explained in the same way as the LAOS. Both groups have to synchronize a larger distance to bind auditive stimuli, but students have less problems doing so, and have enough time to avoid perturbations. The TBI group, on the other hand, often fails to reach synchrony with the auditory stimulus by the time the visual stimulus’ LTR becomes active, and may then synchronize with the second, visual stimulus. The use of modality specific sub-networks was supported by the lower perturbations within modalities than the total perturbations. Apparently, stimuli of the same modality are held in order at a lower level than the task network in which all stimuli have to eventually be integrated. 4.1.4 The cued-lists task The TBI group did not seem to have more trouble integrating the cue information with identity to form order information, as the improvement due to cueing was no less for that group than for the students. The response latency in the cueing condition was not longer than in the uncued mixed modality condition, which argues against a model in which cue information is used to lay temporary connections after stimulus presentation, and hence for a model in which nodes are pre-tuned to the modalities named in the cue. If this latter model and the interpretation of visual dominance in the preceding subsection are correct, the TBI group might be expected to suffer more than the student group if a visual stimulus was cued but an auditory stimulus was presented, but not if a falsely cued visual stimulus was presented. This was the case: the effect of invalid cueing was larger (approaching significance, alpha = .077) in the TBI group than in the student group, but only for falsely cued auditory stimuli. A node tuned to visual information is apparently harder to re-bias to bind with auditory information than the converse. The only significant effect of cue presentation time, the high accuracy of the student group at the shortest presentation time, was unexpected. The hypothesized effect was a monotonous decrease in perturbations, stronger for the student group, as presentation time increased, providing more time for the biasing of task set nodes. The high accuracy at 1000 msec was also present when the cue was invalid, so it appears to have nothing to do with group differences in tuning task set nodes. An alternative explanation could be that quickly presented cues did not have time to build up a specific dorsal network specifying the modalities at various positions and tuning them accordingly, but did prime the LTR’s associated with the words in the cue, and perhaps also enhanced the activation of the task network. 4.1.5 The set switch task In the set-switch task, the students’ reaction times for short ISI’s (which tested the efficiency of exogenously triggered set-switching by not giving subjects a chance to switch set between stimuli) showed a slowing trend that followed the presentation order (alpha = .017, alpha = .031 from first to second and first to third position respectively). Ideally, the TBI group would have shown a magnified slowing, reflecting difficulties laying the correct connections between task set and LTR’s. The TBI reaction times were, however, simply much larger, and unsensitive to the position of the choice in the run. The effect of delay turned out to be more informative. The TBI group showed no increase or decrease in reaction time as a result of increased delay. The student group however showed a large improvement in reaction time. As the only information available in the time between presentations was the target information held by the subject, and the only effect delay would be expected to have in the simple model given for this task is a decrease in activation level of the target-holding assemblies, improvements due to an increase in the delay must have to do with reactivation of the target and / or laying of the connections forming part of the next task set. The difference between reactivating a target and laying connections is slight in this model, as the target is simply a set of causal connections; reactivating the central target node causes the simultaneous activities which in turn cause new bindings and a new task set. In accordance with this, when the ISI was long the second response for the students was no slower than the first response, and the third no slower than the second; in contrast to the effects of position in the short-ISI condition. The accuracies in block three – a test for CE ability, in this case the ability to hold a number of targets and change the task set network to use the right one at the right time - was not significantly worse for the TBI group. 4.2 The tokenizing hypothesis: confirmed? Taking all the evidence supplied by the results together, it seems that the disturbance of CE function in TBI might be explained by a problem in binding assemblies in ventral PFC which are part of a network subserving an abstract (i.e., not including the specific LTR’s needed to perform a task) task set with LTR’s, which only occurs in certain situations. Specifically, it seems to be a delay in synchronization when task set nodes are incorrectly biased which prevents efficient binding of this sort, as suggested by the broader range of LAOS and the interpretation of visual dominance effects. Thus, the disturbance of CE function is not actually a disturbance of the dorsal PFC networks which subserve the executive mechanisms of the CES, whatever that actually is (see below); even though not damaged themselves, their function depends on the efficiency of those processes involving the connections between anterior and posterior areas which the tokenizing hypothesis states to be disturbed. 4.3 Relating the tokenizing hypothesis to Timmerman & Brouwer’s spread of activation theory and ACT-R In the binding network framework, tokenizing is easy to understand. A task set node which is part of one of a number of possible closed circuits which is bound to an LTR comes to symbolize whatever that LTR symbolizes (see Introduction). The information represented in the LTR comes to be embedded in a task network – tokenized - and can thus be used in controlled thought and behaviour. CE / dorsal PFC networks which in reality blindly manipulate nothing more than bound assemblies come in this way to manipulate symbolic information, in an equally real sense. However, the processes which are focussed on in a binding network are somewhat different from those that receive attention in ACT-R. For example, in the set-switch task the dorsal PFC network could be seen as a goal stack. The network approach emphasizes the mechanisms by which the current goal transforms the task set, which could be equated to the set of potentially selectable productions. These mechanisms, the causal reasons for why only certain productions are selectable at a given time, are black boxes in ACT-R. Tokenizing itself doesn’t figure in ACT-R; what is important about the assignment of values to slots in ACT-R are the rules which specify which values are assigned, not the underlying mechanisms, let alone the physical processes. Even so, there are analogies to be drawn. The importance of the structure of declarative memory is not reflected in the binding networks given here. However, LTR’s are the obvious analogy to chunks, and, using ACT-R and Damasio & Damasio’s (1994) convergence zones as inspiration, the very simple LTR’s used here could be built up to the complexity and functionality of chunks. Convergence zones are centres of feedback and feedforward connections which can activate the component parts of memories (Damasio & Damasio, 1994). A way in which the structure of chunks could be built into a binding network is given in figure 4.1, where a task set cell assembly “asks for” and receives the value of a certain slot. Figure 4.1 A more complex LTR The relationship of productions to a binding network is a little less trivial. It was written above that “The network approach emphasizes the mechanisms by which the current goal transforms the task set, which could be equated to the set of potentially selectable productions.” This is really a question of definition. If we simplify a production to whatever links declarative information to a goal slot, and translate this linkage into tokenizing an LTR to a task set cell assembly, then the only productions that could ever fire, are those which link an LTR to the presently active task set network. These available productions are implicit in the potentially closeable circuits (the set of potentially selectable productions) and the consequences of the closed circuit (the action side of the production). The conditional side relates to the prevailing conditions of the network. For example, the retrieval part of the firing of a production searching for an input to the “first letter” slot in the “encode letters” goal for the letter lists task is only performed if the first tokenizing cell assembly is active; the production would contain a “node1 = on” condition to fire, and productions with a “node2 = on” condition would not. The execution of an action part of a production firing is split from retrieval in the models given here; a number of other retrievals are performed first. The slight variation of Timmerman & Brouwer’s theory of the effects of TBI should now be clear. While spread of activation is translated into controlled binding by convergence zones, and the tokenizing hypothesis does not predict effects on this process (except perhaps through focal temporal damage damaging structures necessary for memory function), it is still the retrieval of declarative knowledge that is stated to be disrupted – the difference is that the locus of effect is now the actual mechanism of retrieval, resulting, in ACT-R’s terms, in a weakening of the association strengths between goal and declarative chunk elements, as opposed to relations between the chunks to be retrieved. 4.4 Relation to CE-type models Having a CES in a model does not necessarily imply introducing a homunculus, but the term “central” may be misleading. The models for the cueing and set-switch task show that the mechanisms the CES uses could be implemented by a hierarchical pair of networks, one for creating a task context and one for manipulating that context. These mechanisms do not require an all-knowing centre or cause an infinite regress of explanation. Tokenizing combined with informationally blind binding on the basis of, for example, contiguities in instructions, provides for explicit causal processes doing just what the CES does. An element of CE function not modelled in the tasks used here is the self-programming of the task set and manipulating networks. However, since both task set and manipulating are nothing more than configurations of bound cell assemblies, the processes necessary for self-programming should be no different than those used in applying a cue or changing target set. The deeper question of what is the cause of these processes shows how it may be a fundamental error to look inside the subject for a complete picture of executive control. Baddeley’s approach, accepting an inner homunculus who “steers” executive processes, even if well-defined in a SAS framework, as a “useful lie”, may mean asking the wrong questions, again (Simon, 1994); what if, in reality, the homunculus does really, literally, physically exist – but outside the subject’s head? What if there is an extended executive control system, instead of a central one? The discussion will conclude with an alternative approach, which will accept Baddeley’s “stripping away of functions” but also imply some changes in methodology and the conceptualization of control. Posner & Rothbart’s (1994) neuronal theory of mind could be applied to modelling the mechanisms of the CES, where the anterior cingulate gyrus might be seen as a convergence zone for systematically building up task networks through simultaneous activation of arbitrarily located assemblies in PFC by the environment, with language as an especially important factor. Connections to and from the basal ganglia could perhaps implement the necessary reverberations and inhibitory connections (Posner & Rothbart actually write about the basal ganglia dampening down the activity of anterior locations in the case of mismatches between activation of posterior locations reflecting input and anterior locations reflecting expectation, but this seems close to the general function of inhibitory connections). In this case, the location of the “third assembly” (see Introduction) would not be arbitrary. However, all this still has to do with mechanisms of the CES; as an example, it is, in the context of selective filtering, still answering the question “how is the filter set?”, not “who sets the filter?” (Styles, 1997). The discussion will conclude with some thoughts on an evicted homunculus and how looking at the homunculus problem in this way could, or should, change psychological research of control. 4.5 Further research It seems that it would be interesting to continue research done within this framework of merging binding networks with neuropsychological theories. Fundamental questions to be answered concern the way in which a better declarative memory structure could be implemented, how productions relate to the task set network, and how new task networks are self-programmed; and experiments must be designed and models programmed to test the proposed answers to these questions as objectively and unambiguously as possible. Concerning TBI, it seems important to integrate models for higher level cognition with work on lower-level disturbances, such as the experiments Greenlee and Kraft are going to run on the magno- and parvocellular pathways for visual processing. The lists tasks explicitly cut out perceptual differences in an attempt to avoid confounding, but the fact was that this measure was necessary: the TBI group required a longer presentation time for recognition, and in real life there are no pretests tailoring the perceptual demands to perceptual efficiency. Later experiments might replace the pretest phase with a number of conditions using different presentation times, as an exploration of what kinds of interactions between perceptual and tokenizing difficulties might exist. Finally, in regard to TBI, the severity of tokenizing disturbances could be related to various populations and types of damage. Another area of research would concern the physical structures and processes used speculatively here as the basis of functions of binding networks. For example, the proposed re-entry of LTR’s for stimuli in the list tasks might be observable in ERP’s, and if so, the timing of the processes in the model could be exactly temporally bound to physiological findings. Brain imaging could provide markers for cognitive processes as modelled by conceptual networks and thus test models, and conceptual networks could provide an interpretory framework for brain imaging studies. Specific extensions of the tasks used here could focus on replicating and analyzing the LAOS, using a more fine-grained distribution of separation time in the 30 – 100 msec separation time range. It would be interesting to try and model the effect of TBI on both the LAOS-range, if it turns out to be robust, and the separation time – accuracy curves changing only a parameter representing synchronization time. The influence of response mappings for auditory stimuli could be tested so that the heightened baseline found in this study could be better interpreted. The influence of cue presentation time in the cued lists task could be further studied with a wider range of presentation times. Whereas the time taken in the set-switch task for encoding the targets was a choice of the subject in this study, time for study as an experimenter controlled variable might show interesting effects on accuracy(-profiles), and interactions of study time with group membership. Different analyses of the lists tasks data, some of which would necessitate a reconfiguration of the data set, might also prove interesting. The use of the pretest also seems to require attention; repetitions should be excluded, perhaps a smaller set of similar letters should be used instead of the whole alphabet, and perhaps a longer test, with stricter “progression” criteria and more robust criteria for setting the eventual presentation time would be sufficiently more reliable to justify the longer testing time; or perhaps the perceptual performance on this kind of test is simply inherently variable. In that case, an alternative method might be the use of electrophysiological markers for the activation of a memory trace to calculate effective separation times. A fundamental question of further research is how the homunculus problem should be handled. TBI, with its attentional effects, is an area in which the answer to the homunculus problem may influence the research done and theories proposed. It was suggested earlier that the homunculus, the little man who was thought to live in the brain, activating the carefully analyzed and modelled mechanisms of control, actually lives outside the brain. This is to say that in most cases, the homunculus will be you, the researcher, as you instruct the subject, by doing so activating LTRs which in turn build up task sets. In Styles’ example, the researcher sets the filter, even though the mechanisms by which the filter is set are located inside the subject. In general, the “homunculus” will be the whole of environmental influences, as perceived by the subject, working on the CE mechanisms. So Baddeley’s approach of stripping away functions would result in making an inner homunculus redundant, as there is nothing inside the subject’s head but the CE mechanisms - but this will not capture the complete picture of control. To study the homunculus in this broader sense might require a different way of looking at the world than a western / Catholic (Rabbitt, 1997) one focussing on individialism and free-will. Desmond Tutu presents a different view, which in the opinion of the author might serve the study of control better: the Afrikaans ubuntu. Two possible implications of this worldview on research could be the following: the unit of study should not be just the one subject; and the unit of time should not be just the task itself, but the “build-up” to the task as well. In conclusion, I will cite Bishop Tutu’s words on this worldview. “Ubuntu is very hard to explain in a western language. It is about the essence of being human. […] It says something like: ‘My humanity is intertwined, is inextricably connected to yours.’ We belong to the same package. We say: ‘A person is a person through other people.’ It is not: ‘I think, therefore I am.’ It is: ‘I am human because I belong. I participate, I share.’”