|Risk Assessment for Neurobehavioral
Environmental Health Perspectives, Volume 104, Supplement 2, April 1996
Neuropsychological Approaches for the Detection and Evaluation of Toxic
UMDNJ-Robert Wood Johnson Medical School, Environmental and Occupational
Health Sciences Institute, Piscataway, New Jersey
The purpose of this paper is 3-fold: a) to review briefly the
neuropsychological tests that have been used to evaluate the effects of
neurotoxicants; b) to identify individual factors that may create
heightened sensitivity to neurotoxicants; and c) to discuss test
parameters that will increase the sensitivity of neuropsychological
tests for detecting symptoms in low-level exposure situations. While the
body of literature on neurobehavioral toxicology has increased
dramatically during the past 10 years, it remains difficult to discern
which tests are most effective in detecting behavioral effects even
among workers with significant exposures. Few investigators have
evaluated the interactions between individual differences, such as
gender and psychiatric function, and exposure to neurotoxicants.
Detection of behavioral performance decrements among uniquely
susceptible populations such as those with sensitivities to low-level
exposures (e.g., multiple chemical sensitivities) will require more
difficult tests than are frequently used in current neuropsychological
test batteries. -- Environ Health Perspect 104(Suppl 2):239-245 (1996)
Key words: neuropsychological, neurotoxicants, chemical sensitivity,
individual differences, psychiatric, vigilance
This paper was prepared as background for the Workshop on Risk
Assessment Methodology for Neurobehavioral Toxicity convened by the
Scientific Group on Methodologies for the Safety Evaluation of Chemicals
(SGOMSEC) held 12-17 June 1994 in Rochester, New York. Manuscript
received 1 February; manuscript accepted 17 December 1995.
Address correspondence to Dr. Nancy Fiedler, UMDNJ-Robert Wood Johnson
Medical School, Environmental and Occupational Health Sciences
Institute, 681 Frelinghuysen Road, Room 210, Piscataway, NJ 08855.
Telephone: (908) 932-0190. Fax: (908) 932-0127.
Abbreviations used: CVMT, Continuous Visual Memory Test; MCS, multiple
chemical sensitivities; MMPI-2, Minnesota Multiphasic Personality
Inventory-2; NART-R, National Adult Reading Test-Revised; NES Battery,
neurobehavioral evaluation system; PEA, phenyl ethyl alcohol; SBS,
sick-building syndrome; UPSIT, University of Pennsylvania Smell
Identification Test; VOC, volatile organic compound; WAIS-R, Wechsler
Adult Intelligence Scale-Revised; WHO, World Health Organization.
Extensive literature documents the range of neuropsychological tests
available to detect and evaluate symptoms of exposure to neurotoxicants.
By 1990 approximately 250 different tests had been used to evaluate the
effects of neurotoxicants on behavior (1). Numerous articles have been
written in which the range of functions assessed and the specific tests
used to assess those functions are reviewed (1-5). Tests applied to
evaluate symptoms among humans derive from two different psychological
approaches: traditional neuropsychological testing to diagnose brain
dysfunction and experimental cognitive psychology. Neuropsychology has
traditionally focused on tests to identify deficits due to pathology
(6), while methods within cognitive psychology have been developed to
elucidate normal cognitive processes involved in processing of
information and learning (7). This paper will briefly review the range
of tests used to assess human responses to neurotoxicants, discuss
measurement of individual susceptibility as a factor in establishing
risk status among patients with sensitivities to low-level exposures,
and consider changes in current methods that will advance measurement of
toxic symptoms at low-level exposures.
Review of Current Tests
Traditional neuropsychological tests such as subtests from the Wechsler
Adult Intelligence Scale-Revised (WAIS-R) (8) and the Halstead-Reitan
(9) proved to be sufficiently sensitive to document cognitive deficits
among workers with chronic exposure to neurotoxicants such as organic
solvents and lead (10,11). Increasingly, concerns have been raised
regarding the sensitivity of these tests for detecting subtle cognitive
deficits at lower level exposures (12). Nevertheless, several batteries,
incorporating many of the traditional neuropsychological tests, are
widely used for clinical and research purposes (13).
While batteries vary in the specific tests used, the cognitive functions
assessed are relatively consistent. Table 1 contains a list of the
functions assessed and a sample of the tests used to assess these
functions. Tests have been somewhat arbitrarily categorized into
functional categories; however, as several other reviewers have
suggested, any one test typically relies on more than one function for
performance (5,6). For example, even relatively simple tasks such as
simple reaction time require not only attention and concentration but
also motor speed for accurate and quick responding. Thus, tests overlap
Tests of overall verbal ability such as vocabulary tests [e.g.,
Vocabulary-WAIS-R (8)], multiple choice vocabulary (16), or reading
scores (e.g., NART-R) (14) are used to estimate premorbid ability. Some
studies of chronic organic solvent or lead exposures have suggested that
such exposure results in a general dementia affecting all aspects of
cognitive function including word knowledge and general information
(25,26). However, most studies cite verbal ability tests as methods that
are relatively insensitive to neurotoxicants (6,13). A consistent
problem in studies of neurotoxicants is the lack of baseline
intellectual function before exposure. Therefore, tests of current
verbal ability are used as surrogates for preexposure ability.
Overall Ability--Spatial Relations
Another broad class of tests of ability are those assessing spatial
relations such as block design from the WAIS-R (8) and Raven's
Progressive Matrices (15). Block design has been used extensively to
evaluate the effects of lead and solvents with mixed results (27,28).
Raven's Progressive Matrices (15) was designed to assess overall
intellectual ability by presenting visuospatial conceptual problems
rather than verbal conceptual problems (e.g., Similarities-WAIS-R) and,
although less widely used, it has been sensitive to the effects of
neurotoxicants (e.g., mercury) (29).
Tests of concentration and attention assess the ability to orient and
sustain attention to either visual or auditory stimuli. This ability is
the precursor to learning and memory, two functions emphasized in all
batteries of neuropsychological tests. Tasks of concentration/attention
range from simple reaction time in response to simple auditory or visual
stimuli to more complex tasks in which the individual must sustain
attention to the target stimulus when distractors are present, such as
the Stroop Color-Word Task (17), or signal detection tasks, such as the
Continuous Performance Test (16). Representative tests of
attention/concentration and vigilance are included in most studies of
neurotoxicants and are a part of test batteries applied to worksite
testing [e.g., FIOH Battery (30), WHO Battery (31), London School of
Hygiene Battery (32)]. Gamberale (2) cites simple reaction time as the
most sensitive test for detecting behavioral performance effects due to
As might be expected, tests of cognitive skills are more plentiful than
tests of motor skills, particularly tests of gross as opposed to fine
motor skills. Tests of motor skills assess speed and dexterity by asking
the subject to place pegs in holes while being timed [e.g., grooved
pegboard (9), Santa Ana Pegboard (33)] or measure strength of grip by
pressure against a spring-loaded device (dynamometer) (9). Finger
tapping (9) is another simple test of coordination and speed that has
been widely used and found sensitive to the effects of neurotoxicants
Another category of tests are those designed to assess visuomotor
skills. At some level, tests of attention also require motor skills
since coordination between the perception of visual stimuli and
initiation of motor movement is necessary for response. However, tests
of visuomotor skills typically involve more complex levels of motor
coordination in response to visual stimuli. For example, the hand-eye
coordination test from the Neurobehavioral Evaluation System-NES battery
(16) tests the ability to move a computer cursor with a joystick along a
sine wave pattern on a screen at a constant rate of speed. Another
somewhat more complex, verbally mediated test of eye-hand coordination,
widely used in this literature, is Digit Symbol (8). This coding task
requires that the subject code symbols with letters while being timed.
It has been sensitive to the effects of neurotoxicants (e.g., lead,
solvents, mercury) (35-37). This test is also included in most of the
test batteries used in worksite testing [e.g., FIOH Battery (30); TUFF
Numerous tests of memory have been used to assess neurotoxicant effects.
These tests range from those assessing memory for abstract visual
designs (e.g., Wechsler Memory Scale-Revised) (18) to tests of memory
for verbal materials such as words or numbers [Paired Associates (18),
Digit Span (8)]. The methodology for these tests usually involves
presentation of the stimulus (e.g., drawing or word list) to be encoded.
The subject is asked to recall the stimulus immediately after
presentation as well as after a relatively short delay (e.g., 30 min). A
more recent development in memory testing incorporates the tradition of
cognitive psychology by not only providing a global indicator of memory
(e.g., total score based on the quantity remembered) but also by scoring
various indicators of memory processes (e.g., slope of learning curve,
proactive and retroactive interference) (19). Assessment of learning
curves and memory processes may provide more insight regarding subtle
effects due to relatively low-level exposures and may help elucidate
this frequent complaint that is often not substantiated by global tests
Sensory tests are not as plentiful and have not received as much
emphasis in the literature on neurotoxicants as many of the tests of
cognition and memory. Tests of audition range from simple tests of
hearing with an audiometer to more complex tests assessing the ability
to discern speech or rhythmic patterns (e.g., seashore rhythm, speech
perception) (9). Tests of tactile perception and vibration sense include
simple tactile perception (finger agnosia) (9) and sense of vibration
using a device (21) to measure perception of fine vibrations in the
finger or toe. These tests have evaluated loss of peripheral sensory
perception due to mercury or solvents. Of more recent interest is the
finding of color vision loss among solvent-exposed workers (39,40).
Finally, altered sense of smell due to neurotoxicants has received more
attention recently (28,41). Tests of olfactory discrimination
[University of Pennsylvania Smell Identification Test (UPSIT)] (22) and
of olfactory threshold detection have been used to evaluate the sense of
Affect and Personality
Numerous questionnaires and standardized tests have been used to assess
mood and personality factors that may be affected by exposure to
neurotoxicants. In fact, mood is reported to be one of the first aspects
of functioning where changes due to neurotoxicants can be observed.
Irritability, depression, and lability are mood changes that are
reported to occur in the first stages of solvent neurotoxicity (26).
Representative scales that have been used include the Profile of Mood
States (23), Minnesota Multiphasic Personality Inventory-2 (24), and
Beck Depression Inventory (42). Unfortunately, these measures rely on
self-report of symptoms rather than any objective indicator of mood.
Therefore, they are subject to reporting biases that may be influenced
by the circumstances in which the individual is being evaluated (e.g.,
While numerous cross-sectional studies have been conducted using various
combinations of neurobehavioral measures to sample each of the domains
listed above, results from these studies have been mixed. Some
investigators have attempted to identify patterns of test performance
specific to classes of neurotoxicants (43,44); however, it has been
difficult to determine a consistent pattern of performance. Numerous
reviews have appeared in which the tests and the exposures evaluated are
listed (5,45). Some general impressions can be formed from these reviews
suggesting that behavioral effects are observed for a number of
neurotoxicants. It is difficult, however, to determine from these more
qualitative reviews which tests are most sensitive in detecting effects.
Further, little information is available to document the predictive
validity of these tests for performance in the workplace. Therefore,
even if we can say that a test detects differences in performance
between exposed and nonexposed groups, the meaning of the performance
difference has not been adequately addressed. Information about this
issue could provide the most compelling evidence for controlling or
At this point, there may be sufficient literature on some organic
solvents and heavy metals such as lead to conduct metaanalyses of the
results across studies. These statistical methods have been used in
other literatures to help consolidate disparate findings into a more
cohesive picture. These methods could help clarify which tests are most
sensitive for detecting effects due to specific neurotoxicants (46).
For the field of neurobehavioral toxicology to make meaningful advances
in our understanding of the behavioral effects of neurotoxicants, more
refined studies will be needed. Such studies will also require that
neurobehavioral methods be improved. For example, rather than continue
cross-sectional studies, prospective studies need to be developed in
which workers are followed over a period of time to assess changes from
baseline. These will require a better understanding of the behavior of
neuropsychological tests under repeated measures conditions. Otto et al.
(4) found significant practice effects for several of the tests on the
NES battery. To avoid ceiling effects on these tests after repeated
administration, he suggested that test parameters be altered to make the
tasks more difficult and better suited to repeated measures design (4).
Similarly, increasing demands are being made for neuropsychological
methods to assess subtle effects in acute and unusually low-exposure
circumstances. These conditions also require an increased sensitivity in
neuropsychological test methods.
In summary, we need to take a more systematic approach toward
identifying the most sensitive tests among those cited frequently in the
literature. We then need to test the suitability of these tests for the
study designs proposed to address present concerns such as low-level
exposures. Further consideration of the parameters to be considered in
these studies will be addressed in the subsequent sections of this
The test literature reviewed previously provides an overview of the
broad range of tests and functional categories that have been included
in the literature on the effects of neurotoxicants. While some tests
appear to be more sensitive than others, it is difficult to develop a
clear picture due to the large variability in the demographic profiles
of the subject groups evaluated, the range of different substances to
which these groups were exposed, and the lack of clarity regarding
duration and intensity of exposures. Despite these factors, several
attempts have been made to develop batteries of tests that can be
coordinated across studies [e.g., WHO battery (31), NES battery (16)],
thus allowing direct comparisons between studies. This effort to
increase comparability is to be applauded. However, in an effort to be
broadly applicable, these batteries may prove insensitive for unique
populations or exposure situations.
For example, an increasing number of patients have vague complaints,
including poor concentration and memory, in response to low-level
chemical exposures. This symptom complex, labeled multiple chemical
sensitivities (MCS), involves symptoms reflective of multiple organ
systems, most prominently the nervous system. The question of whether
these patients are uniquely susceptible to chemicals or are a variant of
the psychiatric disorder, somatization, is frequently debated (47-49).
MCS patients may present unique susceptibilities to chemicals for
several reasons. First, while no epidemiological studies have been
conducted to date, most of the investigators observe that approximately
80% of these patients are women (49,50). This is in contrast to the
literature on the neuropsychological effects of neurotoxicants, which is
based largely on men. In one of the few studies of women, Parkinson et
al. (51) reported no significant differences between solvent-exposed
blue-collar women and controls on a relatively brief battery of standard
neuropsychological tests. However, the highest exposure levels were
significantly related to a number of neurologic and somatic symptoms
including depression and headaches. When symptoms are reported by women,
they are more likely to be attributed to psychosomatic causes such as
stress rather than to physiologically based conditions (52). This is
particularly true when objective tests do not substantiate symptom
reports. However, it is also possible that women may have unique
susceptibilities that wax and wane due to hormonal cycles not occurring
in men. For example, women can vary in olfactory acuity according to
hormonal cycles (53). Alternatively, women may simply be more aware of
and likely to report symptoms that occur in response to an exposure or
an illness than men. That is, women may be better observers of the early
signs of physiologic changes (54). The challenge is to develop
methodologies to measure these changes.
Second, MCS patients have a higher rate of psychiatric disorder (e.g.,
depression, anxiety) concurrent with and before the onset of MCS
(49,50). Many use this information to suggest that MCS is not a unique
susceptibility but simply a psychiatric condition that is attributed to
chemicals. On the other hand, one may question whether individuals with
psychiatric conditions are more susceptible to the effects of
neurotoxicants. For example, Morrow et al. (55) reported that
individuals with higher levels of psychological distress on the MMPI-2
were associated with poorer neuropsychological function at follow-up.
From this study it is impossible to know whether the symptoms on the
MMPI-2 were a reflection of continuing neurologic symptoms due to
exposure or a premorbid personality style. Psychiatric and personality
function, as a risk factor for the effects of neurotoxicants, has
infrequently been evaluated and needs further exploration.
Only two studies to date have used standardized neuropsychological tests
to evaluate the cognitive complaints of MCS patients (50,56). Overall,
these cross-sectional studies did not find differences between the MCS
and control groups (i.e., musculoskeletal patients, normal controls) on
tests of concentration, memory, and visuomotor skills. However, these
tests were not administered under controlled exposure conditions. A
primary question is how to test the responses of MCS patients
objectively. The typical evaluation paradigm in which the patient's
neuropsychological performance and physical status is assessed at an
arbitrary point in time is not likely to capture the symptomatic
response that these patients observe in themselves under exposure
Studies more directly relevant to investigation of responses among MCS
patients are exposure chamber studies with sick-building syndrome (SBS)
patients (57,58). These patients are similar to MCS patients in that
they are otherwise healthy individuals who report sensitivities in
response to indoor air mixtures that other individuals apparently
tolerate. Two controlled exposure studies evaluated the effects of a
mixture of 22 volatile organic compounds (VOCs) on sick-building
syndrome patients relative to asymptomatic controls (57,58). Along with
increasing symptom reports of irritation with increasing VOC exposure
(0, 5, 25 mg/m3), Molhave et al. (57) reported reduced performance on
digit span among SBS subjects. This finding was not replicated, however,
when this study was conducted with young, healthy male subjects (4).
Kjaergaard et al. (58) also found impaired digit span performance in SBS-sensitive
subjects but not among non-SBS subjects with exposure at 25 mg/m3 VOC
mixture, which is roughly equivalent to 7 ppm toluene. Otto et al. (4)
suggested that differential effects may be due to differential
sensitivity of the subject groups as well as relative insensitivity of
many of the current neurobehavioral methods.
To test the unique susceptibilities of MCS patients, several factors
must be taken into account. First, like SBS patients, MCS patients
report responses at exposure levels that most individuals tolerate. For
example, in our current protocol we conduct olfactory threshold testing
in response to phenyl ethyl alcohol (PEA), a pleasant olfactory
stimulant. MCS patients reported significantly more symptoms than normal
controls during threshold testing. At suprathreshold levels they
reported PEA to be significantly more unsafe and unpleasant than did
normal controls. From our estimations, the concentrations at the average
olfactory threshold are comparable to 7 ppm, a level well below that
expected to produce neurobehavioral performance decrements.
Second, an overriding concern is that symptomatic responses of MCS
patients are conditioned responses to olfactory cues (59). Even among
healthy individuals, odor has been shown to impact performance (60-62).
Neither the studies on SBS subjects nor controlled exposure studies have
adequately accounted for the impact of odor on performance.
Ideally, controlled exposures with MCS patients will need to occur below
olfactory thresholds to control for psychological expectations due to
odor. Detecting effects at such low levels of exposures (as low as 1 ppm)
will require highly sensitive behavioral performance measures. Measures
such as reaction time and vigilance tasks have been the most sensitive
indicators in previous cross-sectional and chamber studies (2).
Therefore, use of measures of attention and vigilance similar to those
cited in the signal-detection literature may offer the best alternative
to detect effects among susceptible individuals and low-level exposure
Unlike many previous exposure-chamber studies of neurotoxicants,
exposing symptomatic groups such as MCS patients requires some shifts in
the methodologies previously employed. First, the questions to be
answered depart from the traditional concern for developing workplace
standards. These studies sought to establish the upper limit of exposure
before objective behavioral effects were detected. While not necessarily
applicable even to all healthy individuals working with neurotoxicants,
exposure to these levels among hypersensitive groups cannot be
undertaken for obvious health and ethical concerns.
As has been suggested by other investigators, one method for detecting
effects at lower exposure levels is to vary parameters within the
performance test to increase its sensitivity to effects (4,63).
Documentation of the effects of varied test parameters has been the
subject of much attention within the experimental literature (e.g.,
signal-detection paradigms) and virtually no systematic attention within
the neuropsychological literature or in the literature investigating the
behavioral effects of neurotoxicants. For example, in the
signal-detection literature, Jansen et al. (64) found that when signal
probability was low, alcohol affected stimulus sensitivity and reaction
time of hits, but the same dose of alcohol did not affect these
parameters when signal probability was high. These findings were not
replicated for Diazepam (65). The findings with alcohol were interpreted
to suggest that reduced response accuracy to low probability signals
would compromise driving performance since low and variable signals are
likely. If only one stimulus intensity was used, this differential
effect of alcohol would not have been detected.
Detection of effects under exposure conditions will also require that
behavioral tests be repeated within a relatively short period of time.
Therefore, more information is needed to document the effects of
repeated test administration within a brief time period such as before,
during, and after exposure. This will require that tests be of
sufficient difficulty to allow variability in performance both within
and between subjects.
Behavioral tests that focus on process rather than a single summary
outcome will be important in the development of research on low-level
exposures. Even in cross-sectional studies of exposed working
populations, the detection of behavioral effects to objectify
symptomatic complaints of poor memory and concentration has been
problematic. This difficulty may be due to the fact that many of the
neuropsychological tests applied to this field offer a summary score of
performance (WAIS-R subtests) rather than assessment of learning curves
or variables delineating the various functions that contribute to
performance. Thus, several investigators have emphasized the application
of information-processing paradigms and tasks to the assessment of
In our experience with MCS patients, the tasks most sensitive to
behavioral performance decrements were those in which subfunctions of
the task were assessed. For example, obtaining scores on
signal-detection parameters for the Continuous Visual Memory Test (CVMT)
(67) revealed that MCS patients recognized signals at the same rate as
normals (hits) but over responded to nonsignals (false alarms). A
summary score for this task would suggest impaired visual memory;
however, analysis of the subfunctions suggests that response style may
be a more important variable in their performance. Observation of the
distribution of scores for this group of patients also suggests that
only a subgroup of the total group (approximately 39%) exhibited
significant impairment (Figure 1). This finding is consistent with the
observation of hyperactive children of Weiss et al. (68). Inspection of
individual performance was more important than looking at overall group
means, which can mask a subgroup of hyperresponsive individuals. This is
particularly important when case definitions for affected individuals
such as MCS are not clear.
In addition to test parameters, a complete characterization of host
factors such as psychiatric diagnoses and personality traits is
important. For example, among the following variables in the
MMPI-2--age, reading score, and depression--health concerns was the
variable accounting for the highest percentage of variance in
performance on the CVMT (Figure 2). Health concerns measures a range of
somatic symptoms, some of which can be related to neurologic conditions
and some to somatization (69). Previous exposure-chamber studies have
not focused on individual difference variables, such as mood or the
tendency to somatize, in assessing behavioral response to neurotoxicants.
Documentation of these variables may be critical in understanding
individual differences in performance among MCS patients, particularly
since approximately 25% of MCS patients qualify for a diagnosis of
depression (50,56). Little is known about how depression may interact
with the effects of neurotoxicants in producing behavioral performance
Finally, as mentioned above, various odorants may affect behavioral
performance (62). In olfactory research, extensive literature documents
the psychophysical properties and mechanisms of odor perception.
However, this literature does not address the concentration at which
symptoms and objective health effects occur. Studies use objective
behavioral tests (e.g., digit span) to document the effects of an
odorant but relate these effects to properties of the odor (e.g.,
pleasant vs unpleasant, irritating vs nonirritating) rather than to
concentrations in toxicological terms (60). While these odor effects may
not impact healthy individuals, the same cannot be presumed in studies
of symptomatic individuals such as MCS patients. Therefore, control or
measurement of the impact of odor is critical. For example, alternate
methods for administering exposures such as dermal routes could be
Documenting behavioral responses to neurotoxicants among highly
susceptible individuals places greater demands on the sensitivity of
neuropsychological methods. Developing sensitive methods to elucidate
the responses of sensitive individuals will also improve our approaches
in the entire field of neurobehavioral toxicology.
1. Anger WK. Worksite behavioral research: results, sensitive methods,
test batteries and the transition from laboratory data to human health.
Neurotoxicology 11:629-670 (1990).
2. Gamberale F. Use of behavioral performance tests in the assessment of
solvent toxicity. Scan J Work Environ Health 11:65-74 (1985).
3. Russell R. Neurobehavioral time bombs: their nature and their
mechanisms. In: Behavioral Measures of Neurotoxicity (Russell R, Flattau
P, Pope AM, eds). Washington:National Academy Press, 1990;206-225.
4. Otto D, Hudnell HK, House D, Molhave L, Counts W. Exposure of humans
to a volatile organic mixture. Behav Assess 47(1):23-30 (1992).
5. Anger WK, Johnson BL. Human behavioral neurotoxicology workplace and
community assessments. In: Environmental and Occupational Medicine (Rom
WN, ed). Boston:Little, Brown, 1992;573-592.
6. Lezak MD. Neuropsychological Assessment. New York:Oxford University
7. Gullion CM, Eckerman DA. Field testing. In: Neurobehavioral
Toxicology (Annau Z, ed). Baltimore:Johns Hopkins University Press,
8. Wechsler D. Wechsler Adult Intelligence Scale-Revised. San Antonio,
TX:The Psychological Corporation, 1981.
9. Heaton RK, Grant I, Matthews CG. Comprehensive Norms for an Expanded
Halstead-Reitan Battery: Demographic Corrections, Research Findings, and
Clinical Applications. Odessa, FL:Psychological Assessment Resources,
10. Hanninen H. Psychological test methods: sensitivity to long term
chemical exposure at work. Neurobehav Toxicol 1:157-161 (1979).
11. Needleman HL. Lessons from the history of childhood plumbism for a
pediatric neurotoxicology. In: Advances in Neurobehavioral Toxicology:
Applications in Environmental and Occupational Health (Johnson BL, Anger
WK, Durao A, Xintaras C, eds). Chelsea, MI:Lewis Publishers,
12. Hernberg S. Neurotoxic effects of long-term exposure to organic
hydrocarbon solvents. Epidemiologic aspects. In: Mechanisms of Toxicity
and Hazard Evaluation (Holmstedt B, Lauwerys R, Mercier M, Roberfroid M,
eds). Amsterdam:Elsevier/North Holland Biomedical Press, 1980;307-317.
13. Hartman DE. Neuropsychological Toxicology: Identification and
Assessment of Human Neurotoxic Syndromes. New York:Pergamon Press, 1988.
14. Nelson HE. National Adult Reading Test (NART): Test Manual. Windsor,
15. Raven JD. Guide to Standard Progressive Matrices. London:H.K. Lewis,
16. Letz R, Baker EL. NES2. Neurobehavioral Evaluation Systems. June,
17. Trenerry MR, Crosson B, DeBoe J, Leber WR. Stroop Neuropsychological
Screening Test Manual. Odessa, FL:Psychological Assessment Resources,
18. Wechsler, D. Wechsler Memory Scale-Revised Manual. San Antonio,
TX:The Psychological Corporation, 1987.
19. Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning
Test. San Antonio, TX:The Psychological Corporation, 1987.
20. Visser RSH. Manual of the Complex Figure Test CFT: A Test Involving
Restructuring Behaviour Developed for the Assessment of Brain Damage.
Amsterdam:Sweta & Zeitlinger BV, 1973.
21. Vibratron II Operating Manual. Clifton, NJ:Physitemp Instruments,
22. Doty RL. The Smell Identification Test. Haddon Heights, NJ:Sensonics,
23. McNair DM, Lorr M, Droppleman LF. Profile of Mood States. San
Diego:Educational and Industrial Testing Service, 1981.
24. Hathaway SF, McKinley JC. Minnesota Multiphasic Personality
Inventory 2. Minneapolis, MN:University of Minnesota Press, 1989.
25. Oberg RGE, Udesen H, Thomsen AM, Gade A, Mortensen EL. Psychogenic
behavioral impairments in patients exposed to neurotoxins.
Neuropsychological assessment in differential diagnosis. In:
Neurobehavioural Methods in Occupational and Environmental Health.
Environmental Health Documents 3. Copenhagen:World Health Organization,
26. NIOSH. Organic Solvent Neurotoxicity. Cincinnati, OH:National
Institute for Occupational Safety and Health, 1987.
27. Bleecker ML, Agnew J, Keogh JP, Stetson DS. Neurobehavioural
evaluation in workers following a brief exposure to lead. In: Advances
in the Biosciences, Neurobehavioral Methods in Occupational Health. Vol
46 (Gilioli R, Cassito MG, Foa V, eds). New York:Pergamon Press,
28. Ryan CM, Morrow LA, Hodgson M. Cacosmia and neurobehavioral
dysfunction associated with occupational exposure to mixtures of organic
solvents. Am J Psychiatry 145:1442-1445 (1988).
29. Istoc-Bobis M, Gabor S. Psychological dysfunctions in lead- and
mercury-occupational exposure. Revue Roumaine des Sciences
Sociales--Serie de Psychologie 31:183-191 (1987).
30. Hanninen H, Lindstrom K. Neurobehavioral Test Battery of the
Institute of Occupational Health. Revised ed. Helsinki:Institute of
Occupational Health, 1989.
31. Johnson BL, Baker EL, El Batawi M, Gilioli R, Hanninen H,
Seppalainen AM, Xintaras C, eds. Prevention of Neurotoxic Illness in
Working Populations. New York:John Wiley and Sons, 1987.
32. Cherry N, Venables H, Waldron HA. Description of the tests in the
London School of Hygiene test battery. Scan J Work Environ Health
10(Suppl 1):18-19 (1984).
33. Fleishman EA. Dimensional analysis of psychomotor abilities. J Exp
Psychol 48:437-454 (1954).
34. Grandjean P, Beckmann J, Ditlev G. Relation between subjective
symptoms and psychometric tests results. In: Advances in the
Biosciences, Neurobehavioral Methods in Occupational Health. Vol 46 (Gilioli
R, Cassitto MG, Foa V, eds). New York:Pergamon Press, 1983;301-308.
35. Valciukas JA, Lilis R, Eisinger J, Blumberg WE, Fischbein A,
Selikoff IJ. Behavioral indicators of lead neurotoxicity: results of a
clinical field survey. Int Arch Occup Environ Health 41:217-236 (1978).
36. Angotzi G, Camerino D, Carboncini F, Cassitto MG, Ceccarelli F,
Cioni R, Pradiso C, Sartorelli E. Neurobehavioral follow-up study of
mercury exposure. In: Advances in the Biosciences, Neurobehavioral
Methods in Occupational Health. Vol 46 (Gilioli R, Cassitto MG, Foa V,
eds). New York:Pergamon Press, 1983;247-253.
37. Orbaek P, Lindgren M, Olivecrona H, Haeger-Aronsen B. Computed
tomography and psychometric test performance in patients with solvent
induced chronic toxic encephalopathy and healthy controls. Br J Ind Med
38. Hogstedt C, Hane M, Axelson O. Diagnostic and health care aspects of
workers exposed to solvents. In: Developments in Occupational Medicine (Zenz
C, ed). Chicago:Year Book Medical Publishers, 1980;249-258.
39. Mergler D, Blain L. Assessing color vision loss among
solvent-exposed workers. Am J Ind Med 12:195-203 (1987).
40. Braun CMJ, Daigneault S, Gilbert B. Color discrimination testing
reveals early printshop solvent neurotoxicity better than a
neuropsychological test battery. Arch Clin Neuropsychol 4:1-13 (1989).
41. Schwartz BS, Doty RL, Monroe C, Frye R, Barker S. Olfactory function
in chemical workers exposed to acrylate and methacrylate vapors. Am J
Public Health 79:613-618 (1989).
42. Beck AT. Beck Depression Inventory. San Antonio, TX:The
Psychological Corporation, 1978.
43. Williamson AM. The development of a neurobehavioral test battery for
use in hazard evaluations on occupational settings. Neurotoxicol Teratol
44. Hanninen H. The psychological performance profile on occupational
intoxications. Neurotoxicol Teratol 10:485-488 (1988).
45. White RF, Feldman RG, Travers PH. Review: neurobehavioral effects of
toxicity due to metals, solvents, and insecticides. Clin Neuropharmacol
46. Glass GV, McGraw B, Smith ML. Meta-analysis of Social Research.
Beverly Hills, CA:Sage Publications, 1981.
47. Brodsky CM. Psychological factors contributing to somatoform
diseases attributed to the workplace. J Occup Med 25:459-464 (1983).
48. Stewart DE, Raskin J. Psychiatric assessment of patients with "20th
century disease" ("total allergy syndrome"). Can Med Assoc J
49. Simon GE, Katon W, Sparks PJ. Allergic to life: psychological
factors in environmental illness. Am J Psychiatry 147:901-906 (1990).
50. Fiedler N, Kipen HM, DeLuca J, Kelly-McNeil K, Natelson B. A
controlled comparison of multiple chemical sensitivity and chronic
fatigue syndrome. Psychosom Med (in press).
51. Parkinson DK, Bromet EJ, Cohen S, Dunn LO, Dew MA, Ryan C, Schwartz
JE. Health effects of long-term solvent exposure among women in
blue-collar occupations. Am J Ind Med 17:661-675 (1990).
52. Heston TF, Lewis LM. Gender bias in the evaluation and management of
acute nontraumatic chest pain. The St. Louis Emergency Physician's
Association Research Group. Fam Pract Res J 12:383-389 (1992).
53. Filsinger EE, Monte WC. Brief report: commentaries, remarks, and
notes pertaining to sex research. J Sex Res 22:243-248 (1986).
54. Leventhal E. Gender and aging: women and their aging. In:
Psychological Perspectives on Women's Health (Reddy DM, Adesso VJ,
Flemming, eds). New York:Hemisphere Publishing, in press.
55. Morrow LA, Ryan CM, Hodgson MJ, Robin N. Risk factors associated
with persistence of neuropsychological deficits in persons with organic
solvent exposure. J Nerv Ment Dis 179:540-545 (1991).
56. Simon GE, Daniell W, Stockbridge H, Claypoole K, Rosenstock L.
Immunologic, psychological, and neuropsychological factors in multiple
chemical sensitivity: a controlled study. Ann Intern Med 19:97-103
57. Molhave L, Back B, Pedersen OF. Human reactions to low
concentrations of volatile organic compounds. Environ Int 12:167-175
58. Kjaergaard SK, Molhave L, Pedersen OF. Human reactions to a mixture
of indoor air volatile organic compounds. Atmos Environ 25:1417-1426
59. Bolla-Wilson K, Wilson RJ, Bleecker ML. Conditioning of physical
symptoms after neurotoxic exposure. J Occup Med 30:684-686 (1988).
60. Knasko SC. Performance, mood, and health during exposure to
intermittent odors. Arch Environ Health 48(5):305-308 (1993).
61. Shusterman D. Critical review: the health significance of
environmental odor pollution. Arch Environ Health 47(1):76-87 (1992).
62. Rotton J. Affective and cognitive consequences of malodorous
pollution. Basic Appl Soc Psychol 4:171-191 (1983).
63. Teichner WH. Carbon monoxide and human performance: a methodological
exploration. In: Behavioral Toxicology (Weiss B, Laties VC, eds). New
York:Plenum Press, 1975;77-104.
64. Jansen AAI, de Gier JJ, Slangen JL. Alcohol effects on signal
detection performance. Neuropsychobiology 14:83-87 (1985).
65. Jansen AAI, de Gier JJ, Slangen JL. Diazepam-induced changes in
signal detection performance: a comparison with the effects on the
critical flicker-fusion frequency and the digit symbol substitution
test. Neuropsychobiology 16:193-197 (1986).
66. Stollery BT, Banks HA, Broadbent DE, Lee RW. Cognitive functioning
in lead workers. Br J Ind Med 46:698-707 (1989).
67. Trahan DE, Larrabee GJ. Continuous Visual Memory Test. Odessa,
FL:Psychological Assessment Resources, 1988.
68. Weiss B, William JH, Margen S, Abrams B, Citron LJ, Cox C, McKibben
J, Ogar D, Schultz S. Behavioral responses to artificial food colors.
Science 207(4438):1487-1489 (1980).
69. Alfano DP, Paniak CE, Finlayson ALJ. The MMPI and closed head
injury: a neurocorrective approach. Neuropsychiatry: Neuropsychol Behav
Neurol. 6:111-116 (1993).