The Daubert Test and Multi-Method Psychological Testing

(Prep. for Testimony, TOH to Robert Erard, Ph.D.)

In Daubert, the Court listed three criteria, in addition to the Frye standard of “general acceptance”:

In Daubert general acceptance” is described as the “explicit identification of a relevant scientific community and an express determination of a particular degree of acceptance within that community”

From this perspective, four guidelines were promulgated in Daubert (pp. 2796-2997) for the admissibility of scientific evidence:

1, Testability or falsifiability of the opinion or test’s basis: 

Is the proposed theory (or technique), on which the testimony is to be based, testable?

-The testability criterion is based on philosopher Karl Popper’s observation that scientific theories and methods must be falsifiable, i.e., testable in principle

Formal psychological tests are based on empirical validation. More specifically, interpretations of tests should be considered under Daubert because of their nomothetic basis, grounded in reliability, validity, and generalizability.

Yes: All tests used by me are evidence-based; construct, convergent and discriminant validity (p.4) have been measured, as well as reliability, temporal stability, external correlates, positive predictive power have been measured and subjected to peer review and meta-analysis

Meyer et al. (2001) published an article that summarizes evidence and issues associated with psychological assessment. Data from more than 125 meta-analyses on test validity and 800 samples examining multi- method assessment suggested four general conclusions:

(a) psychological test validity is strong and compelling,

(b) psychological test validity is comparable to medical test validity,

(c) distinct assessment methods provide unique sources of information, and

(d) clinicians who rely exclusively on interviews are prone to incomplete understanding.

Multi-method assessment” is a strategy increasingly used by clinicians all around the world. Gacono et al. (2001) write: “No single variable, score, or ratio is interpreted in isolation from history, response style, and other data.

2. Peer review and publication:  (Have the basis for the opinions been subject to peer review or publication?)

– Not that the validity of every possible interpretation be established by peer review, but merely that the method yields testable interpretations that must be peer-reviewedYes: the diagnostic opinions provided by me are based on procedures/methods that have been peer reviewed and published.               

3. Considering the known or potential rate of error :

As explained in Daubert’s Footnote 9, evidentiary reliability is construed as trustworthiness of the data; for scientific evidence, “evidentiary reliability will be based on scientific validity”

Instrument or procedures must be derived from recognized scientific knowledge: the theory and data should be recognized, time-tested, falsifiable, and committed to refinement.  Court can look at facts, techniques, methodology, reasoning.Yes:  All test procedures used by me are trustworthy, based on established and reliable scientific evidence (meta-analysis)which has established a rate of error. (see below on error types and ) with a known positive predictive power: the percentage of true positives (among those identified by test as positive) among those identified/predicted by the procedure as a positive test finding. (A/(A+B)).                 

Appraising the general acceptance within the relevant scientific community.-this is not whether accepted by everyone in psychology, but rather whether opinion/theory/technique is accepted by experts in the area of forensic psychology and personality assessment (Erard and Viglione argue that this criterion has been met)

  • Yes:  All test procedures used by me are generally accepted within the community of forensic and assessment psychologists, many of whom are also academic psychologists.
  • The MMPI-2  is a revision of the original MMPI (Hathaway & McKinley, 1940) for use with adults. New scales and cut-off scores, among other changes, were included in an updated manual
  • In 2008, the MMPI-2 Restructured Form (MMPI-2-RF) was released (Ben-Porath & Tellegen, 2008/2011) as a revision of the MMPI-2 and based on the same test items. The MMPI-2-RF uses 60% of MMPI-2 item pool and its 1989 normative data.

The MMPI-2 and MMPI-2 RF combined were administered 500-600 K times per year from 2007-2014.

Ben-Porath argued that the MMPI-2-RF is a restructured MMPI-2, not a new test, because it contains only MMPI-2 items and its normative data set.