Using decision cut points for the recovery of low probability events

Thumbnail Image
Smith, Kendall A. (Student at Ball State University)
Bolin, Jocelyn H.
Issue Date
Thesis (M.S.)
Department of Educational Psychology
Other Identifiers
CardCat URL

This study attempts to determine if the manipulation of the decision cut point impacts the recovery of low probability events. Manipulating decision cut points involves moving the default cut point value in order to maximize pre-determined metrics such as overall model accuracy, model sensitivity, or model specificity. Holistically, little work has been done using simulation methodologies to test the validity of adjusting decision cut point locations under different conditions. This study hopes to add to the literature by manipulating decision cut points with traditional methods such as Logistic Regression (LR) and newer, more complex classifiers like Classification and Regression Trees (CART) and Random Forest (RF). The manipulation of decision cut points potentially offers researchers and practitioners a statistical method for increasing the recovery of low probability events that can be implemented easily in software packages such as SPSS. Results indicate that the use of decision cut points is still case specific; however it appears that for the recovery of low probability events it is best to set the cut point low (0.1- 0.3). Researchers can utilize tools such as Receiver Operator Characteristics curves (ROC Curves) to optimize this statistical method for their problem.