Establishing construct validity for a department perception inventory : utilizing exploratory factor analysis with small samples

Abstract

Exploratory factor analysis (EFA) is often used in the scale development process to determine the existence of latent factors and establish construct validity. In this study, EFA was performed on data from a department perception inventory deployed prior to and after a team retreat. Since EFA is known to require large sample sizes, this study examined whether factors could be extracted in situations with smaller samples, and whether the results from the factor loadings could be replicated. Responses to the inventory prior to the team retreat (n = 87) and after (n = 83) were obtained from marketing and communication professionals from a healthcare company in the mountain west. Parallel analysis and minimum average partials supported a twofactor solution while a scree test supported a three-factor solution. Principal axis factoring with promax rotation was used as the factor retention method for both two- and three-factor solutions. Due to the availability of pre- and post-event data, external replicability analysis was performed which led to the decision to retain three factors. However, replication analysis indicated that neither factor solution was perfect. Notwithstanding, the three-factor solution made the most theoretical sense, highlighting the importance of theory in factor retention and construct validity. The results reaffirmed an existing framework in EFA where replication analysis helped identify problematic variables to be retooled within the scale development process.