The current study introduces a new direct evaluation method for discriminating the strengths and weaknesses of educational materials. Six areas or "standards" of good materials were identified in the literature and used to create a 33-item survey. The measure scores a variety of educational materials from textbooks to multimedia web-based tools on a scale of 1-25 in each of the six standards. Two experiments were conducted to determine the reliability and validity of the new measure. In experiment 1, a group of expert judges completed the evaluation for three different materials: a textbook, an e-book and a viziswap module. The scores were not reliable between judges but they were consistent with previous research indicating increased student exam scores and student attitude surveys. In experiment 2, a group of undergraduates completed the survey twice after a two-week delay. The results of experiment 2 indicated that the survey does not have test-retest reliability. The current measure needs further research to produce a reliable measure but is useful in identifying the strengths and weaknesses of educational materials.