This research studies the impact that characteristics of textiles within the built environment have on tactile defensive behaviors for individuals with sensory sensitivities, and the effect of perceived temperature on these behaviors. This study investigates the tactile sensory response to six different textiles. Each textile contains various fibrous components, textures, and construction mechanisms to identify characteristics that are likely to trigger behaviors of tactile defensiveness. This data was collected through a three-part survey, concluding that textiles containing natural fibers such as cotton and silk yielded the best results overall. Textiles containing synthetic fibers such as polyester, acrylic, and rayon were perceived as least pleasant, particularly in warm temperatures. There was no significance found in perceived tactile defensiveness frequency between individuals with sensory sensitivities and individuals without. Significance in the change of tactile defensiveness frequency under “warm” and “cold” temperatures revealed that small fibers or woven components were favorable characteristics of textiles in cold temperatures, providing stimulating textured components to compensate for reduced hand-feel receptor function in this condition. Individuals without sensory sensitivities showed more variability in perceived frequency of tactile defensiveness behavior, while individuals with sensory sensitivities had more regular responses, even within variations of temperature perception. Comfortable temperature conditions yielded the lowest frequency of perceived tactile defensiveness, while warm temperature conditions resulted in the highest levels of perceived tactile defensiveness, and cold temperatures following close behind. These results conclude that small synthetic fibers are more prone to eliciting negative tactile defensive responses in warm, humid climates, while natural fibers and non-distinct fibers are least likely to produce a tactile defensiveness response for cold temperature conditions. This research defines physical characteristics of textiles that are associated with tactile defensiveness behavior, furthering the knowledge of the neurological impact that textiles within the built environment have on individuals with sensory sensitivities.