In this paper, we explore whether perceptual adjustments for gender are equally strong for Japanese- and English-speaking listeners' categorization of the sibilant fricatives /s/ and /ʃ/ in CV sequences. These stimuli were created by combining a set of eight fricatives with a set of natural vocalic bases produced by a variety of men. We hypothesized that Japanese listeners' categorization would be more-strongly influenced by gender typicality, given the overall heightened attention to gendered speech features in Japanese speakers (Van Bezooijen, 1995), and the greater role that vocalic features play in fricative categorization in Japanese compared to English (Li, Edwards, & Beckman, 2009; Li, Munson, Yoneyama, Edwards, & Hall, 2011). Some evidence is found that Japanese listeners' categorization of fricatives is influenced more heavily on the gender typicality of men's voices in the vocalic portion of the stimulus than is English listeners, but the effects are neither consistent, nor in the direction predicted by previous research. Results point to the need for more research on how talker attributes affect the way that L2 listeners perceive L1 speech.