Using a ‘brief orientation adaptation paradigm’, we investigated how orientation selective adaptation was modulated by responses and performance of subjects in an orientation discrimination task during the adaptation period. Subjects performed a delayed match-to-orientation task in the peripheral visual field. The test stimulus was preceded by a briefly presented adapting target stimulus embedded among adjacent distractors (‘crowding condition’). Subjects reported the orientation of adapting target stimulus before reporting whether sample and test stimuli differed in orientation. In the condition of ‘crowding at threshold’ (75% correct), adaptation was on the basis of responses in the adaptation phase rather than which orientation was physically presented; however, there was a robust stimulus-based adaptation when we analyzed trials where subjects discriminated the orientation of adapting target stimulus correctly. In the condition of ‘crowding at chance-level’ (50% correct), we found opposite results (adaptation was stimulus-based specifically in trials where subjects discriminated the orientation of adapting target stimulus inaccurately). We suggested primary visual cortex (V1) as a plausible candidate for interactions between subjects' perceptual judgments and orientation-selective adaptation.