Tissue-resident memory T cells are a highly abundant, non-blood circulating subset of memory T cells. These appear to be the most protective population of memory T cells at barrier surfaces. Long-term retention and survival of tissue-resident memory CD8⁺ T cells (Trm) is determined by tissue-derived signals, such as keratinocyte-mediated activation of transforming growth factor β (TGFβ) in the epidermis. We found that T cell clones compete for limited amounts of active TGFβ and pre-existing Trm could be replaced with newly recruited effector T cells in the epidermis. On the other hand, when effector T cells transition into Trm, the presence of cutaneous cognate antigen increases the fitness of individual Trm clones in the epidermal niche. Thus, antigen-specific Trm are more efficiently retained than bystander Trm that have not encountered cognate antigens when they compete with newly recruited effector T cells for limited active TGFβ. Therefore, competition between T cells for active TGFβ represents a selective pressure that promotes the accumulation of antigen-specific Trm cells in the epidermal niche. Furthermore, our model implies that the epidermis offers a finite niche for maintaining Trm. Although the epidermal niche of Trm cannot represent the capacity of T cell-mediated immune memory in our body, these findings might suggest a challenge for the accommodation of memory T cells specific to multiple pathogens throughout a lifetime.