We propose trust to enhance security in adaptive and non-intrusive user authentication in controlled and pervasive environments. In addition to who a user is (e.g., via biometrics) and what a user knows (e.g., a password, a PIN), recent authentication solutions evaluate what a user has. The user's identity is then derived from what detectable accredited items (e.g., badges, RFIDs) and personal devices (e.g., smart-phones, PDAs) the user shows when authenticating. The level of security of the access is set consequently. Position information is also considered in authentication; only those users carrying authorised items in proximity of certain places can benefit from available resources at those places. Unfortunately, items such as badges, mobile phones, smart phones, RFID-ed cards can be stolen, forgotten, or lost with a consequent risk of identity theft and intrusion. In controlled environment like buildings, where sensors can detect a wide range of different types of items, the security of authentication can be improved by evaluating the amount of trust that can be reposed on the user standing in the area from where he tries to access a resource. This piece of information can be calculated from the positions of all the items linkable to the requester as sensed along time by the different sensors available. Sensors are seen as recommenders that give opinions on a user being in a requested position depending on what they have perceived in the environment. We apply Subjective Logics to model recommendations that originate from different types of location detectors and to combine them into a trust value. Our solution has been tested to improve authentication in an intelligent coffee corner of our research institute. A user at the coffee corner can see, displayed on a wall screen, the position of his colleagues depending on the level of authentication he obtains. The user authentication level depends on the number and on the quality of tokens he provides when authenticating. We comment how the use of a location-based trust (on the requester standing at the coffee corner) improves the adaptability, the non-intrusiveness, and the security of the authentication process. We validate our proposal with a simulation that shows how location-based trust changes when a user device moves away from the coffee corner.