Virtual and augmented reality technologies offer opportunities to create training and rehabilitation tools in virtual or augmented environments, giving users a first-person experience. Designing such systems requires defining the most critical components to be simulated, and to what degree they can be successfully simulated, considering the context of their use and the effectiveness of interactions. There are several research questions to which we have provided answers, including: How can the user be represented in the virtual environment, and what is the added value of this representation for medical training purposes? How can interaction fidelity impact user performance in surgical simulation-based training? What is the impact of augmented reality feedback on the performance of a patient's gait rehabilitation process? In addition, I will present two rehabilitation assistance systems for improving patient usability and motivation: 1) for upper limb motor rehabilitation after stroke in virtual reality; 2) gait rehabilitation in augmented reality.