Robotics technology has been increasingly applied to healthcare contexts to enhance efficiency and safety in healthcare processes in recent years. People with mobility impairments and disabilities often require caretakers in their lives. Fortunately, robots can provide attention and assistance consistently for them instead of human caretakers. Motivated by this, we develop a robot-assisted e-health solution to empower the patients' daily lives and improve their wellbeing in this study. A transfer learning-based approach is proposed to train the robot to understand and identify patients' needs through a small dataset. Using the proposed approach, the robot is able to understand the patient's needs through speech recognition and recognize objects that the patient has requested. The proposed solution is experimentally implemented in real-world human-robot interactive healthcare contexts. Results and analysis indicate the success and accuracy of our approaches.