Smartwatches are expected to become the world's best-selling electronic product after smartphones. Various smart-watches have been released to the private consumer market, but the data on smartwatches is not well protected. In this paper, we show for the first time that photoplethysmography (PPG)signals influenced by hand gestures can be used to authenticate users on smartwatches. The insight is that muscle and tendon movements caused by hand gestures compress the arterial geometry with different degrees, which has a significant impact on the blood flow. Based on this insight, novel approaches are proposed to detect the starting point and ending point of the hand gesture from raw PPG signals and determine if these PPG signals are from a normal user or an attacker. Different from existing solutions, our approach leverages the PPG sensors that are available on most smartwatches and does not need to collect training data from attackers. Also, our system can be used in more general scenarios wherever users can perform hand gestures and is robust against shoulder surfing attacks. We conduct various experiments to evaluate the performance of our system and show that our system achieves an average authentication accuracy of 96.31 % and an average true rejection rate of at least 91.64% against two types of attacks.