A zero-shot visual tracking model was proposed to ensure that the unmanned vehicles maintained high tracking accuracy even when the target was deformed or partially/fully occluded. Based on the classical Kalman filtering, a motion modeling module was introduced during the mask prediction phase and the predicted masks were iteratively refined by considering temporal-spatial consistency and integrating motion cues. A hybrid scoring system was employed to select the optimal mask from the predicted masks. For the historical optimal masks, a memory-aware selection module was designed to create an ideal mask candidate library and dynamically choose the most suitable mask by combining the historical features and information cues. The performance of the proposed method and several classical visual tracking models such as HIPTrack-B₃₈₄ was evaluated and compared on the LaSOT, GOT-10k, and OTB100 datasets. The results showed that, compared with the optimal values of the corresponding metrics in the comparison methods, the area under ROC curve (AUC), precision, average overlap, overlap precision at IoU thresholds 0.5 and 0.75, and success rate of the proposed model were improved by 2.87%, 2.73%, 2.84%, 3.18%, 5.46%, and 1.62%, respectively, indicating that the algorithm achieved good performance on multiple metrics.