The template-based matching method for building morphology recognition is prone to interference from local features. To improve the accuracy of building morphology recognition and classification, on the basis of the advantages of traditional template-based matching method, a building shape classification model was constructed by introducing least squares template, skeleton lines and feature vector. The impact of small depressions and protrusions was avoided in the method by constructing least squares templates and extracting skeleton lines. Based on that, the feature vectors of the building and the templates’ skeleton lines were calculated, the cosine similarity was introduced to calculate their similarity, and the template with the highest similarity was treated as the classification result for building shapes. Data of 4540 buildings were used for classification validation in the experiment, to verify the feasibility and effectiveness of the proposed method. Results show that the method can resolve the interference problem of local features on the overall shape description of the building, resulting in high classification accuracy. The method lays foundation for further building matching and generalization evaluation operations.