An automatic registration between vehicle 3D laser point cloud and panoramic image based on gray similarity was proposed, under the scenes of unknown sensor parameters, unclear environmental structure characteristics and small amount of image data. Firstly, multiple single images were spliced into panoramic image and 3D laser point cloud was converted into 2D depth image, respectively, based on the panoramic stitching algorithm and cylindrical projection principle. Secondly, based on the principle of gray similarity, the panoramic image and 2D depth map were subdivided into region pairs at equal intervals along the horizontal and vertical directions, and the panoramic image was moved along the horizontal and vertical directions. The proportion of the sum of pixel gray values between each pair of subdivided regions after each move was calculated, and its mean square deviation was solved, and the region move value with the smallest mean square deviation was taken as the final matching offset. Finally, the horizontal rotation angle and vertical translation distance of the panoramic image relative to the 3D laser point cloud were calculated according to the offsets. Experimental results show that the algorithm has good adaptability to the scenes and the average registration error was 2 pixels, while the comparison method cannot achieve effective registration.