To address the problem of real-time defect detection and shape grading of soft capsule, a telecentric vision based ellipse measurement method and its system were developed. An ellipse was obtained by the least square fitting of the pixel points of the capsule contour which was employed as the evaluation standard of ellipticity. Sequentially, the ellipticity was defined and the least square-based ellipticity evaluation model was established. The ellipticity was the difference between the diameter of the smallest circumscribed ellipse and that of the largest inscribed ellipse. Then, a telecentric vision inspection system for capsule defect and shape grading was established and the detection process was given. Finally, the experiments were carried out on the health products from a pharmaceutical company. The experimental results show that the accuracy of shape grading can reach 100% when the threshold is set reasonably and the detection time of a single capsule is about 0.712 s. The efficiency can be improved by developing parallel algorithm. The proposed ellipticity error evaluation method and the telecentric vision system can be utilized for capsule in line defect detection and shape grading.