This paper proposes a two-pass clustering algorithm with a combination of the linear assignment and fuzzy C-means methods (FCM) for polarimetric SAR (PolSAR) image segmentation. To avoid the inconsistency of clustering results from the fuzzy C-means method with random initialization, the linear assignment method with the least similar cluster representatives is applied first to generate initial clusters, and then followed with the FCM method. Appropriate initial clustering centres adjacent to the actual final clustering centres can be found to promote the convergence speed of the overall iterative process and drastically reduce the calculation time. Otherwise, the modified algorithm is updated from multidimensional data analysis to PolSAR image clustering. This approach is applied to four well-known practical UCI datasets and public PolSAR image segmentation. The results are compared with those from the fuzzy C-means method with other initialization methods. It is shown that the two pass approach consistently results in the best clustering results. The application results on PolSAR image segmentation are also demonstrated.