Casting Microstructure Inspection Using Computer Vision: Dendrite Spacing in Aluminum Alloys

by Filip Nikolić 1,2,3,Ivan Štajduhar 4,* andMarko Čanađija 1,*
1Department of Engineering Mechanics, Faculty of Engineering, University of Rijeka, 51000 Rijeka, Croatia
2Research and Development Department, CIMOS d.d. Automotive Industry, 6000 Koper, Slovenia
3CAE Department, Elaphe Propulsion Technologies Ltd., 1000 Ljubljana, Slovenia
4Department of Computer Engineering, Faculty of Engineering, University of Rijeka, 51000 Rijeka, Croatia
*Authors to whom correspondence should be addressed.

Download PDF

Abstract

This paper investigates the determination of secondary dendrite arm spacing (SDAS) using convolutional neural networks (CNNs). The aim was to build a Deep Learning (DL) model for SDAS prediction that has industrially acceptable prediction accuracy. The model was trained on images of polished samples of high-pressure die-cast alloy EN AC 46000 AlSi9Cu3(Fe), the gravity die cast alloy EN AC 51400 AlMg5(Si) and the alloy cast as ingots EN AC 42000 AlSi7Mg. Color images were converted to grayscale to reduce the number of training parameters. It is shown that a relatively simple CNN structure can predict various SDAS values with very high accuracy, with a R2 value of 91.5%. Additionally, the performance of the model is tested with materials not used during training; gravity die-cast EN AC 42200 AlSi7Mg0.6 alloy and EN AC 43400 AlSi10Mg(Fe) and EN AC 47100 Si12Cu1(Fe) high-pressure die-cast alloys. In this task, CNN performed slightly worse, but still within industrially acceptable standards. Consequently, CNN models can be used to determine SDAS values with industrially acceptable predictive accuracy.

Korea

이 논문은 컨볼루션 신경망 (CNN)을 사용하여 이차 수상 돌기 팔 간격 (SDAS)의 결정을 조사합니다. 목표는 산업적으로 허용 가능한 예측 정확도를 가진 SDAS 예측을 위한 딥 러닝 (DL) 모델을 구축하는 것이었습니다. 이 모델은 고압 다이 캐스트 합금 EN AC 46000 AlSi9Cu3 (Fe), 중력 다이 캐스트 합금 EN AC 51400 AlMg5 (Si) 및 잉곳으로 주조 된 합금 EN AC 42000 AlSi7Mg의 연마 된 샘플 이미지로 훈련되었습니다. 컬러 이미지는 훈련 매개 변수의 수를 줄이기 위해 회색조로 변환되었습니다. 비교적 간단한 CNN 구조는 매우 높은 정확도로 다양한 SDAS 값을 예측할 수 있습니다. 아르자형291.5 %의 가치. 또한 모델의 성능은 훈련 중에 사용되지 않은 재료로 테스트됩니다. 중력 다이 캐스트 EN AC 42200 AlSi7Mg0.6 합금 및 EN AC 43400 AlSi10Mg (Fe) 및 EN AC 47100 Si12Cu1 (Fe) 고압 다이 캐스트 합금. 이 작업에서 CNN은 약간 더 나빴지만 여전히 산업적으로 허용되는 표준 내에 있습니다. 결과적으로 CNN 모델은 산업적으로 허용 가능한 예측 정확도로 SDAS 값을 결정하는 데 사용할 수 있습니다.

Keywords: secondary dendrite arm spacing; convolutional neural network; casting microstructure inspection; deep learning; aluminum alloys

Figure 1. SDAS definition: the distance between two secondary dendrites.
Figure 1. SDAS definition: the distance between two secondary dendrites.
Figure 2. The procedure of deriving different S values using different magnifications on the microscope: (a) 5× magnification image; (b) 10× magnification image. The scale bar corresponds to S value.
Figure 2. The procedure of deriving different S values using different magnifications on the microscope: (a) 5× magnification image; (b) 10× magnification image. The scale bar corresponds to S value.
Figure 3. 200×200 pixel images used for training, their derived S values and alloy.
Figure 3. 200×200 pixel images used for training, their derived S values and alloy.
Figure 4. Images of different types of defects used for the training: (a,b)—air and shrinkage porosity defects; (c,d)—scratches; (e)—blurred image; (c,f,g,i)—different brightness and contrast of the image; (g,h)—ECS (some images are purportedly of lower quality).
Figure 4. Images of different types of defects used for the training: (a,b)—air and shrinkage porosity defects; (c,d)—scratches; (e)—blurred image; (c,f,g,i)—different brightness and contrast of the image; (g,h)—ECS (some images are purportedly of lower quality).
Figure 5. Schematic representation of dataset generation: the original image was first split into 70 images with a pixel resolution of 208×211, which were then resized to 200×200 pixel.
Figure 5. Schematic representation of dataset generation: the original image was first split into 70 images with a pixel resolution of 208×211, which were then resized to 200×200 pixel.
Figure 6. The CNN architecture used for estimating SDAS directly from microstructure image patches.
Figure 6. The CNN architecture used for estimating SDAS directly from microstructure image patches.
Figure 7. Images of materials used during the training: The images were split into 200×200 pixel images following the procedure in Figure 5 and used for the prediction task.
Figure 7. Images of materials used during the training: The images were split into 200×200 pixel images following the procedure in Figure 5 and used for the prediction task.
Figure 8. Images of materials that were not used during the training: The images were split into 200×200 pixel images following the procedure in Figure 5 and used for the prediction task.
Figure 8. Images of materials that were not used during the training: The images were split into 200×200 pixel images following the procedure in Figure 5 and used for the prediction task.

References

  1. Vandersluis, E.; Ravindran, C. Relationships between solidification parameters in A319 aluminum alloy. J. Mater. Eng. Perform. 2018, 27, 1109–1121. [Google Scholar] [CrossRef]
  2. Vandersluis, E.; Ravindran, C. Influence of solidification rate on the microstructure, mechanical properties, and thermal conductivity of cast A319 Al alloy. J. Mater. Sci. 2019, 54, 4325–4339. [Google Scholar] [CrossRef]
  3. Djurdjevič, M.; Grzinčič, M. The effect of major alloying elements on the size of the secondary dendrite arm spacing in the as-cast Al-Si-Cu alloys. Arch. Foundry Eng. 2012, 12, 19–24. [Google Scholar] [CrossRef]
  4. Brusethaug, S.; Langsrud, Y. Aluminum properties, a model for calculating mechanical properties in AlSiMgFe-foundry alloys. Metall. Sci. Tecnol. 2000, 18, 3–7. [Google Scholar]
  5. Shabestari, S.; Shahri, F. Influence of modification, solidification conditions and heat treatment on the microstructure and mechanical properties of A356 aluminum alloy. J. Mater. Sci. 2004, 39, 2023–2032. [Google Scholar] [CrossRef]
  6. Kabir, M.S.; Ashrafi, A.; Minhaj, T.I.; Islam, M.M. Effect of foundry variables on the casting quality of as-cast LM25 aluminium alloy. Int. J. Eng. Adv. Technol. 2014, 3, 115–120. [Google Scholar]
  7. Seifeddine, S.; Wessen, M.; Svensson, I. Use of simulation to predict microstructure and mechanical properties in an as-cast aluminium cylinder head comparison-with experiments. Metall. Sci. Tecnol. 2006, 24, 7. [Google Scholar]
  8. Morri, A. Empirical models of mechanical behaviour of Al-Si-Mg cast alloys for high performance engine applications. Metall. Sci. Technol. 2010, 28, 25–32. [Google Scholar]
  9. Grosselle, F.; Timelli, G.; Bonollo, F.; Della Corte, E. Correlation between microstructure and mechanical properties of Al-Si cast alloys. La Metallurgia Italiana 2009, 27, 25–32. [Google Scholar]
  10. Qi, M.; Li, J.; Kang, Y. Correlation between segregation behavior and wall thickness in a rheological high pressure die-casting AC46000 aluminum alloy. J. Mater. Res. Technol. 2019, 8, 3565–3579. [Google Scholar] [CrossRef]
  11. Singh, S.; Bhadeshia, H.; MacKay, D.; Carey, H.; Martin, I. Neural network analysis of steel plate processing. Ironmak. Steelmak. 1998, 25, 355–365. [Google Scholar]
  12. Hancheng, Q.; Bocai, X.; Shangzheng, L.; Fagen, W. Fuzzy neural network modeling of material properties. J. Mater. Process. Technol. 2002, 122, 196–200. [Google Scholar] [CrossRef]
  13. Agrawal, A.; Deshpande, P.D.; Cecen, A.; Basavarsu, G.P.; Choudhary, A.N.; Kalidindi, S.R. Exploration of data science techniques to predict fatigue strength of steel from composition and processing parameters. Integr. Mater. Manuf. Innov. 2014, 3, 8. [Google Scholar] [CrossRef]
  14. Liu, R.; Kumar, A.; Chen, Z.; Agrawal, A.; Sundararaghavan, V.; Choudhary, A. A predictive machine learning approach for microstructure optimization and materials design. Sci. Rep. 2015, 5, 11551. [Google Scholar] [CrossRef]
  15. Yang, X.W.; Zhu, J.C.; Nong, Z.S.; Dong, H.; Lai, Z.H.; Ying, L.; Liu, F.W. Prediction of mechanical properties of A357 alloy using artificial neural network. Trans. Nonferrous Met. Soc. China 2013, 23, 788–795. [Google Scholar] [CrossRef]
  16. Santos, I.; Nieves, J.; Penya, Y.K.; Bringas, P.G. Machine-learning-based mechanical properties prediction in foundry production. In Proceedings of the 2009 ICCAS-SICE, Fukuoka City, Japan, 18–21 August 2009; pp. 4536–4541. [Google Scholar]
  17. Liao, H.; Zhao, B.; Suo, X.; Wang, Q. Prediction models for macro shrinkage of aluminum alloys based on machine learning algorithms. Mater. Today Commun. 2019, 21, 100715. [Google Scholar] [CrossRef]
  18. Agrawal, A.; Choudhary, A. Deep materials informatics: Applications of deep learning in materials science. MRS Commun. 2019, 9, 779–792. [Google Scholar] [CrossRef]
  19. Herriott, C.; Spear, A.D. Predicting microstructure-dependent mechanical properties in additively manufactured metals with machine-and deep-learning methods. Comput. Mater. Sci. 2020, 175, 109599. [Google Scholar] [CrossRef]
  20. Ferguson, M.K.; Ronay, A.; Lee, Y.T.T.; Law, K.H. Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning. Smart Sustain. Manuf. Syst. 2018, 2, 1–42. [Google Scholar] [CrossRef]
  21. Hanumantha Rao, D.; Tagore, G.; Ranga Janardhana, G. Evolution of Artificial Neural Network (ANN) model for predicting secondary dendrite arm spacing in aluminium alloy casting. J. Braz. Soc. Mech. Sci. Eng. 2010, 32, 276–281. [Google Scholar] [CrossRef]
  22. Wei, J.; Chu, X.; Sun, X.Y.; Xu, K.; Deng, H.X.; Chen, J.; Wei, Z.; Lei, M. Machine learning in materials science. InfoMat 2019, 1, 338–358. [Google Scholar] [CrossRef]
  23. Azimi, S.M.; Britz, D.; Engstler, M.; Fritz, M.; Mücklich, F. Advanced steel microstructural classification by deep learning methods. Sci. Rep. 2018, 8, 2128. [Google Scholar] [CrossRef]
  24. Li, X.; Zhang, Y.; Zhao, H.; Burkhart, C.; Brinson, L.C.; Chen, W. A transfer learning approach for microstructure reconstruction and structure-property predictions. Sci. Rep. 2018, 8, 13461. [Google Scholar] [CrossRef]
  25. Mery, D. Aluminum Casting Inspection Using Deep Learning: A Method Based on Convolutional Neural Networks. J. Nondestruct. Eval. 2020, 39, 12. [Google Scholar] [CrossRef]
  26. DeCost, B.L.; Lei, B.; Francis, T.; Holm, E.A. High throughput quantitative metallography for complex microstructures using deep learning: A case study in ultrahigh carbon steel. arXiv 2018, arXiv:1805.08693. [Google Scholar] [CrossRef]
  27. Chowdhury, A.; Kautz, E.; Yener, B.; Lewis, D. Image driven machine learning methods for microstructure recognition. Comput. Mater. Sci. 2016, 123, 176–187. [Google Scholar] [CrossRef]
  28. Exl, L.; Fischbacher, J.; Kovacs, A.; Oezelt, H.; Gusenbauer, M.; Yokota, K.; Shoji, T.; Hrkac, G.; Schrefl, T. Magnetic microstructure machine learning analysis. J. Phys. Mater. 2018, 2, 014001. [Google Scholar] [CrossRef]
  29. Yucel, B.; Yucel, S.; Ray, A.; Duprez, L.; Kalidindi, S.R. Mining the Correlations Between Optical Micrographs and Mechanical Properties of Cold-Rolled HSLA Steels Using Machine Learning Approaches. Integr. Mater. Manuf. Innov. 2020, 9, 240–256. [Google Scholar] [CrossRef]
  30. Pokuri, B.S.S.; Ghosal, S.; Kokate, A.; Sarkar, S.; Ganapathysubramanian, B. Interpretable deep learning for guided microstructure-property explorations in photovoltaics. NPJ Comput. Mater. 2019, 5, 95. [Google Scholar] [CrossRef]
  31. DeCost, B.L.; Holm, E.A. A computer vision approach for automated analysis and classification of microstructural image data. Comput. Mater. Sci. 2015, 110, 126–133. [Google Scholar] [CrossRef]
  32. DIN EN 1706: Aluminium und Aluminiumlegierungen-Gussstücke-Chemische Zusammensetzung und mechanische Eigenschaften; Beuth: Berlin, Germany, 1998.
  33. Vandersluis, E.; Ravindran, C. Comparison of measurement methods for secondary dendrite arm spacing. Metallogr. Microstruct. Anal. 2017, 6, 89–94. [Google Scholar] [CrossRef]
  34. Nikolic, F.; Štajduhar, I.; Čanađija, M. Aluminum microstructure inspection using deep learning: A convolutional neural network approach toward secondary dendrite arm spacing determination. In 4th Edition of My First Conference; University of Rijeka, Faculty of Engineering: Rijeka, Croatia, 2020. [Google Scholar]
  35. Alexander Mordvintsev, A.K.R. Image Processing in OpenCV. 2013. Available online: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_table_of_contents_imgproc/py_table_of_contents_imgproc.html (accessed on 5 September 2020).