<p>Ultrasound images are widely used for diagnosis of congenital abnormalities of the kidney and urinary tract (CAKUT). Since a typical clinical ultrasound image captures 2D information of a specific view plan of the kidney and images of the same kidney on different planes have varied appearances, it is challenging to develop a computer aided diagnosis tool robust to ultrasound images in different views. To overcome this problem, we develop a multi-instance deep learning method for distinguishing children with CAKUT from controls based on their clinical ultrasound images, aiming to automatic diagnose the CAKUT in children based on ultrasound imaging data. Particularly, a multi-instance deep learning method was developed to build a robust pattern classifier to distinguish children with CAKUT from controls based on their ultrasound images in sagittal and transverse views obtained during routine clinical care. The classifier was built on imaging features derived using transfer learning from a pre-trained deep learning model with a mean pooling operator for fusing instance-level classification results. Experimental results have demonstrated that the multi-instance deep learning classifier performed better than classifiers built on either individual sagittal slices or individual transverse slices.</p>

<p><strong>OBJECTIVE: </strong>To reliably and quickly diagnose children with posterior urethral valves (PUV), we developed a multi-instance deep learning method to automate image analysis.</p>

<p><strong>METHODS: </strong>We built a robust pattern classifier to distinguish 86 children with PUV from 71 children with mild unilateral hydronephrosis based on ultrasound images (3504 in sagittal view and 2558 in transverse view) obtained during routine clinical care.</p>

<p><strong>RESULTS: </strong>The multi-instance deep learning classifier performed better than classifiers built on either single sagittal images or single transverse images. Particularly, the deep learning classifiers built on single images in the sagittal view and single images in the transverse view obtained area under the receiver operating characteristic curve (AUC) values of 0.796±0.064 and 0.815±0.071, respectively. AUC values of the multi-instance deep learning classifiers built on images in the sagittal and transverse views with mean pooling operation were 0.949±0.035 and 0.954±0.033, respectively. The multi-instance deep learning classifiers built on images in both the sagittal and transverse views with a mean pooling operation obtained an AUC of 0.961±0.026 with a classification rate of 0.925±0.060, specificity of 0.986±0.032, and sensitivity of 0.873±0.120, respectively. Discriminative regions of the kidney located using classification activation map demonstrated that the deep learning techniques could identify meaningful anatomical features from ultrasound images.</p>

<p><strong>CONCLUSION: </strong>The multi-instance deep learning method provides an automatic and accurate means to extract informative features from ultrasound images and discriminate infants with PUV from male children with unilateral hydronephrosis.</p>

<p>Ultrasound imaging (US) is commonly used in nephrology for diagnostic studies of the kidneys and lower urinary tract. However, it remains challenging to automate the disease diagnosis based on clinical 2D US images since they provide partial anatomic information of the kidney and the 2D images of the same kidney may have heterogeneous appearance. To overcome this challenge, we develop a novel multi-instance deep learning method to build a robust classifier by treating multiple 2D US images of each individual subject as multiple instances of one bag. Particularly, we adopt convolutional neural networks (CNNs) to learn instance-level features from 2D US kidney images and graph convolutional networks (GCNs) to further optimize the instance-level features by exploring potential correlation among instances of the same bag. We also adopt a gated attention-based MIL pooling to learn bag-level features using full-connected neural networks (FCNs). Finally, we integrate both instance-level and bag-level supervision to further improve the bag-level classification accuracy. Ablation studies and comparison results have demonstrated that our method could accurately diagnose kidney diseases using ultrasound imaging, with better performance than alternative state-of-the-art multi-instance deep learning methods.</p>

<p>It remains challenging to automatically segment kidneys in clinical ultrasound images due to the kidneys' varied shapes and image intensity distributions, although semi-automatic methods have achieved promising performance. In this study, we developed a novel boundary distance regression deep neural network to segment the kidneys, informed by the fact that the kidney boundaries are relatively consistent across images in terms of their appearance. Particularly, we first use deep neural networks pre-trained for classification of natural images to extract high-level image features from ultrasound images, then these feature maps are used as input to learn kidney boundary distance maps using a boundary distance regression network, and finally the predicted boundary distance maps are classified as kidney pixels or non-kidney pixels using a pixel classification network in an end-to-end learning fashion. Experimental results have demonstrated that our method could effectively improve the performance of automatic kidney segmentation, significantly better than deep learning based pixel classification networks.</p>

<p>It remains challenging to automatically segment kidneys in clinical ultrasound (US) images due to the kidneys' varied shapes and image intensity distributions, although semi-automatic methods have achieved promising performance. In this study, we propose subsequent boundary distance regression and pixel classification networks to segment the kidneys automatically. Particularly, we first use deep neural networks pre-trained for classification of natural images to extract high-level image features from US images. These features are used as input to learn kidney boundary distance maps using a boundary distance regression network and the predicted boundary distance maps are classified as kidney pixels or non-kidney pixels using a pixelwise classification network in an end-to-end learning fashion. We also adopted a data-augmentation method based on kidney shape registration to generate enriched training data from a small number of US images with manually segmented kidney labels. Experimental results have demonstrated that our method could automatically segment the kidney with promising performance, significantly better than deep learning-based pixel classification networks.</p>