FeEcho4D

Background & Motivation

Fetal echocardiography is the primary tool for prenatal detection of congenital heart disease (CHD) and assessment of cardiac function, guiding early intervention and parental decision-making. However, 2D fetal CHD screening achieves only ~50% detection accuracy and exhibits substantial variability across studies, often due to human error, subjectivity, and limited anatomical coverage. While 2D imaging remains standard in clinical practice, it can miss structural defects or provide incomplete views due to foreshortening, probe misalignment, or restricted field of view. In contrast, 3D echocardiography captures the full heart volume and can improve both detection and functional assessment. Nonetheless, although 3D/4D fetal echocardiography is available via Spatio-Temporal Image Correlation (STIC) acquisitions, it remains underused due to challenges in visualization and quantification. Deep learning (DL)–based 3D/4D cardiac reconstruction offers a practical solution to these limitations. To date, however, no DL algorithm has been developed for reconstructing cardiac geometry from 3D/4D fetal echocardiography. This gap is driven by two key factors: the lack of a public 3D/4D dataset and the inherent difficulty of generating accurate 3D ground-truth masks. Preparing such annotations is especially time-consuming in fetal imaging, where myocardial boundaries can be ambiguous, such as determining the transition from a U-shaped myocardium to a circular cross-section near the lateral wall. To address this, we introduce FeEcho4D, a first benchmark 4D fetal echocardiography dataset via 2.5D radial segmentation and slice-to-mesh reconstruction (3D) over time (4D), as shown in the following figure.

Radial slicing is a promising yet underexplored alternative to conventional Cartesian slicing, short-axis (SAX) and long-axis (LAX) views. By rotating a 2D slicing plane around the left ventricular long axis at fixed angular intervals (θ∈[0,π]), it provides structured coverage of the myocardium from multiple orientations. Compared with SAX and LAX views, radial views are easier to annotate and deliver more uniform anatomical coverage across the heart.

Figure: Overview of the FeEcho4D dataset generation pipeline (A) Radial data preparation and (B) 4D reconstruction of the fetal left ventricular heart.

FeEcho4D is the first benchmark dataset for fetal echocardiography with full 4D (3D over time) left ventricular meshes, designed to advance prenatal cardiac functional assessment and support clinical translation. It contains over 1,845 expert-annotated 3D volumes from 52 subjects, covering diverse gestational ages, disease conditions, and fetal poses to capture the complexity of real-world imaging. Additional segmented cases are in progress and will be published in future releases. FeEcho4D enables reliable segmentation, motion estimation, and 3D reconstruction for AI-driven fetal cardiac analysis. As illustrated in the figure above, the development of the dataset from raw 4D volumes to 4D mesh generation proceeds through six key steps:

Data extraction from raw STIC acquisitions using 4DView software (GE Healthcare, Chicago, IL, USA).
Radial slice extraction at each time frame.
Manual delineation of myocardium at clinically important time points—end-diastole (ED) and end-systole (ES).
Motion tracking to generate masks for intermediate frames.
Manual refinement of the propagated masks to correct tracking errors.
Slice-to-mask reconstruction for all time frames, producing complete 4D segmentations of each fetal echocardiography volume.

All steps are described in detail below, including technical and implementation specifics, to ensure reproducibility.

Dataset Preparation & Annotation

Data Extraction Step

Raw STIC acquisitions were retrieved directly from the fetal echocardiography system and processed with 4DView software (GE Healthcare, Chicago, IL, USA) to export temporally resolved 3D volumes (illustrated in the figure below). For each temporal extraction, slice number and spacing were kept constant to preserve the spatial–temporal integrity of the original 4D ultrasound data for subsequent analysis. The following figure shows examples of the 3D volume at different times for the same patient.

Figure: Raw STIC fetal echocardiography volume exported via 4DView software and corresponding radial slices at selected time frames.

Radial Slice Extraction Step

All volumes were radially resampled using Algorithm 1, as illustrated in the figure below. This procedure generates a set of uniformly sampled radial slices (H×W) at fixed angular intervals, which together form structured radial 3D volumes. Compared to conventional SAX and LAX views, where the left ventricular shape often transitions from an apparent U-shape to blob-like contours, radial slices consistently preserve a U-shaped anatomy across depths. This structural consistency greatly facilitates accurate and efficient manual annotation, as shown in the segmentation of all S slices in the figure below.

Algorithm 1 — Radial Slice Extraction

Inputs: 3D volume \( \mathbf{V}\in \mathbb{R}^{H \times W \times D} \), spacing \( \mathbf{S}=[s_x,s_y,s_z] \), angle set \( \Theta \).
Outputs: Radial slices \( \{ I_\theta \mid \theta \in \Theta \},\; I_\theta \in \mathbb{R}^{H \times W} \).

Select middle slice index \( \mathbf{M} \) along z and rotation center \( \mathbf{C}=[C_x,C_y] \) on that slice (LV center).
Generate coordinate mesh \( \mathbf{X}=\{(x_i,y_j,z_k)\} \) with voxel spacing: \[ \begin{aligned} x_i &= i\cdot s_x,\quad i=1,\dots,W \\ y_j &= j\cdot s_y,\quad j=1,\dots,H \\ z_k &= k\cdot s_z,\quad k=1,\dots,D \end{aligned} \]
Translate to center: \[ \mathbf{X}^C_p = \mathbf{X}_p - [\,C_x s_x,\; C_y s_y,\; M s_z\,],\quad p\in\{i,j,k\}. \]
For each \( \theta \in \Theta \):
1. Define y-axis rotation \[ R_y(\theta^{\mathrm r})= \begin{bmatrix} \cos\theta^{\mathrm r} & 0 & \sin\theta^{\mathrm r}\\ 0&1&0\\ -\sin\theta^{\mathrm r}&0&\cos\theta^{\mathrm r} \end{bmatrix},\quad \theta^{\mathrm r}=\theta\cdot\frac{\pi}{180}. \]
2. Rotate and inverse-translate: \[ \mathbf{X}^{\mathrm{rot}}=R_y(\theta^{\mathrm r})\,\mathbf{X}^C,\qquad \mathbf{X}^{\mathrm{target}}_p=\mathbf{X}^{\mathrm{rot}}_p+[\,C_x s_x,\; C_y s_y,\; M s_z\,]. \]
3. Interpolate: \[ V^{\mathrm{rot}}(i,j,k)=\mathrm{interp3}\!\left(\mathbf{V},\;\mathbf{X}^{\mathrm{target}}\right). \]
4. Slice at \( k=M \) to obtain \( I_\theta = V^{\mathrm{rot}}(:,:,M) \).

Figure: Radial slice extraction at a given rotation angle (θ) with corresponding ultrasound image and myocardium mask.

Manual Delineation Step

Using a custom-built annotation tool (details in our SegmentationApp), experts manually delineated the left ventricular myocardium and cavity by marking points along their boundaries (see video below). The tool supports 3D volume loading, interactive contour drawing, and real-time quality control through mask overlay. It also provides temporal navigation to capture cardiac motion, particularly valuable in cases affected by acoustic dropout, and annotation toggling for iterative refinement. To ensure accuracy and consistency, all annotations underwent multiple rounds of expert review followed by cardiologist validation.

Figure: Manual delineation of myocardium at ED and ES by SegmentationApp.

Motion Tracking & Manual Refinement Step

FeEcho4D generates dense cardiac masks by propagating sparse ED and ES annotations using B-spline Fourier (BSF) motion model (Algorithm 2), which blends local and global deformations and regularizes trajectories to capture periodic motion (as shown in the following figure). This ensures temporally coherent, anatomically consistent masks, further refined with skeleton-based width correction (Algorithm 3) to maintain uniform myocardial thickness, and all results are manually reviewed for accuracy.

Algorithm 2 — BSF-Based Temporal Mask Propagation

Inputs: Radial slices \( \{x_\theta^t\}_{t=1}^T \), ED/ES masks \( M_{\mathrm{ED}}, M_{\mathrm{ES}} \), spacing \( s \), Fourier order \( N \), B-spline level \( L \).
Outputs: Dense masks \( M_t \in \mathbb{R}^{S\times H\times W},\ t=1,\dots,T \).

(Registration) For \( t=0 \) to \( T-2 \):
- \(\mathrm{Register}(x_\theta^t, x_\theta^{t+1}) \Rightarrow \mathbf{d}_{t\to t+1}\)
- \(\mathrm{Register}(x_\theta^0, x_\theta^{t+1}) \Rightarrow \mathbf{d}_{0\to t+1}\)
(Grid Initialization): \[ \mathbf{X}_0 = \mathrm{GridFromMask}(M_{\mathrm{ED}}) \]
(Grid Propagation) For \( t=0 \) to \( T-2 \): \[ \mathbf{X}_{t+1} = w(t)\cdot(\mathbf{X}_t + \mathbf{d}_{t\to t+1}) + (1-w(t))\cdot(\mathbf{X}_0 + \mathbf{d}_{0\to t+1}) \]
(BSF Fitting): \[ \mathbf{X}^{\mathrm{BSF}}_t = \sum_{f=0}^N \mathbf{F}_f \cos\!\left(\frac{2\pi f t}{P}\right) + \mathbf{F}_{f+N+1} \sin\!\left(\frac{2\pi f t}{P}\right) \] \[ \mathbf{F}_f(x,y) = \sum_{i,j} B_i(x) B_j(y)\cdot \mathbf{C}_{i,j,f} \]
(Trajectory Correction): Repeat until cost < threshold:
- For each control point \( u \): \[ \mathbf{C}_u = \arg\min \sum_{t,\Delta t} \left\| \mathbf{X}^{\mathrm{BSF}}_{t+\Delta t} - \mathbf{X}^{\mathrm{BSF}}_t - \mathbf{d}_{t\to t+\Delta t} \right\|^2 \]
(Mask Synthesis): For \( t=0 \) to \( T-1 \), for each \( p\in M_{\mathrm{ED}} \):
- \( p_t = \mathbf{X}^{\mathrm{BSF}}_t(p) \), set \( M_t(p_t) \leftarrow 1 \)

Return: \( \{M_t\}_{t=1}^T \)

Algorithm 3 — Skeleton-Based 2D Mask Refinement

Inputs: Raw propagated mask \( M_t \in \mathbb{R}^{S\times H\times W} \); tail–ratio \( \tau\in(0,1) \); opening radius \( r \ge 0 \).
Outputs: Refined mask \( \widetilde{M}_t \in \mathbb{R}^{S\times H\times W} \) with consistent width.

For \( s = 1 \) to \( S \):
1. (Component Selection): \( M'_s \gets \mathrm{LargestComponent}(M_t[s]) \)
2. (Skeleton Extraction): \( S_s \gets \mathrm{Skeletonize}(M'_s) \)
3. (Tail Extrapolation): Let \( S_{\mathrm{tail}} \) be the \( \lceil \tau |S_s| \rceil \) farthest points from \( \mathrm{centroid}(S_s) \); Fit line \( \ell_s \) via least squares and set \( S_s \gets S_s \cup (\ell_s \setminus S_s) \)
4. (Mean Half-Width Estimation): \[ d(x) = \min_{y \in \overline{M'_s}} \| x - y \|_2,\quad x \in \Omega \] \[ h_s = \left\lfloor \frac{1}{|S_s|} \sum_{x \in S_s} d(x) + \frac{1}{2} \right\rfloor \]
5. (Uniform Mask Synthesis): \[ \widetilde{M}_t[s] = \{ x \in \Omega \mid \mathrm{dist}(x, S_s) \le h_s \} \] If \( r > 0 \): \[ \widetilde{M}_t[s] \gets \mathrm{Opening}(\widetilde{M}_t[s],\ \mathrm{disk}(r)) \]

Return: \( \widetilde{M}_t \)

3D Reconstruction & Biomarker Computation

3D Reconstruction with GHD

The 3D over time left ventricular meshes are reconstructed into smooth, temporally consistent meshes using Graph Harmonic Deformation (GHD) (details in our teamwork: GHD). The meshes are further processed with Differentiable Voxelization & Slicing (DVS) (as shown in the following figure below) (details in Algorithm 4), enabling differentiable conversion to voxel grids and slices for precise dynamic metrics and seamless integration with learning-based models. The resulting 4D meshes of ten different example patients are shown in the following video.

Algorithm 4 — GHD-Based 3D Mesh Reconstruction

Inputs: Binary mask sequence \( \{M_t\}_{t=1}^T, \; M_t \in \mathbb{R}^{S\times H\times W} \); canonical mesh \( \mathcal{M}_0 \in \mathbb{R}^{V\times 3} \); graph basis \( U \in \mathbb{R}^{V\times B} \); radial slices \( \{x_\theta^t\}_{t=1}^T \).
Outputs: Deformed mesh sequence \( \{\mathcal{M}_t\}_{t=1}^T,\; \mathcal{M}_t \in \mathbb{R}^{V\times 3} \).

(Point Cloud Sampling): For \( t = 1 \) to \( T \):
- Sample myocardium and background point clouds from \( M_t \): \( P_{\mathrm{myo}}^t, \; P_{\mathrm{bg}}^t \subset \mathbb{R}^{N\times 3} \)
(Mesh Deformation): For \( t = 1 \) to \( T \):
- Deform canonical mesh using spectral basis: \( \mathcal{M}_t = \mathcal{M}_0 + U \cdot \phi_t \)
- Optimize \( \phi_t \in \mathbb{R}^{B\times 3} \) to minimize total loss
(DVS Supervision): For \( t = 1 \) to \( T \):
- Project mesh via DVS: \( \hat{x}_\theta^t = \mathrm{DVS}(\mathcal{M}_t) \)
- Compute Dice loss: \( \mathcal{L}_{\mathrm{DVS}}^t = 1 - \mathrm{Dice}(x_\theta^t, \hat{x}_\theta^t) \)
(Objective Optimization): \[ \min_{\phi_t} \; \mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{DVS}} + \lambda_1 \mathcal{L}_{\mathrm{lap}} + \lambda_2 \mathcal{L}_{\mathrm{norm}} + \lambda_3 \mathcal{L}_{\mathrm{thick}} \]

Return: \( \{\mathcal{M}_t\}_{t=1}^T \)

Figure: Slice-to-mesh reconstruction using DVS with GHD mesh fitting.

Figure: Reconstructed 4D left ventricular meshes from the FeEcho4D dataset.

Clinical Metrics Calculation

Clinical biomarkers, such as Stroke Volume (SV), Ejection Fraction (EF), Global Longitudinal Strain (GLS), and Global Circumferential Strain (GCS), are derived from the reconstructed left ventricular meshes, as explained in detail in Algorithm 5 and the following figure below. SV and EF are computed from endocardial volumes, while GLS and GCS are obtained by averaging changes in apex-to-base lengths and mid-ventricular circumferences across representative planes or cross-sections, excluding extreme outliers for robustness.

Algorithm 5 — Clinical Metrics Computation

(Volume-Based Metrics):
- Measure endocardial volumes \( V_{\mathrm{ENDO,ED}} \) and \( V_{\mathrm{ENDO,ES}} \) at end-diastole (ED) and end-systole (ES).
- Compute: \[ SV = EDV - ESV,\quad EF = \frac{EDV - ESV}{EDV} \times 100\%. \]
(Global Longitudinal Strain — GLS):
- Slice the LV epicardial mesh into \( N \) radial planes aligned with the long axis, indexed by \( \theta \). (In our study: \( N = 30 \))
- For each plane, measure apex-to-base lengths \( L_\theta^{\mathrm{ED}}, L_\theta^{\mathrm{ES}} \).
- Exclude the 5 longest and 5 shortest planes to retain \( N' \) representative planes. (In our study: \( N' = 20 \))
- Compute: \[ GLS = \frac{\frac{1}{N'}\sum_{\theta\notin OL} L_\theta^{\mathrm{ES}} - \frac{1}{N'}\sum_{\theta\notin OL} L_\theta^{\mathrm{ED}}}{\frac{1}{N'}\sum_{\theta\notin OL} L_\theta^{\mathrm{ED}}} \times 100\%. \]
(Global Circumferential Strain — GCS):
- Extract \( N \) short-axis cross-sections within the mid-ventricular region, indexed by \( i \). (In our study: \( N = 6 \))
- Measure circumferences \( C_i^{\mathrm{ED}}, C_i^{\mathrm{ES}} \) using ellipse fitting.
- Exclude the 1 largest and 1 smallest circumferences to retain \( N' \) representative cross-sections. (In our study: \( N' = 4 \))
- Compute: \[ GCS = \frac{\frac{1}{N'}\sum_{i\notin OL} C_i^{\mathrm{ES}} - \frac{1}{N'}\sum_{i\notin OL} C_i^{\mathrm{ED}}}{\frac{1}{N'}\sum_{i\notin OL} C_i^{\mathrm{ED}}} \times 100\%. \]

Outlier removal (OL) ensures robust estimation by excluding extreme geometric deviations caused by noise or irregular contours.

Dataset Access & Licensing

Data access is available upon request for non-commercial research use. Please contact the dataset team or fill in the application form (coming soon).

Code Repository & Publications

Code

Coming soon. Code and pre-trained models will be released on our GitHub.

Paper

The FeEcho4D dataset and framework are described in our paper:
"4D Reconstruction of Fetal Left Ventricle from Echocardiography via 2.5D Radial Segmentation and Graph-Fourier Reconstruction"
Preprint available soon.

[Download PDF]