NVIDIA releases NV-Raw2Insights-US, an AI system for adaptive ultrasound imaging from raw sensor data

NVIDIA and Siemens Healthineers have jointly developed NV-Raw2Insights-US, an AI model designed to reconstruct ultrasound images directly from raw sensor signals rather than from pre-processed data. Traditional ultrasound imaging uses hand-engineered beamforming pipelines that make simplifying assumptions—notably assuming constant sound speed throughout tissue—and discard much of the original signal information during reconstruction. The new approach trains on raw ultrasound channel data, the uncompressed echoes captured by the probe, to learn patient-specific characteristics of how sound propagates through individual anatomy.

The core capability demonstrated is real-time estimation of local sound-speed variation within a patient. By inferring a personalized sound-speed map, the system adapts image focusing parameters on the fly, correcting for anatomical differences that traditional fixed-speed assumptions cannot accommodate. NVIDIA frames this as part of a broader 'Raw2Insights' class of models that extract actionable clinical insights from raw sensor data rather than finished images.

Hardware integration uses NVIDIA's open-source Holoscan Sensor Bridge (HSB), an FPGA implementation that bridges existing ultrasound scanners to GPU infrastructure. A demonstration uses an Altera Agilex-7 FPGA paired with an ACUSON Sequoia scanner to capture raw data via DisplayPort outputs and transmit it over Ethernet to NVIDIA IGX or DGX systems running Holoscan software. Inference on Blackwell-class GPUs produces sound-speed estimates that are streamed back to the scanner to improve live imaging focus.

The architecture is positioned as modular and software-defined, enabling continuous updates and integration of new AI models without hardware changes. The team released the model weights, dataset, and GitHub repositories on Hugging Face. However, the technology remains under investigational development without FDA clearance or availability for clinical sale, and future availability is not guaranteed. The work builds on published research including differentiable beamforming methods and deep learning-based sound speed estimation from 2023.