Cruz & Thompson - Coming Online: Enabling Real-Time and AI-Ready Scientific Discovery | SciPy 2024

HoloScan is an AI-enabled, real-time sensor processing platform from NVIDIA that enables high-bandwidth, low-latency data processing on GPUs

The platform supports data rates up to 400 Gbps with direct GPU memory access, eliminating CPU bottlenecks for packet processing

Key features include:

• Multi-GPU and multi-node support
• Python and C++ bindings
• Zero-copy data movement between operators
• Plug-and-play AI inferencing with ONIX, TorchScript, and TensorRT
• Built-in profiling and visualization tools

The architecture uses a directed acyclic graph (DAG) approach with operators that can be connected via input/output ports

The platform includes two types of network operators:

• Basic network operator (Python + C++)
• Advanced network operator (C++ only) for high bandwidth/low latency

Demonstrated real-world applications include:

• Allen Telescope Array data processing
• GPU-based FM demodulation
• Real-time AI inferencing for radio astronomy

Future development focuses on:

• Digital twin simulation capabilities
• Automated experiment steering
• Improved data reduction and storage optimization
• Extended processing pipeline capabilities

The project is open source (Apache 2.0) with strong community involvement and integration with existing scientific Python tools

Designed to be sensor-agnostic and work with various data types including RF, medical imaging, and general scientific instruments

Emphasizes developer productivity by abstracting away complex hardware interactions while maintaining high performance