Community

Hi everyone,

I'm currently working on a project that requires transitioning from offline video processing to real-time streaming video analysis using the JapanCV Cloud SDK (v3.5.1).

Initially, I used BatchProcessor.analyzeVideo() to process MP4 files, achieving a stable inference latency of about 25ms/frame.

However, when I switched to the RealtimeStreamProcessor module and called the processFrame(frameBuffer) function with the exact same model (ResNet50 optimized to ONNX format) and ran it on the same VM Compute Noto-L4 configuration (which is designated for real-time tasks), my average latency spiked to 45-55ms/frame for the first 30 minutes of operation. After the 30-minute mark, the latency gradually stabilized back to around 30ms.

The precise issue is: The initial latency is unexpectedly high and inconsistent. I have already verified:

Cold Start: The issue occurs after the model has been loaded and initialized.

Network Bandwidth: No bottleneck detected (network API latency is stable below 5ms).

Garbage Collection/Memory: Background memory cleanup tasks have been disabled.

My exact question is:

Has anyone observed this behavior? Does the RealtimeStreamProcessor perform any deep internal warming-up or caching activity that lasts over 30 minutes and is not documented? Or is there a specific configuration flag within StreamConfig that I should be setting to force immediate resource allocation on the Noto-L4?

Thanks very much for any insights! google baseball