Forest Runtime is designed to execute compiled neural network models seamlessly across diverse hardware platforms. Offering C++ APIs compatible with C and Python bindings, it caters to a wide range of AI applications. This runtime is modular and retargetable, making it adaptable for platforms from data centers to mobile and TinyML devices. Notable for its novel 'hot batching' technology, Forest Runtime flexibly alters batch sizes and input shapes during runtime, without needing compiler transformation, significantly speeding up processing times.
