Skip to main content
Version: release-2.2.0

What is a Loader?, what is an Engine?

  • Engine

Engine refers to a package that has been semi-engineered and provides an interface to execute AI algorithms based on the model.

  • AI capability

AI capability refers to the AI engine that has been completed as a cloud service. The AI capability user can directly integrate and invoke the API and the generated interface document.

  • Engine Loader

The loader is the core component in the engine hosting platform. By integrating the loader, the engine and various cloud native components are connected to realize the purpose of cloud service.

Loader code

  • Decoder (loader plugin)

The decoder is a collection of loaders, deep learning algorithm computing modules, and middleware connecting the loader and computing modules. The middleware requires AI algorithm engine developers to develop and implement interfaces.

  • Interface Protocol

The interface protocol is the definition of AI capability, which is a collection of the functional characteristics, input and output parameters, and error codes of the AI engine.

Summarize:

The engine loader realizes service by loading different models or engine services with the loader plug-in. The engine or model loaded by the engine loader is the core of this AI inference service framework.

Combined with the peripheral governance methods of the unified AI inference service framework, such as load balancing, distribution strategy, data processing, etc., to achieve unified management of various AI capabilities as a service.

Loader Language

The main process of the core loader is a Loader implemented by Go, that is, the project aiges

What is a plugin ?

That is, the layer between the loader and the final engine or model is the plug-in layer.

Plugin language

  • C/C++ See

  • Python (essentially implements a C/C++ so, which implements the interface required by the above C/C++, but at the same time exposes the python language programming interface to the user)

See details