What is a Loader?, what is an Engine?
- Engine
Engine refers to a package that has been semi-engineered and provides an interface to execute AI algorithms based on the model.
- AI capability
AI capability refers to the AI engine that has been completed as a cloud service. The AI capability user can directly integrate and invoke the API and the generated interface document.
- Engine Loader
The loader is the core component in the engine hosting platform. By integrating the loader, the engine and various cloud native components are connected to realize the purpose of cloud service.
- Decoder (loader plugin)
The decoder is a collection of loaders, deep learning algorithm computing modules, and middleware connecting the loader and computing modules. The middleware requires AI algorithm engine developers to develop and implement interfaces.
- Interface Protocol
The interface protocol is the definition of AI capability, which is a collection of the functional characteristics, input and output parameters, and error codes of the AI engine.
Summarize:
The engine loader realizes service by loading different models or engine services with the loader plug-in. The engine or model loaded by the engine loader is the core of this AI inference service framework.
Combined with the peripheral governance methods of the unified AI inference service framework, such as load balancing, distribution strategy, data processing, etc., to achieve unified management of various AI capabilities as a service.
Loader Language
The main process of the core loader is a Loader implemented by Go, that is, the project aiges
What is a plugin ?
That is, the layer between the loader and the final engine or model is the plug-in layer.
Plugin language
C/C++ See
Python (essentially implements a C/C++ so, which implements the interface required by the above C/C++, but at the same time exposes the python language programming interface to the user)