Standard Protocol

ASE API protocol

1. Overall description

This protocol is oriented to AthenaServing, and it constrains and defines user development and internal business protocols.

2 Application Developer protocol

The protocol is described in json.

2.1 Metadata

Since AI service requests are generally used for data calculation and processing, some basic data needs to be carried in the interaction protocol. This protocol abstracts and binds data. In order to improve adaptability, this protocol enumerates and defines metadata. The basic description of the corresponding data type is defined in the metadata structure, and the description of the K-V single-layer structure is also allowed to be added. When the user defines data, after selecting the data type, the system will actively provide structure fields.

2.1.1 text description

Structure example:

{
    "encoding":"utf8", 
    "status":0,
    "seq": 1，
    "compress":"gzip",
    "custom1":"zz",
    "text":"hello word"
} 

Field	Meaning	Type	Description
encoding	Text encoding	string	Value range can be enumerated
status	Data status	int	The value range is 0 (start), 1 (continue), 2 (end), 3 (one transmission)
text	text data	string	text data, base64
seq	Data serial number	int	Indicates the block number of the data, optional
custom1	User-defined parameter	string or int	User-defined, first-level parameter

2.1.2 audio description

{
    "status":0，
    "sample_rate":16000,
    "channels":1,
    "bit_depth":16,
    "encoding":"opus",
    "seq": 1,
    "custom1":"zz",
    "audio":"xxxxxxxxxxxxxxxxxxxxxxxxxx" # 音频数据
} 

Field	Meaning	Type	Description
encoding	Audio encoding	string	Value range can be enumerated
status	Data status	int	The value range is 0 (start), 1 (continue), 2 (end), 3 (one transmission)
audio	audio data	string	is text data, base64
sample_rate	sample rate	int	audio sample rate, enumerable
channels	number of channels	int	number of channels, enumerable
bit_depth	bit depth	int	unit bit, enumerable
seq	Data serial number	int	Indicates the block number of the data, optional
custom1	User-defined parameter	string or int	User-defined, first-level parameter

2.1.3 to be improved

1 image
2 Audio
3 Other binary data, etc.

2.2 Request Protocol Description

It is divided into three parts: platform parameters, capability parameters, and transmission data. The request protocol template is as follows:

{
     "header":{},
     "parameter":{
         "service_iat":{
              "accept_1":{
            }
         }
     }, 
     "payload":{
        "service_iat":{
        }
     }
}

Field Description:

Field	Meaning	Type	Description
header	Platform parameters	Object	Parameters used to control platform features, such as appid, etc. This structure has only one level and cannot be modified by developers.
parameter	Ability parameter	Object	Used to control the AI engine feature switch. This segment will be transparently transmitted directly to the engine, multi-level structure, structure is determined, unenumerated structure is a first-level structure
payload	input data	Object	data used to carry the request, multi-level structure, structure determination
accept_1	Input data	Object	User-defined parameters used to describe the encoding\data format of the returned result, this structure is defined by the attributes defined in the metadata
service_iat	Input data	Object	User-defined, can be the same as accept_1 by default. Used to carry the requested data, multi-level structure, structure determination

2.2.1 Example of single capability and single data

{
    "header":{
    "appid":"1234",
    "uid":"d233"
    },
    "parameter":{
        "service_iat":{
           "language": "zh_cn",
           "domain":"iat",
           "accent": "mandarin"
           }
    }, 
    "payload":{
        "service_iat":{
        "status":0,
        "format":"audio/L16;rate=16000",
        "encoding":"raw",
        "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
        }
     }
 } 

2.2.2 Example of single capability and multiple data

    {
    "header":{
        "appid":"1234",
        "uid":"1234"
    },

    "parameter":{
      "service_iat":{
           "language": "zh_cn",
           "domain":"iat",
           "accent": "mandarin"
           }
    },

    "payload":{
        "data_1":{
            "status":0,
            "format":"audio/L16;rate=16000",
            "encoding":"raw",
              "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
         },
         "data_2":{
            "status":0,
            "format":"audio/L16;rate=16000",
            "encoding":"raw",
               "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
     }
}
} 

Structure description:

1. The payload is a multi-level structure.
2. In the case of multiple data, the payload is described by different data IDs as keys, such as data_1, data_2 in the example.
3. The data ID is user-defined.

2.2.3 Description when multiple data ranges feature

For the return of multiple data streams, the description of multiple data characteristics needs to be specified in the input parameter (parameter section).

"parameter":{
      "service_iat":{
           "language": "zh_cn",
           "domain":"iat",
           "accent": "mandarin",
           "accept_1":{
            "format":"audio/L16;rate=16000",
            "encoding":"raw"
            },
           "accept_2":{
            "format":"audio/L16;rate=16000",
            "encoding":"raw"
           }
       ｝

    } 

Structure description:

1. At this time, the parameter is a multi-level structure
2. In the case of multiple data, parameter uses different IDs as keys to describe different results, such as accept_1, accept_2 in the example
3. Accept ID is user-defined

2.2.4 Example of multiple capability and multiple data

    {
        "header":{
            "appid":"1234",
            "uid":"1234"
        },
        "parameter":{
            "service_1":{

                   "language": "zh_cn",
                   "domain":"iat",
                   "accent": "mandarin"
            },
            "service_2":{

                   "language": "zh_cn",
                   "domain":"iat",
                   "accent": "mandarin"
            }
        },

        "payload":{
            "service_1_data_1":{
                "status":0,
                "format":"audio/L16;rate=16000",
                "encoding":"raw",
                 "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
                  },

             "service_2_data_1":{
                "status":0,
                "format":"audio/L16;rate=16000",
                "encoding":"raw",
                 "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
              },
             "service_2_data_2":{
                "status":0,
                "format":"audio/L16;rate=16000",
                "encoding":"raw",
                "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
                 }
        }
    } 

Structure description:

1. At this time, the parameter is a multi-level structure, and the payload is a multi-level structure at this time.
2. In the case of multiple data, the parameter uses different capability IDs as keys to describe the feature descriptions of different capabilities, service_1, service_2.
3. Ability ID is user-defined.
4. In the case of multiple data, the payload is described by different data IDs as keys, such as service_1_data_1, service_2_data_1, service_2_data_2 in the example.
5. In this scenario, the data ID is generated by the interface of the orchestration platform.

2.2.5 In streaming scenarios, subsequent packet descriptions

The header and parameter are optional and are generally not carried. Take multiple data as an example:

 {
    "header":{
         "sid":"iat000704fa@dx16ade44e4d87a1c802", # optional
        },
    
    "payload":{
        "data_1":{
            "status":1,
            "format":"audio/L16;rate=16000",
            "encoding":"raw",
                    "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
         },

        "data_2":{
            "status":1,
            "format":"audio/L16;rate=16000",
            "encoding":"raw",
                    "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
         }
 } 

Field Description:

Field	Meaning	Type	Description
sid	The platform returns the session handle	Object	Returns the session handle carried in the message, optional

2.3 Back to protocol description

The return protocol defines the computational state of the session, as well as the data segment.

2.3.1 single output description

{
  "header":{
     "code": 0, 
     "message": "success", 
     "sid": "iat000704fa@dx16ade44e4d87a1c802"

},
# This structure is a metadata structure (data) that describes the return result.
 "payload": {
     “result_1”：{
        "status":0, # Data status
        "format":"audio/L16;rate=16000",
        "encoding":"raw",
           "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
        }
   }
}

Field	Meaning	Type	Description
code	error code	int	enumerable
message	error description	string	error message description
sid	The platform returns the session handle	Object	Returns the session handle carried in the message, optional
payload	input data	Object	used to carry the returned data, see [2.1 Definition of Metadata]("2.1 Definition of Metadata")

2.3.2 Multiple output description

{
    "header":{
        "code": 0,
         "message": "success",
         "sid": "iat000704fa@dx16ade44e4d87a1c802",
         "status":0     
        },
 
"payload":{
     "result_1":{
        "status":0,
        "format":"audio/L16;rate=16000",
        "encoding":"raw",
        "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
         },

     "result_2":{
        "status":0,
        "format":"audio/L16;rate=16000",
        "encoding":"raw",
        "audio":"exSI6ICJlbiIsCgkgICAgInBvc2l0aW9uIjogImZhbHNlIgoJf..."
         }
}
} 

Field Description: Same as [2.3.1 Description of single output]("2.3.1 single output description") Structure description:

1. At this time, the payload is a multi-level structure.
2. In the case of multiple data, the payload uses different result IDs as keys to describe different result feature descriptions, such as: result_1, result_2. .
3. The result ID is user-defined.

2.4 Constraints on System Parameters

2.4.1 header field

Field	Meaning	Type	Description
app_id	App id	string	Required
ath_id	3rd party user ID	string	optional

3.1 routing information

Added session state (SessState) to describe streaming or not

message GlobalRoute {
    string session_id = 1; //session id
    string trace_id = 2; //trace id
    string up_router_id = 3; //Uplink data routing ID
    string guider_id = 4; //Dispatch center ID
    string down_router_id = 5; //Downlink data routing ID
    string appid = 6; //App ID
    string uid = 7; //User ID
    string did = 8; //device ID
    string client_ip = 9; //client ip

    SessState session_state = 10; //Add: session state, streaming, non-streaming, STATE is enumeration

} 

3.1 Metadata adjustment

Remove the original format and encoding fields, put the description into the desc_args field, and adjust the original byte type to string type.

message GeneralData {
    string data_id = 1; //data number
    uint32 frame_id = 2; //Data serial number

    // distinguish data types
    enum DataType {
    TEXT = 0; // text
    AUDIO = 1; // audio
    IMAGE = 2; // image
    VIDEO = 3; // video
    }

    DataType data_type = 3; //Data type

    //distinguish data state
    enum DataStatus {
    BEGIN = 0; //begin
    CONTINUE = 1; //Follow the stream
    END = 2; //end
    ONCE = 3; //End of one call
    }

    DataStatus status = 4; //Data status

    map<string, string> desc_args = 5; //Data description parameters

    bytes data = 8; //data content
}