Alibaba Cloud

max.ldp@alibaba-inc.com

Cisco

suresh.krishnan@gmail.com

This document specifies a Agent-to-Agent communication framework enabling structured, low-latency, and semantically rich communication between autonomous agents over the Media over QUIC (MoQ) protocol. It leverages MoQ's efficient media transport capabilities while introducing a new application-layer framing mechanism to support control signaling, session management, and large data fragmentation. The design supports both intra-domain and inter-domain deployment, with an emphasis on interoperability, extensibility, and minimal overhead.

The multimodal capabilities of AI Agents, has created a need for a real-time communication framework that supports both media streaming and structured control signaling. Existing protocols such as HTTP/2, gRPC, either suffer from head-of-line blocking or lack native support for real-time media and bidirectional streaming. Media over QUIC (MoQ) offers a compelling transport foundation with its low-latency, multiplexed, and connection-robust design. However, MoQ is originally optimized for media delivery and lacks native semantics for agent-to-agent control, session establishment, and structured data exchange. This document proposes an application-layer protocol framework that operates over MoQ. It introduces:

• A binary framing structure to carry both control and media payloads.
• A session management mechanism for multi-turn agent dialogues.
• Strategies for fragmenting large control signals with partial reliability.
• Support for both MoQ domain-local and internet-wide communication.

The proposed framework is designed to be minimal, modular, and compatible with existing MoQ implementations. It does not modify MoQ's transport semantics but instead defines a new application-layer envelope that can be carried within MoQ Objects or Datagrams.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

MoQ:

Media over QUIC, a protocol for real-time media delivery.

Agent:

An autonomous entity capable of sending and receiving structured commands and media.

Session:

A logical dialogue between two agents, identified by a Session ID.

Frame:

The basic unit of agent to agent communication, consisting of a header and payload.

Agent Protocol over MoQ operates as an application-layer protocol on top of MoQ. The protocol stack is illustrated below:

Protocol Stack of Agent Protocol over MoQ

The stack consists of four main layers:

• Transport Layer (QUIC): Provides UDP-based, low-latency transport with built-in encryption (TLS 1.3), 0-RTT connection establishment, stream multiplexing, and congestion control.
• Media Transport Layer (MoQ): Handles track-based pub/sub, group and object organization, priority management, and datagram support for real-time media delivery.
• Application Framing Layer: Defines the Agent Protocol over MoQ binary frame structure with fixed header fields and variable payload, enabling efficient encapsulation of different message types.
• Application Logic: Implements agent-specific functionality including reasoning engines, command interpretation, media processing, and business logic.

Agents establish a MoQ session (via WebTransport or native QUIC client) and use one or more Tracks for communication. Each agent-to-agent message is encapsulated in a binary frame before being sent as a MoQ Object or Datagram. The framing layer provides session management, fragmentation support, and extensibility through version and flag fields.

The Agent to Agent Communication frame is a binary structure designed for efficiency and extensibility. It consists of a fixed-length header and a variable-length payload.

Field	Length (bytes)	Description	MoQ Mapping
Magic Number	2	Identifier: 0xA2A2	Detects agent-to-agent traffic
Version	1	Protocol version (e.g., 0x01)	Forward compatibility
Msg Type	1	Message type: CMD(0x01), DATA(0x02), STREAM(0x03), HEARTBEAT(0x04)	Route to handler
Session ID	8	Unique session identifier (UUIDv7 or hash)	Correlate multi-turn dialogues
Stream ID	4	Identifier for fragmented streams	Reassemble large data
Flags	1	Bitmask: compressed(0x01), encrypted(0x02), fragmented(0x04)	Processing hints
Payload Len	4	Length of the following payload	Parse boundary

The payload is interpreted based on the Msg Type and Flags:

• CMD: JSON or MessagePack-encoded command structure.
• DATA: Raw binary chunk of a large file or buffer.
• STREAM: Raw media sample or simplified RTP-like header + frame.
• HEARTBEAT: Empty payload; used for liveness detection.

Agent Protocol over MoQ uses JSON (JavaScript Object Notation) as the standard encoding format for structured message content:

JSON (JavaScript Object Notation) is used for all CMD and control messages:

• Human-readable and widely supported across programming languages
• Self-describing with key-value pairs
• Schema-flexible allowing dynamic field addition
• Standard library support in most modern languages

Example CMD message in JSON:

Media streams and large data payloads use raw binary encoding:

• STREAM messages: Raw media samples (H.264, Opus, etc.) without additional encoding overhead
• DATA messages: Raw binary chunks for file transfers and large payloads
• Maximum efficiency for bandwidth-constrained scenarios

Each message type has specific field requirements and processing rules:

CMD messages carry structured commands encoded in JSON. In addition to session management commands, CMD messages include specific actions for subscription management in pub/sub scenarios.

Field	Type	Required	Description	Example
action	String	Yes	Command identifier	"session_init", "session_ack", "subscribe_request", "unsubscribe_request", "track_ready"
session_id	String/UUID	Yes (for session cmds)	Session identifier	"abc-123-def-456"
auth_token	String	Yes (for session_init)	JWT with operation claim	"eyJhbGciOiJIUzI1NiIsIn..."
status	String	No	Response status	"approved", "denied", "error"
error_code	Integer	No	Error identifier	401, 403, 500
payload	Any	No	Command-specific data	{"key": "value"}

For subscription-related actions such as "subscribe_request" and "unsubscribe_request", CMD messages include additional fields:

Field	Type	Required	Description	Example
track_name	String	Yes (for subscribe)	Name of the track to subscribe to	"agent2.session456.video"
quality_params	Object	No	Quality parameters for the subscription	{"resolution": "1080p", "fps": 30}
subscriber_id	String	Yes (for subscribe)	ID of the subscribing agent	"agent1"

DATA messages carry raw binary chunks for large file transfers.

Field	Type	Required	Description	Constraints
chunk_index	Integer	Yes	Position in sequence	0-based, sequential
total_chunks	Integer	Yes (last chunk)	Total number of chunks	Must match in all chunks
checksum	String	No	Data integrity hash	SHA-256 hex encoded
metadata	Object	No	File/type information	{"filename": "file.txt"}

STREAM messages carry media samples with optional timing information.

Field	Type	Required	Description	Example
timestamp	Integer	Yes	Sample timestamp (ms)	1678901234567
media_type	String	Yes	Media content type	"video/h264", "audio/opus"
sequence_number	Integer	Yes	Frame sequence	12345
key_frame	Boolean	No	Is key frame	true
bitrate	Integer	No	Current bitrate (bps)	1000000

HEARTBEAT messages have empty payload and are used for liveness detection.

Field	Value	Description
Payload Length	0	No payload data
Processing	Ack only	Recipient should acknowledge receipt

Agent Protocol over MoQ frames are mapped to MoQ Objects and Datagrams based on their reliability and latency requirements:

• CMD frames: Mapped to MoQ Objects to ensure reliable delivery of control messages. These require acknowledgment and retransmission if lost.
• STREAM frames: Mapped to MoQ Objects by default for ordered delivery. May use MoQ Datagrams for loss-tolerant real-time streams where occasional packet loss is acceptable.
• DATA frames: Large data chunks are mapped to MoQ Objects for reliable transfer. Small data may use MoQ Datagrams for lower latency.
• HEARTBEAT frames: Mapped to MoQ Datagrams for efficient keep-alive functionality without requiring acknowledgments.

This mapping ensures that the transport characteristics of MoQ align with the application requirements of agent-to-agent communication.

The MoQ pub/sub mechanism is integrated with Agent Protocol over MoQ through specific CMD message types that manage subscriptions at the application layer:

• Subscription Requests: Agents send CMD messages with action "subscribe_request" to initiate track subscriptions.
• Subscription Management: The "subscribe_request" includes track name, quality parameters, and authorization tokens.
• Track Naming Convention: Track names follow the format "{agent_id}.{session_id}.{stream_type}", e.g., "agent1.abc-123-video".
• Authorization: Subscription requests are validated against the agent's authorization token before establishing the underlying MoQ subscription.

This approach provides application-layer control over the underlying MoQ pub/sub mechanism, allowing fine-grained access control and session management.

Agent Protocol over MoQ supports two communication modes:

Agents establish a session using a handshake that includes authorization verification. Once a session is established, agents can use subscription management commands to establish pub/sub relationships:

• Agent A obtains an authorization JWT containing an operation claim as defined in .
• Agent A sends a CMD frame with {action: "session_init", session_id: 0x123..., auth_token: "<JWT>"}.
• Agent B validates the JWT and verifies the operation permissions before responding.
• Agent B responds with CMD {action: "session_ack", session_id: 0x123..., status: "approved|denied"}.
• Subsequent messages use the same Session ID in the frame header.

The session terminates with a session_bye command or MoQ session closure.

For stateless interactions (e.g., heartbeat, fire-and-forget command), agents may send frames without session setup. The Session ID field is set to zero. Authorization for such interactions is typically pre-established or implicit.

Agent Protocol over MoQ defines three strategies based on payload size:

Sent as a single MoQ Datagram or Object. No application-layer fragmentation. Ideal for commands and heartbeats.

Application-layer fragmentation is required:

• Data is split into chunks (e.g., 1400 bytes).
• Each chunk is encapsulated in an agent protocol over MoQ frame with the same Stream ID.
• The last chunk sets the fragmented flag to 0 and includes final metadata.

Reassembly is performed at the receiver using Stream ID and order of arrival.

Continuous media (audio/video) is sent as a sequence of agent protocol over MoQ STREAM frames. Each frame carries one media sample. Loss of a single frame does not block subsequent ones, leveraging MoQ's partial reliability.

Agent Protocol over MoQ is designed to operate in two modes:

• Intra-MoQ-Domain: Agents communicate within the same MoQ infrastructure (e.g., same edge cluster). Low latency, high trust. MoQ-native discovery is RECOMMENDED.
• Inter-Domain (Internet-wide): Agents communicate across networks. STUN/TURN/ICE MAY be used if direct QUIC path fails.

Agent Protocol over MoQ does not define its own discovery mechanism but RECOMMENDS integration with MoQ's track discovery or external service directories.

Agent Protocol over MoQ inherits security from the underlying transport:

• QUIC provides TLS 1.3 encryption and 0-RTT resumption.
• Application-layer authorization uses JWT with operation claim as defined in .
• Session establishment requires validation of authorization tokens before communication begins.

Implementations MUST validate authorization tokens before processing any agent-to-agent communication. Session IDs MUST be cryptographically random to prevent session hijacking.

This document requests IANA to create new registries and register initial values for the Agent Protocol over MoQ.

IANA is requested to create a new registry for Agent Protocol over MoQ Magic Numbers under the "Media over QUIC (MoQ)" group. The registration procedure is Standards Action as defined in . Initial registration:

Value	Description	Reference
0xA2A2	Agent-to-Agent Communication Frame	[This document]

IANA is requested to create a new registry for Agent Protocol over MoQ Versions. The registration procedure is Standards Action. Initial registration:

Value	Description	Reference
0x01	Agent Protocol over MoQ Version 1	[This document]

Values 0x00 and 0xFF are reserved. Values 0x02-0xFE are available for assignment.

IANA is requested to create a new registry for Agent Protocol over MoQ Message Types. The registration procedure is Specification Required as defined in . Initial registrations:

Value	Message Type	Description	Reference
0x00	Reserved	Reserved for future use	[This document]
0x01	CMD	Command message with structured payload	[This document]
0x02	DATA	Raw binary data chunk	[This document]
0x03	STREAM	Media stream sample	[This document]
0x04	HEARTBEAT	Liveness detection message	[This document]

Values 0x05-0xFE are available for assignment. Value 0xFF is reserved.

IANA is requested to create a new registry for Agent Protocol over MoQ Frame Flags. The registration procedure is Specification Required. Each flag is represented by a bit position (0-7). Initial registrations:

Bit	Flag Name	Description	Reference
0	Compressed	Payload is compressed (0x01)	[This document]
1	Encrypted	Payload has application-layer encryption (0x02)	[This document]
2	Fragmented	Frame is part of a fragmented stream (0x04)	[This document]
3-7	Unassigned	Available for assignment	[This document]

IANA is requested to register a new well-known URI in the "Well-Known URIs" registry as defined in . The "/.well-known/a2a" endpoint facilitates agent discovery in inter-domain deployments. When accessed via HTTPS GET, the endpoint returns a JSON response containing the MoQ endpoint information for Agent Protocol over MoQ communication.

URI suffix:

a2a

Change controller:

IETF

Specification document:

This document

Related information:

Used for WebTransport-based Agent Protocol over MoQ endpoint discovery. The endpoint returns a JSON object with the MoQ endpoint information, e.g., { "moq_endpoint": "quic://api.example.com:443" }

Normative References Informative References

This appendix provides detailed examples of Agent Protocol over MoQ interactions based on a multimodal collaborative design scenario.

Agents:

• Agent 1 (Personal Assistant): Resides on a user's smartphone.
• Agent 2 (Professional Designer): Resides on a cloud server.

Used for lightweight, fire-and-forget interactions like liveness checks or simple status updates.

Agent 1 sends a heartbeat to Agent 2 to maintain the MoQ track activity.

Heartbeat Frame Implementation

Used for complex dialogues requiring continuous media processing, such as real-time video analysis.

Step 1: Session Handshake with Authorization

Session Establishment Sequence "} | |------------------------------------->| | | | CMD {action:"session_ack", | | session_id:"abc-123", | | status:"approved"} | |<-------------------------------------| ]]>

Step 2: Media Streaming Agent 1 captures video and sends it as STREAM frames within the established session.

Video Sample Frame Implementation

Step 1: Subscription Request Agent 1 wants to subscribe to Agent 2's video feed. It sends a subscription request within the established session.

Subscription Request Sequence | | | | CMD {action:"track_ready", | | session_id:"abc-123", | | status:"approved", | | track_url:"moq://track/agent2.video.h264"}| |<------------------------------------------------| ]]>

Step 2: Media Streaming via MoQ pub/sub After successful subscription negotiation via CMD messages, the actual media stream flows through the underlying MoQ pub/sub mechanism.

Used for transferring high-resolution assets or large model weights.

Agent 2 sends a 5MB design file back to Agent 1. Fragmentation Strategy: The file is split into chunks of 1400 bytes.

First Chunk Frame

Last Chunk Frame

Agents are deployed in the same regional data center (e.g., Alibaba Cloud Hangzhou Region).

• Discovery: Uses MoQ-native track discovery.
• Latency: Extremely low (< 5ms).
• Trust: High; encryption is present but overhead is minimal due to short physical distance.

Agent 1 (on Smartphone) calls Agent 2 (on Cloud).

Inter-Domain Connection Flow | | | | | { "moq_endpoint": "quic://api.cloud.example.com:443" } | |<----------------------------------------------| | | | | QUIC/MoQ Handshake (0-RTT) | |---------------------------------------------->| | | | | Agent Protocol over MoQ Frame | | (Session Init with Auth) | |---------------------------------------------->| ]]>

• Discovery: Uses HTTPS Well-Known URI as a bootstrap.
• Traversal: Relies on QUIC's robustness against NAT changes.
• Security: Inherits TLS 1.3 from QUIC; ensures end-to-end privacy across public networks.

Type	Session ID	Msg Type	Typical Usage	MoQ Mapping
Session-Less	0	HEARTBEAT / CMD	Keep-alive, Simple Queries	MoQ Datagram (HEARTBEAT), MoQ Object (CMD)
Streaming	UUID	STREAM	Audio/Video/Sensor Stream	MoQ Object (Ordered Delivery), MoQ Datagram (Loss-Tolerant)
Bulk Data	UUID	DATA	Large Files, Images	MoQ Object (Reliable), MoQ Datagram (Small Data)
Control	UUID	CMD	Logic Execution, ACKs, Subscription Management	MoQ Object (Reliable)