Governance, Audit and Observability - AG2
Governance & Audit
The hub enforces governance and surfaces observability through a few layered seams:
- Evaluators — pure functions over channel state that return zero or more
Violationrecords when their thresholds are breached. - Violation handlers — what to do when a violation fires: log to the audit trail, notify the channel, or auto-close.
- The audit log — append-only record of every governance-relevant event the hub processes (itself a
HubListener). HubListener— read-only observers the hub fans state transitions out to, after the fact.HubArbiter— the gatekeeper the hub consults before committing register / channel-open / send / dispatch decisions.
Evaluators
Three evaluators ship today, addressed by name in adapter manifests:
| Name | Class | Threshold |
|---|---|---|
"acks_within" |
AcksWithinEvaluator |
All invitees must ack within params["seconds"] of channel creation. |
"reply_within" |
ReplyWithinEvaluator |
The respondent must reply within params["seconds"] of the initiator's first send (consulting only). |
"max_silence" |
MaxSilenceEvaluator |
No participant may go silent for longer than params["seconds"]. |
"turn_within" |
(composes from the above) | The next speaker must speak within params["seconds"] of being scheduled. |
Each evaluator implements:
class ExpectationEvaluator(Protocol):
name: ClassVar[str]
def evaluate(self, ctx: ExpectationContext) -> list[Violation]: ...
ExpectationContext is a small dataclass holding the metadata, WAL slice, current time, and the expectation's params. Evaluators are pure — no I/O, no mutation — so they're trivially testable.
The default registry exposes them as default_evaluators(). Custom evaluators register similarly to custom transition targets.
Adapter-Declared Expectations
Each adapter's manifest declares its defaults (see Adapters Overview for the table). Examples:
# ConsultingAdapter
expectations = [\
Expectation(name="acks_within", on_violation="auto_close", params={"seconds": 30}),\
Expectation(name="reply_within", on_violation="auto_close", params={"seconds": 600}),\
]
# ConversationAdapter
expectations = [\
Expectation(name="max_silence", on_violation="audit", params={"seconds": 3600}),\
]
Expectation.on_violation selects the handler:
on_violation |
Handler | Effect |
|---|---|---|
"audit" |
AuditHandler |
Write to the audit log only. Channel continues. |
"warn" |
NotifyChannelHandler |
Post EV_EXPECTATION_VIOLATED on the channel WAL. |
"auto_close" |
AutoCloseHandler |
Close the channel with reason="expectation_violated:<name>"; record to audit. |
"hide" |
(custom) | Hide later turns from the offending participant; not yet implemented as a built-in. |
The default registry exposes them as default_handlers().
The Sweeper Loop
When the hub is open, an expectation sweeper task wakes every expectation_sweep_interval (default 10 s), walks every active channel, runs each expectation's evaluator, and dispatches any violations to the configured handler.
For deterministic tests / examples:
<br> 1<br> 2<br> 3<br> 4<br> 5<br> 6<br> 7<br> 8<br> 9<br>10<br>11<br> |
<br>from autogen.beta.network import Hub<br>from autogen.beta.knowledge import MemoryKnowledgeStore<br>hub = await Hub.open(<br> MemoryKnowledgeStore(),<br> expectation_sweep_interval=0, # disable background loop<br>)<br># Manually advance state and tick:<br>clock.advance(45) # mock-clock pattern<br>await hub._expectation_tick() # operator API<br> |
hub._expectation_tick() is a public-by-convention test entry point — a leading underscore, but exercised explicitly by the test suite.
Audit Log
hub.audit_log is an AuditLog instance — append-only, and itself a registered HubListener (so every state transition the hub fans out also lands as one structured record). It writes a single audit.jsonl under the hub's KnowledgeStore.
<br>1<br>2<br>3<br> |
<br>records = await hub.audit_log.read_all()<br>for r in records:<br> print(r["kind"], r["at"], r)<br> |
Each record is a plain dict with at minimum kind and at; kind-specific fields appear alongside.
| Member | Purpose |
|---|---|
await hub.audit_log.read_all() |
Read + parse the whole log. [] if absent. |
await hub.audit_log.append(record) |
Write one record. The kind set is open — tenants/subclasses append their own kind values here. |
hub.audit_log.subscribe(cb) / unsubscribe(cb) |
Live tail — cb(record) fires per appended record (no polling). Subscriber exceptions are logged and swallowed. |
hub.audit_log.bytes_written |
Process-local byte counter (resets on hub restart). Surfaced by hub.health() as audit_log_bytes. |
hub.replace_audit_log(custom) |
Swap in a tenant-provided AuditLog subclass (e.g. a different on-disk format). The replacement is registered as the first listener so audit writes still complete before tenant listeners observe the same event. |
Audit kinds
Re-exported as constants from autogen.beta.network:
| Constant | Notes |
|---|---|
AUDIT_KIND_AGENT_REGISTERED |
Records agent_id, name. |
AUDIT_KIND_AGENT_UNREGISTERED |
Records agent_id, name. |
AUDIT_KIND_RESUME_SET |
Records the source: RESUME_SOURCE_TENANT (a set_resume call) or RESUME_SOURCE_OBSERVED (a record_observation). |
AUDIT_KIND_SKILL_SET |
Records updated skill markdown. |
AUDIT_KIND_RULE_SET |
Records the new rule. |
AUDIT_KIND_CHANNEL_CREATED |
Records creator_id, manifest type/version, participants. |
AUDIT_KIND_CHANNEL_CLOSED |
Records reason. |
AUDIT_KIND_CHANNEL_EXPIRED |
Records the TTL details. |
AUDIT_KIND_TASK_TERMINATED |
Records owner_id, capability, outcome, latency_ms. |
AUDIT_KIND_EXPECTATION_VIOLATED |
Records expectation, channel_id, evaluator details. |
AUDIT_KIND_TURN_FAILED |
A notify handler crashed processing an inbound envelope. Records channel_id, agent_id, envelope_id, exc_type, exc_message. |
Inspection patterns
<br> 1<br> 2<br> 3<br> 4<br> 5<br> 6<br> 7<br> 8<br> 9<br>10<br>11<br>12<br>13<br>14<br>15<br>16<br> |
<br># Filter to violations only.<br>violations = [<br> r for r in await hub.audit_log.read_all()<br> if r["kind"] == AUDIT_KIND_EXPECTATION_VIOLATED<br>]<br># Filter to one channel.<br>channel_records = [<br> r for r in await hub.audit_log.read_all()<br> if r.get("channel_id") == channel_id<br>]<br># Live tail.<br>async def on_record(record: dict) -> None:<br> print("[audit]", record["kind"], record)<br>hub.audit_log.subscribe(on_record)<br> |
The AuditLog is durable when the hub is backed by DiskKnowledgeStore; with MemoryKnowledgeStore it lives only as long as the hub.
HubListener — observing state transitions
A HubListener is a read-only observer. Attach one with hub.register_listener(...); the hub awaits the matching method after the corresponding state change commits — listeners observe, they don't gate (that's HubArbiter, below). Registration itself is inert: there's no startup hook, and methods only fire on subsequent transitions. Each listener call is wrapped in its own try/except so a throwing listener can't stall dispatch — the exception is logged at ERROR and the next listener still runs.
BaseHubListener is a no-op base — subclass it and override only the events you care about (you don't have to implement the full Protocol surface). The built-in AuditLog is a HubListener, pre-registered on every hub, which is why a fresh hub already reports registered_listeners: 1.
<br> 1<br> 2<br> 3<br> 4<br> 5<br> 6<br> 7<br> 8<br> 9<br>10<br>11<br>12<br>13<br>14<br> |
<br>from autogen.beta.network import BaseHubListener<br>class MetricsListener(BaseHubListener):<br> async def on_envelope_posted(self, envelope, metadata) -> None:<br> metrics.incr("network.envelopes", tags={"type": envelope.event_type})<br> async def on_envelope_rejected(self, envelope, reason) -> None:<br> metrics.incr("network.rejected", tags={"reason": type(reason).__name__})<br> async def on_turn_failed(self, channel_id, agent_id, envelope_id, exc) -> None:<br> logger.error("turn failed: agent=%s channel=%s", agent_id, channel_id, exc_info=exc)<br>hub.register_listener(MetricsListener())<br># hub.unregister_listener(listener) # detach later — no-op if absent<br> |
| Method | Fires when |
|---|---|
on_envelope_posted(envelope, metadata) |
An envelope was validated, WAL-appended, folded, and dispatched. |
on_envelope_rejected(envelope, reason) |
An envelope was rejected before WAL append. reason is the typed NetworkError the sender saw. |
on_dispatch_failed(envelope, recipient_id, reason) |
Delivery of an accepted envelope to one recipient failed (the rest of the audience may have received it). |
on_channel_event(channel_id, kind, payload) |
kind ∈ opened / closed / expired / participant_removed / participant_hidden. |
on_agent_event(agent_id, kind, payload) |
kind ∈ registered / unregistered / resume_set / skill_set / rule_set / observation_recorded. |
on_expectation_fired(channel_id, expectation, violation) |
An evaluator emitted a violation (deduped per (channel, expectation, violator)). |
on_turn_failed(channel_id, agent_id, envelope_id, exc) |
A notify handler crashed processing an inbound envelope. |
on_task_event(task_id, kind, payload) |
kind ∈ started / progress / completed / failed / expired / cancelled / mirror_failed. |
on_inbox_pressure(agent_id, pending, cap) |
A recipient's pending count first crossed its inbox high-water mark (fires once per crossing). |
Subclassing the Hub instead of registering a listener
The same on_* methods are available on Hub itself (with no-op defaults) — so if you're building a custom hub you can override them directly rather than registering the hub as a listener of itself:
<br> 1<br> 2<br> 3<br> 4<br> 5<br> 6<br> 7<br> 8<br> 9<br>10<br> |
<br>from autogen.beta.network import Hub<br>class ObservingHub(Hub):<br> async def on_envelope_posted(self, envelope, metadata) -> None:<br> metrics.incr("network.envelopes", tags={"type": envelope.event_type})<br> async def on_inbox_pressure(self, agent_id, pending, cap) -> None:<br> logger.warning("inbox pressure: %s at %d/%d", agent_id, pending, cap)<br>hub = await ObservingHub.open(store)<br> |
Subclass overrides fire alongside any externally-registered listeners, with the same per-callee try/except isolation. Use a subclass when the observation logic belongs to your hub implementation; use register_listener(...) when it's a separate concern (metrics shipper, audit tap) you want to attach and detach independently.
on_task_event is the one listener hook that isn't purely hub-driven — TaskMirror (and other tenant code) emit "mirror_failed" and other kinds through it. The public way to fan one out is await hub_client.fire_task_event(task_id, kind, payload) (from a registered tenant) or await hub.fire_task_event(...) (direct); neither touches the hub's private fan-out.
Health snapshot
hub.health() is a cheap, in-memory operational snapshot — wire it to a /health endpoint or dashboard:
<br> 1<br> 2<br> 3<br> 4<br> 5<br> 6<br> 7<br> 8<br> 9<br>10<br> |
<br>hub.health()<br># {<br># "active_channels": 2,<br># "registered_agents": 5,<br># "pending_inbox_total": 3,<br># "max_pending_inbox_depth": 2, # None when nothing queued — indicative of a stuck agent<br># "registered_listeners": 1, # the built-in AuditLog counts<br># "adapters_loaded": 4,<br># "audit_log_bytes": 8192,<br># }<br> |
HubArbiter — the decision seam
Where a HubListener only observes, a HubArbiter decides. The hub consults the active arbiter inline before committing register / channel-open / send / dispatch decisions. Exactly one arbiter is active at a time; install yours with hub.register_arbiter(arbiter) (and read it back via hub.arbiter).
Each gate returns a Decision — Allow() or Deny(reason, error=...), where error selects which NetworkError subclass the hub raises back to the caller (defaults to AccessDeniedError):
| Gate | Called before | Default RuleBasedArbiter checks |
|---|---|---|
authorize_send(envelope, sender, sender_rule, recipients) |
post_envelope WAL append |
access.outbound_to, limits.delegation_depth |
authorize_inbox(envelope, recipient, recipient_rule, current_pending) |
per-recipient, on post_envelope |
limits.inbox.max_pending (denies with InboxFull) |
authorize_dispatch(envelope, sender, recipient, recipient_rule) |
each notify frame | access.inbound_from (deny ⇒ silently skip that recipient) |
authorize_channel_open(manifest, creator, creator_rule, invitees, invitee_rules, active_creator_channels) |
create_channel |
each invitee's access.inbound_from, creator's limits.max_concurrent_channels |
authorize_register(passport, resume, rule) |
(reserved — not yet wired by the hub) | always Allow |
resolve_unknown_audience(envelope, unknown_ids) |
dispatch to ids the hub doesn't know | returns None (drop silently) — the federation hook |
The default RuleBasedArbiter enforces the per-agent Rule (access + limits) — exactly the behavior the hub had inline before this seam existed. Two ways to customise:
- Add policy on top — subclass
RuleBasedArbiterandawait super()in the gates you extend. - Start from scratch — subclass
BaseHubArbiter(all gates returnAllowby default) and implement only the ones you need.
<br> 1<br> 2<br> 3<br> 4<br> 5<br> 6<br> 7<br> 8<br> 9<br>10<br>11<br>12<br>13<br>14<br>15<br>16<br>17<br> |
<br>from autogen.beta.network import RuleBasedArbiter, Allow, Deny, EV_TEXT<br>class ContentGuardArbiter(RuleBasedArbiter):<br> BANNED = ("password", "ssn")<br> async def authorize_send(self, envelope, sender, sender_rule, recipients):<br> base = await super().authorize_send(envelope, sender, sender_rule, recipients)<br> if isinstance(base, Deny):<br> return base<br> if envelope.event_type == EV_TEXT:<br> text = str(envelope.event_data.get("text", "")).lower()<br> hit = next((b for b in self.BANNED if b in text), None)<br> if hit is not None:<br> return Deny(reason=f"message blocked: contains {hit!r}")<br> return Allow()<br>hub.register_arbiter(ContentGuardArbiter())<br> |
A Deny from authorize_send / authorize_inbox / authorize_channel_open surfaces to the caller as the chosen NetworkError (so channel.send(...) raises AccessDeniedError); a Deny from authorize_dispatch just drops that one recipient. resolve_unknown_audience is the seam a federated arbiter uses to re-route to a local proxy id instead of dropping.
Turn-failure resilience
The default notify handler wraps its entire substantive path — channel resolve, view projection, adapter.extract_turn_input, agent.ask, round-envelope build, outbound send. If any step raises, the handler:
- routes the failure through
HubClient.report_turn_failure→Hub.report_turn_failure, - which fans
on_turn_failed(channel_id, agent_id, envelope_id, exc)out to everyHubListener— the built-inAuditLogwrites anAUDIT_KIND_TURN_FAILEDrecord, - then returns cleanly. No reply envelope is posted, but the channel stays active and the next envelope flows normally — a buggy turn no longer takes down the receive loop.
React however you like — retry, escalate, surface to a UI — by registering a listener that overrides on_turn_failed.
Custom Evaluators
Same shape as the built-ins:
<br> 1<br> 2<br> 3<br> 4<br> 5<br> 6<br> 7<br> 8<br> 9<br>10<br>11<br>12<br>13<br>14<br>15<br>16<br>17<br>18<br>19<br>20<br> |
<br>from typing import ClassVar<br>from autogen.beta.network.hub import (<br> ExpectationContext,<br> ExpectationEvaluator,<br> Violation,<br>)<br>class TooManyMessagesEvaluator:<br> name: ClassVar[str] = "too_many_messages"<br> def evaluate(self, ctx: ExpectationContext) -> list[Violation]:<br> threshold = ctx.params["max"]<br> text_count = sum(1 for e in ctx.wal if e.event_type == EV_TEXT)<br> if text_count > threshold:<br> return [Violation(<br> expectation=self.name,<br> channel_id=ctx.channel.channel_id,<br> detail=f"text count {text_count} exceeds {threshold}",<br> )]<br> return []<br> |
Register on a custom registry and pass to Hub.open(..., evaluators=registry). The default registry can also be mutated via the module-level register_evaluator(...) helper.