Docs
Aetower Diagnostics And Debug Console Spec
Status
Current state: partial diagnostics, no real debug console.
What exists today:
- live snapshot UI
- timeline of notable changes
- persisted history view
- capability health states like
configured,live,cached,degraded - OTLP export settings
- snapshot JSON export
- adapter-side cached error strings
What is missing:
- structured logs
- a searchable in-app diagnostics console
- subsystem-level debug state
- event correlation across engine ticks
- operator-grade export bundles
- stable diagnostics schemas and retention policy
This document defines the target diagnostics system for Aetower.
Goals
- Make Aetower explain itself when attribution, ranking, enrichment, or export look wrong.
- Give operators a first-class in-app console instead of forcing terminal-only debugging.
- Preserve low overhead. Diagnostics must be bounded, sampled, and tiered.
- Make diagnostics stable enough to compare behavior across versions.
- Keep the system useful for both local debugging and longitudinal product improvement.
Non-Goals
- Turning Aetower into a generic log aggregation system.
- Shipping unbounded verbose logs by default.
- Capturing raw user-sensitive content unless explicitly allowed.
- Building a remote SaaS diagnostics backend in the first phase.
Product Position
Aetower should have three diagnostics layers:
- User-facing explanation layer.
This is already partly present through ranking reasons, timeline, history, and recommendations.
- Operator diagnostics layer.
This is missing. It should expose internal health, adapter freshness, attribution decisions, exporter state, and runtime errors.
- Engineering trace layer.
This is also missing. It should make bugs reproducible with bounded structured events and snapshot-linked context.
Core Principles
- Structured first.
Diagnostics events must be typed records, not freeform strings.
- Snapshot-linked.
Every important diagnostics event should reference sequence, captured_at_millis, and when possible entity_id, component, adapter, or capability.
- Bounded by design.
No infinite buffers. Every queue, log store, and export path needs a cap and a retention window.
- Tiered verbosity.
Default should stay lightweight. Deep tracing must be opt-in and time-limited.
- Privacy-aware.
Diagnostics should default to metadata, counters, state, and hashes. Raw titles, command lines, URLs, and socket endpoints should be redacted or explicitly marked as sensitive.
Current Gap Summary
Existing surfaces
TimelineViewHistoryView- capability health in settings
exportSnapshotJSON- adapter runtime detail strings
Missing surfaces
- logs browser / console
- live engine status
- adapter fetch traces
- attribution decision traces
- friction calculation inspection
- telemetry exporter delivery status
- persistence read/write status
- diagnostics bundle export
Target Architecture
Introduce a new crate:
aetower-diagnostics
Responsibilities:
- diagnostics event schema
- bounded in-memory ring buffer
- optional persisted diagnostics store
- filtering and query API
- redaction helpers
- event correlation helpers
- lightweight counters and gauges for internal runtime health
Dependency direction
aetower-coreemits diagnosticsaetower-diagnosticsstores and queries diagnostics- FFI exposes read/query/export APIs
- macOS app renders a diagnostics console
Do not invert this. Diagnostics must not pull UI or app concerns into core crates.
Event Model
Every diagnostics event should share a common envelope:
struct DiagnosticsEvent {
id: String,
timestamp_millis: u64,
level: DiagnosticsLevel,
subsystem: DiagnosticsSubsystem,
event_type: DiagnosticsEventType,
sequence: Option<u64>,
entity_id: Option<String>,
adapter: Option<String>,
capability: Option<String>,
message: String,
fields: BTreeMap<String, DiagnosticsValue>,
sensitive: bool,
}
Levels
tracedebuginfowarnerror
Subsystems
enginecollectoridentityattributionfrictionhistorypersistencetelemetrygpuffiuiadapter.chromiumadapter.dockeradapter.helperadapter.chau7adapter.vscode
Event Types
tick_startedtick_completedtick_over_budgetsnapshot_publishedhistory_persistedhistory_load_failedadapter_refresh_startedadapter_refresh_succeededadapter_refresh_failedcapability_state_changedcapability_health_changedattribution_decisionentity_reclassifiedfriction_reason_emittedtelemetry_export_startedtelemetry_export_succeededtelemetry_export_failedgpu_sample_missinggpu_sample_readffi_call_failedui_refresh_slow
Required Diagnostics Producers
Engine
Emit:
- tick start/end
- tick duration breakdown
- publish sequence changes
- stale frontmost state
- export worker cadence
- over-budget warnings
Required fields:
sequencetick_milliscollect_millisidentity_millisattribution_millisfriction_millisentity_countprocess_count
Collector
Emit:
- host environment refresh
- process refresh mode changes
- wakeup sampling failures
- cwd probing failures
- GPU sample availability
Required fields:
full_scanknown_pid_countprocess_counthost_cpuhost_memory_used_byteswakeups_total
Identity / Attribution
Emit on meaningful changes only:
- entity seed merged
- provenance assigned
- launcher inferred
- entity kind changed
- helper grouped into root app
- browser/container/terminal classification decision
Required fields:
pidentity_iddisplay_nameprovenance_kindruleconfidence
Friction
Emit when:
- top reason set changes
- total friction crosses thresholds
- recommendation set changes
Required fields:
entity_idscore_totalscore_cpuscore_memoryscore_diskscore_networkscore_wakeupsscore_pressurescore_energyreasons
Adapters
Each adapter must emit:
- refresh start
- refresh result
- data freshness
- cache vs live state
- failures with normalized error class
Required fields:
adapterfetch_millishealthlive_item_countcached_item_counterror_classerror_message
Telemetry
Emit:
- export attempt
- endpoint resolution
- HTTP status result
- dropped exports
- disabled state changes
Required fields:
endpointpayload_bytesmetric_countlatency_millishttp_status
Diagnostics Storage
In-memory ring buffer
Always on.
Requirements:
- default capacity: 2,000 events
- per-level counts
- O(1) append
- query by time window
- query by subsystem
- query by entity
Persisted diagnostics store
Opt-in or debug-mode only in phase 1.
Requirements:
- SQLite
- retention configurable, default 24h when enabled
- separate from history snapshots
- capped size budget
- indexed by timestamp, level, subsystem, sequence, entity_id
Log Redaction Policy
Mark the following sensitive by default:
- full command lines
- raw window titles
- raw URLs
- file paths outside the app bundle unless explicitly user-approved
- socket endpoints
- telemetry endpoints with embedded credentials
Diagnostics UI should support:
RedactedReveal onceAlways reveal in this session
In-App Diagnostics Console
Add a new top-level app tab:
Diagnostics
Primary sections
- Overview
- engine status
- current mode
- diagnostics level
- ring buffer fill
- persistence enabled/disabled
- telemetry enabled/disabled
- Live Event Stream
- newest first
- colored by severity
- filter chips
- pause/resume stream
- copy selected event JSON
- Subsystem Panels
- engine
- adapters
- attribution
- telemetry
- persistence
- Query
- search by entity, adapter, subsystem, PID, sequence
- time range filter
- severity filter
- Export
- export visible events as JSON
- export redacted diagnostics bundle
- export support bundle with snapshot + diagnostics + version metadata
UX details
- default view should be useful without debug mode
- debug-only detail rows can be hidden behind a disclosure toggle
- every event row should show:
- timestamp
- level
- subsystem
- short message
- badges for
sequence,entity,adapter
Minimum operator cards
- Engine tick p50 / p95
- Adapter last success time
- Current top error classes
- Persisted history write success/failure
- Telemetry exporter last status
- GPU sampler availability
Settings Surface
Extend settings with:
- diagnostics enabled
- diagnostics verbosity
- persist diagnostics locally
- diagnostics retention window
- include sensitive fields in exports
- enable temporary trace mode for N minutes
Trace mode must auto-expire.
Export Bundle Spec
Support one-click export of:
- app version and git revision if available
- platform and hardware summary
- current snapshot
- recent timeline
- persisted history summary
- recent diagnostics events
- capability states
- adapter health states
- benchmark/profile summary if present
Output formats:
support-bundle.json- optionally
support-bundle.zipif multiple artifacts are included later
Performance Budget
Diagnostics must not violate Aetower’s low-footprint goals.
Always-on budget
- event append: sub-100 microseconds typical
- diagnostics memory: under 4 MiB default ring buffer
- no synchronous disk writes on hot tick path
- no UI polling for diagnostics when diagnostics tab is closed
Debug-mode budget
- trace mode may increase overhead, but must be visibly marked
- trace mode should show an estimated overhead badge
Versioning And Stability
Define a diagnostics schema version:
diagnostics_schema_version
Rules:
- additive fields are allowed
- field removals require version bump
- exported bundle must include schema version
This matters because diagnostics should become comparable across releases.
Rollout Plan
Phase 1
- add
aetower-diagnosticscrate - add event schema and ring buffer
- emit engine, adapter, telemetry, persistence events
- expose diagnostics query API through FFI
- add Diagnostics tab with live event stream and filters
Phase 2
- attribution and friction explanation traces
- support bundle export
- persisted diagnostics store
- redaction controls
Phase 3
- cross-session diagnostics comparisons
- anomaly clusters
- automatic “why this changed” summaries
- diagnostics-driven regression detection
Acceptance Criteria
- A user can answer “why is this entity ranked here?” without leaving the app.
- A developer can answer “which subsystem failed?” without adding ad hoc prints.
- An operator can export a bounded diagnostics bundle after a bad run.
- Diagnostics remain bounded and do not materially regress the benchmark budget.
- Adapter failures and telemetry failures are visible, queryable, and timestamped.
- Attribution and provenance changes become inspectable over time.
Immediate Follow-Up Backlog
- Create
aetower-diagnosticscrate with ring buffer and event schema. - Add diagnostics producer hooks in
engine,adapters,telemetry, andpersistence. - Expose
queryDiagnosticsandexportDiagnosticsBundlethrough FFI. - Add
DiagnosticsView.swiftwith overview, stream, and export actions. - Add a support-bundle JSON schema and golden tests.
- Add benchmark assertions for diagnostics-on default mode.
Recommendation
Treat this as a core feature, not a debug extra.
Aetower’s product value is not only collecting observability signals. It is being able to justify, inspect, and improve its own interpretation over time. Without a proper diagnostics layer, every future observability feature will be harder to trust, harder to debug, and harder to evolve cleanly.
Aetower is a free early-alpha download for macOS 14+ (Apple silicon).
Download for macOS