Daily Brief Pipeline
Deliverable AGT-001. Structure per cc02-standards/SYSTEM-STANDARD.md. Every component traces to a design basis (NEED/REQ) and to an acceptance criterion.
1. Purpose and Design Basis
1.1 Scope and Objective
The Daily Brief Pipeline is an automated data processing system that aggregates security logs, regional intelligence, and protective alerts to compile a structured, prioritized briefing document for executive protection detail planning and management.
1.2 Needs
- Need: The Team Leader (RF) requires a consolidated daily intelligence brief before shift start to eliminate manual search bottlenecks.
- Source: RF (CEO/TL) operational bottleneck concern (PROJECT.md).
- Context: RF currently spends up to three hours daily checking individual feeds manually before planning operations.
1.3 Requirements
- REQ-AGT-001-01: Under the condition that the system scheduler triggers the pipeline, the Daily Brief Pipeline shall aggregate security logs and intelligence feeds on an hourly schedule.
- Verification: Test
- REQ-AGT-001-02: When a threat alert with a severity level of Critical (L×I 21-25) is parsed in the alert stream, the Daily Brief Pipeline shall elevate the corresponding alert to the top section of the generated brief.
- Verification: Demonstration
2. Architecture As Built
The pipeline is a deterministic, dependency-light compiler (Python 3.12 standard library plus PyYAML). It consumes the ART-threat-alert-stream (the vsi-feed match-log JSONL) for a coverage window and renders the intelligence surface of an EP-016-shaped daily brief (the ART-daily-brief). There is no network, no LLM, and no external API on this path; live RSS ingestion and LLM SITREP enrichment are deferred adapters. Compile time is an explicit argument, so a given stream and coverage window always produce the same brief.
Data flow: load stream, bound to the coverage window, cluster multi-source reports of one event, rank on the cc02 Likelihood x Impact key (Critical first), render, write the dated brief.
| Component | Module | Responsibility | Design Basis |
|---|---|---|---|
| Stream loader | engine/daily_brief/aggregate.py (load_stream, in_window) | Parse the match-log JSONL and bound it to the coverage window | REQ-AGT-001-01 |
| Clusterer | engine/daily_brief/aggregate.py (cluster) | Collapse multi-source reports of one event into a single incident | Brief integrity (page once per event) |
| Ranker | engine/daily_brief/rank.py | Map place tier and threat weight to cc02 L x I 1-25; float Critical to the top | REQ-AGT-001-02 |
| Renderer | engine/daily_brief/render.py | Emit the EP-016-shaped markdown brief | provides ART-daily-brief |
| Orchestrator | engine/daily_brief/compile.py | Deterministic load to render with an explicit as-of | REQ-AGT-001-01, REQ-AGT-001-02 |
| CLI | engine/daily_brief/cli.py (daily-brief) | Wall-clock default, context load, file output | Run and handover |
Boundary: the stream producer (the vsi-feed hot path, boundary defined in SYS-008) emits the ART-threat-alert-stream; this system consumes it and does not fetch feeds or run the matcher.
3. Components and Bill of Materials
| Component | Spec | Vendor / Link | Price | Tier | Status |
|---|---|---|---|---|---|
| Compiler package | Python 3.12 standard library, engine/daily_brief/ (six modules) | Internal | $0.00 | - | BUILT |
| YAML parser | PyYAML >= 6.0 (context loading only) | PyPI | $0.00 | - | BUILT |
| Console script | daily-brief entry point (pyproject.toml) | Internal | $0.00 | - | BUILT |
| Test suite | pytest cases plus a golden-brief fixture, engine/tests/ | Internal | $0.00 | - | BUILT |
| Scheduler | systemd timer or cron (hourly trigger) | OS | $0.00 | - | DEFERRED |
| RSS ingestion adapter | Stream producer (vsi-feed hot path / SYS-008) | Internal | $0.00 | - | DEFERRED |
| LLM SITREP enrichment | Off-hot-path claude -p digest (vsi-feed daily_brief.py ancestor) | Internal | $0.00 | - | DEFERRED |
4. Build and Deployment
- Install dependencies with uv (uv sync); the project install registers the daily-brief console script.
- Point the compiler at a threat-alert stream (match-log JSONL) and, optionally, a region context YAML: uv run daily-brief —region nyc —stream
[—context ]. - The brief is written to briefs/
/ -daily-brief.md, or to —out, or to stdout with —stdout. - For scheduled operation (deferred), wrap the command in a systemd timer or cron entry at the required cadence. The compiler is deterministic for a fixed —as-of, so runs are reproducible and testable.
5. Hardening Baseline
- Restrict directory execution permissions to the runtime service owner.
- Store sensitive configuration variables inside container environment files only.
6. Integration Points
| Interface | Type | Direction | Contract |
|---|---|---|---|
| ART-threat-alert-stream | File (JSONL) | Consumed | One match-log record per line: ts, severity, alerting, tier, weight, threat_terms, place_terms, source, title, link, published |
| Region context | File (YAML) | Consumed (optional) | weather, first_last_light, airspace, ao_activity, disruptions, prev_posture |
| ART-daily-brief | File (Markdown) | Produced | EP-016-shaped daily detail brief with the intelligence sections filled |
| Downstream dissemination | File (Markdown) | Produced for | AGT-002 and AGT-003 consume the ART-daily-brief |
7. Acceptance Tests
| Criterion | Method | Result | Evidence |
|---|---|---|---|
| REQ-AGT-001-01 | Test | PARTIAL | Windowed aggregation is Test-verified (engine/tests/test_daily_brief_cli.py::test_cli_window_aggregation_bounds_items and test_daily_brief_aggregate.py). The hourly scheduler is a deferred operations adapter and is the open gap against the “on an hourly schedule” clause. |
| REQ-AGT-001-02 | Demonstration | PASS | engine/tests/test_daily_brief_rank.py::test_critical_alert_floats_to_top and the golden render check test_daily_brief_render.py::test_critical_item_renders_above_lower_severity. |
8. Run and Maintenance Handover
- Run: uv run daily-brief —region
—stream <match-log.jsonl> [—context <ctx.yaml>] [—as-of ] [—window-hours N]. The compiler prints an incident count and posture to stderr and writes the brief to briefs/ / -daily-brief.md. - Output: generated briefs under briefs/ are runtime artifacts and are gitignored; the committed golden fixture (engine/tests/fixtures/daily_brief/) is the render contract.
- Tests: uv run pytest exercises aggregation, ranking, rendering, and the CLI; the render test is a byte-for-byte golden comparison, so any intended output change updates the golden alongside the code.
- Ownership: Systems Engineer role. Deferred adapters (scheduler, RSS ingestion, LLM enrichment, Telegram dissemination via AGT-002/003) are tracked as separate increments.
END OF SYSTEM
Model wiring
Generated from cell frontmatter at publish time.