Engineering Context¶
Purpose¶
This page explains how ETIS treats operations and monitoring.
Operations is where the system, users, environment, support model, and evidence discipline are tested by reality.
Monitoring is the discipline of watching for signals that the deployed scope is behaving as approved or drifting toward risk.
Operations is controlled exposure¶
ES-113 does not grant permission to expand release scope.
If the approved deployment is internal review only, monitoring must enforce that boundary. If the approved deployment uses sample data only, monitoring must watch for real or sensitive data. If AI is disabled, monitoring must confirm it remains disabled. If known defects exist, monitoring must track whether they appear during operation.
Monitoring signals¶
Monitoring signals may be technical, procedural, or human.
Examples include:
- system availability;
- user access;
- data entered;
- evidence events created;
- denied access attempts;
- guardrail violations;
- AI feature status;
- defect recurrence;
- user confusion;
- feedback patterns;
- stop criteria triggers.
Not every project needs heavy telemetry. Every project needs appropriate visibility.
Operational events¶
Not every event is an incident.
Operational events may include review session completed, sample data reset, approved user added, support question answered, monitoring check completed, known defect observed, stop criterion reviewed, or user feedback received.
Recording operational events makes operations reviewable.
Incidents¶
An incident is an operational event that threatens scope, trust, safety, security, evidence integrity, or approved conditions.
Examples include unauthorized access succeeds, real data appears, evidence event missing, AI feature activates unexpectedly, reviewer uses system for real work, or support owner unavailable during an issue.
Incidents must be recorded and acted on.
Guardrail monitoring¶
Guardrails should continue to be monitored after deployment.
For LMU/COICP-style systems, this may include evidence event creation, access-control enforcement, AI-disabled status, sample-data-only rule, emergency workflow exclusion, and retention assumptions not being treated as finalized.
Guardrails are not one-time checks.
Feedback as evidence¶
User feedback is operational evidence.
Feedback should be captured with enough structure to support later learning:
- source;
- date;
- context;
- issue or observation;
- severity;
- related capability;
- action taken;
- follow-up needed.
Do not lose feedback in informal chat.
Operational decision points¶
Monitoring may support several decisions:
- continue;
- continue with condition;
- pause;
- stop;
- restrict scope;
- return to testing;
- return to release readiness;
- proceed to ES-114 learning and stewardship.
The decision should be recorded.
Do not wait until the end of operations to record monitoring evidence. Evidence captured later is weaker and less trustworthy.
Operations reveals whether the engineering story survives contact with real use.