Purpose of this document is to Logging and Tracing Guidelines while developing application to avoid false alarm and do best practices.
Core Log Levels & Span Statuses
Effective observability requires distinguishing between system-level failures and business-level logic.
Log Levels
- Fatal/Panic: Use fatal or panic error strictly when system should not continue. For example- Important environment initialization or resource initialization etc.
- Error: Reserved strictly for System Failures. Use this when the application cannot perform its intended function due to external factors like a database being down, network timeouts, or infrastructure issues.
- Warn: Used for Security Issues or Unusual Behavior. These aren’t necessarily failures yet but require investigation.
- Info: Used for Business Validation and user-driven errors (e.g., an “Invalid Input” or “User Already Exists”).
- Debug/Trace: High-verbosity logs intended for Development and deep-dive troubleshooting.
Span Statuses
In distributed tracing, spans help track the flow of a request.
- Ok: Indicates success or successful business validation.
- Error: Indicates a system failure.
- Event: Used via AddEvent to attach specific business context or observability markers to a span.
The Five Golden Rules
- System Failures Only: Use the Error log level only when the system itself fails.
- Business Info: Use the Info log level for business logic outcomes (like validation failures).
- Real Errors Only: Use RecordError in spans specifically for technical failures, not business logic deviations.
- Contextual Events: Use AddEvent within spans to document business-related issues.
- Data Privacy: Never log sensitive information (PII, passwords, secrets).
Implementation Workflow
The following table illustrates how to handle logging and tracing across different architectural layers based on the workflow diagram:
| Layer | Action | Type |
|---|---|---|
| Initializer | System initialization | Log: Fatal/Panic; Span: No span event |
| Controller | Initial Request | Log: Info/Error; Span: Add Event |
| Service | Handle Logic | Log: Info (Optional); Span: Business Op |
| Repository | System Failure | Log: Error; Span: Record Error |
| Repository | Business Validation | Log: Info; Span: Add Event |
| External Service | External Failure | Log: Error; Span: Add Event |
Practical Scenarios
Here is how to apply these guidelines to common real-world scenarios:
- Invalid Login: This is a business logic outcome. Log as Info and use Add Event in the span.
- Database Error / Timeout: This is a system failure. Log as Error and use Record Error in the span.
- Unauthorized: Typically logged as Warn (security) or Info (business validation) depending on context.
Bonus
You can Download this Handbook in a diagram form to keep it handy.
