Description:
- Allows you to store, search, analyze, monitor, and alert on log data and events from Google Cloud and AWS.
- Logging includes storage for logs, a user interface called Logs Explorer, and an API to manage logs programmatically.
- can also be used in realtime using streaming like dataflow
- Types of logs:
- Platform log: Gcloud services
- Component log: like platform log but generated by cloud-provided software components like GCE instances log
- Security log: who did what where and when
- User-written log: written by custom applications and services by API or client sdk
- GKE, Run, Functions: log written to stdout and stderr are saved automatically
- GCE: Install Ops Agent on VM
- Multi/hybrid cloud log
1. Log explorer:
- Query builder:
- Make uses of the LHS log fields
2. Log analytics:
- Powered by BigQuery
- Enable analytics for the GCP Log Bucket
- Different from export log to bq
- Log data in BigQuery is managed by Cloud Logging.
- BigQuery ingestion and storage costs are included in your Logging costs.
- Data residency and lifecycle are managed by Cloud Logging.
- Cant use join or other operations with other datasets in bq
- Cant be downgrade to non-analytics bucket
3. Log dashboard:
4. Log-based metrics:
- Derive data from content of log entries
- 2 types:
- System-defined log-based metrics:
- Calculated only from logs that have been ingested by Cloud Logging
- Can be: counter, distribution
- At project level
- User-defined log-based metrics:
- Created by admin to track things
- Can be: counter, distribution and boolean
- can be project or bucket level
- Labels are used for grouping and filtering with filed name
- Allows for one time-series but multiple lines for each label
- Can use extraction regular expression to extracts label value from field value
- leave empty for the entire content of the field
5. Log router:
- Can route from one project to a bucket of another project
- GCP Log Sink
- Log routing:
- All types of logs is contralized in Cloud Logging API
- Then send to 3 GCP Log Sink . Each contains the inclusion and exclusion filters
- _reqrired log sink
- _Default log sink
- User-defined log sink
- Then send to log buckets
6. Log storage:
- Each project, logging automatically create
- required log bucket and required log sink
- Holds admin activity, system event and access transparency logs
- Retention of 400 days, non-changeable
- Cant be deleted or modified
- default log bucket and default log sink
- Holds all other logs excepts in required logs
- Retention of 30 days by default, up to 365 days
- Cant be deleted but can be disabled
7. Integration: