Design Container Log Collection · System Design Interview

Design Container Log CollectionDesign how the output that containers write to their standard streams is captured on each host and made searchable, including for containers that no longer exist.What it has to do

Everything a container writes is captured, in order, per container

Logs can be searched by service and time, after the container is gone

A container that logs faster than the pipeline can carry does not stop the container

Scale

Containers producing logs: 100,000

Log volume per day: 40 TB

Out of scope

Alerting on log content

Metrics and tracing

Parsing every possible log format

Practise this challengeAll system design challenges

LetsDiscuss

Loading your session…