Logging infrastructure for EPICS-based control systems using Loki and Promtail

To support multiple scientific facilities at SLAC, a modern logging system capable of high message throughput and easy record filtering is required. A new logging system deployed at SLAC meets these requirements by blending an existing EPICS logging server with open-source technologies for log storage and retrieval. To save log data, Loki was chosen as the data storage technology. Promtail was selected to push existing log files generated by various EPICS clients into Loki. To encourage increased accelerator operator engagement, multiple interfaces are provided for interacting with logs. Grafana dashboards offer a user-friendly way to build displays with minimal code, while a custom PyQt-based user interface displays the results of direct queries to Loki in a table-based display. A wrapper script around LogCLI called qlog provides a command-line interface for interacting with the logs. The rationale behind these decisions and their integration into our controls infrastructure will be considered.