How to collect and centralize logs in golang app

This article summarizes insights drawn from How to collect, standardize, and centralize Golang logs | Datadog.

When using logs, there are several key points to keep in mind:

1. Treat logs as parameters to be passed; whenever you need to use a log, pass it as an argument.
2. Standardize by encapsulating logs in a context, and retrieve them from the context when needed.
3. Wrap the logger in a package and expose a variable for use by other packages.

Clearly, the first approach will soon lead to issues, as your parameters will become quite convoluted. For example:

func (p *Post) CreatePost(name string, content string, logger *log.Logger) {
  ///
  if err != nil {
    logger.Fatal("error!")
  }
}

A CreatePost function logic that requires passing a logger can easily clutter your parameters and may lead to situations where the logger is passed through multiple layers. The main benefit of this approach is that the dependency is transparent; it’s clear from the function signature that a logger will be used, and testing becomes straightforward as you can easily mock the logger.

Alternatively, we can adopt the second approach, explicitly passing a context in main, with the logger embedded within it. When needed, we can retrieve it using log.Logger.FromContext(ctx). So far, I haven’t identified any significant drawbacks.

const (
  ContextKeyLogger = "logger"
)
func main() {

  ctx := context.Background()
  loggerCtx := context.WithValue(ctx, ContextKeyLogger, log.Logger)
  
}

The third approach, which I currently use, involves wrapping the logger into a package and exposing a variable for other packages to utilize. The primary advantage here is convenience and plug-and-play functionality. However, the downside is that testing becomes more challenging since the logger is not parameterized and is difficult to mock. (Perhaps there are other good solutions?)

package logging

var Logger *log.Logger

func init() {
  Logger = &log.Logger{
    // your setting
  }
}

Now, let’s explore the recommendations from this article:

1. It is recommended to use logrus. Logrus is a highly effective logging package that fully supports the native log, allowing for a seamless transition. With its hook mechanism, you can easily integrate third-party reporting platforms (like Datadog, Sentry, etc.).
2. Use JSON as the log file format: JSON is easy to parse and widely supported across major programming languages, making it highly analyzable for third-party platforms. It’s highly recommended; in logrus, you can set the output format directly with SetFormatter(&logrus.JSONFormatter{}).
3. Utilize a unified interface.
4. Avoid calling the logger within goroutines. Beyond concurrency concerns, the internal implementation of the logger may also invoke goroutines, complicating the overall mechanism.
5. Store logs as local files, even if you are using other platforms. This ensures that network issues do not lead to the loss of records, and you can always locate log files.
6. In a clustered architecture, you may need to centralize logs on a single logging server using a syslog approach.
7. If using container services like Docker, you may need to listen to Docker’s STDOUT.

Additional Notes

log.Fatal() and log.Panic() have distinct behaviors; Fatal will secretly call os.Exit(1). I know the documentation mentions this, but many people might overlook it, while Panic secretly calls panic, so be particularly cautious when using them.