Kalan's Blog

Kalan 頭像照片,在淡水拍攝,淺藍背景

四零二曜日電子報上線啦!訂閱訂起來

Software Engineer / Taiwanese / Life in Fukuoka
This blog supports RSS feed (all content), you can click RSS icon or setup through third-party service. If there are special styles such as code syntax in the technical article, it is still recommended to browse to the original website for the best experience.

Current Theme light

我會把一些不成文的筆記或是最近的生活雜感放在短筆記,如果有興趣的話可以來看看唷!

Please notice that currenly most of posts are translated by AI automatically and might contain lots of confusion. I'll gradually translate the post ASAP

How to collect and centralize logs in golang app

This article is a summary after reading How to collect, standardize, and centralize Golang logs | Datadog.

There are several points to note when using logs:

  1. Pass log as a parameter and pass it when needed.
  2. Encapsulate using context and retrieve it when needed.
  3. Wrap the logger as a package and expose a variable for other packages to use.

Obviously, the first approach quickly encounters a problem where the parameters become cumbersome. For example:

func (p *Post) CreatePost(name string, content string, logger *log.Logger) {
  ///
  if err != nil {
    logger.Fatal("error!")
  }
}

It's quite strange that the logic of CreatePost requires passing a logger as a parameter. It is easy to end up with messy parameters and possibly a situation where the logger is passed through multiple layers. The biggest advantage is that the dependency is transparent, and through the function, it is clear that a logger will be used, making it easy to mock the logger for testing.

We can adopt the second approach, explicitly pass the context in main, and store the logger inside it. Retrieve the logger using log.Logger.FromContext(ctx). Currently, I haven't thought of any significant drawbacks.

const (
  ContextKeyLogger = "logger"
)
func main() {

  ctx := context.Background()
  loggerCtx := context.WithValue(ctx, ContextKeyLogger, log.Logger)
  
}

The third approach is the current practice, which is to wrap the logger as a package and expose a variable for other packages to use. The biggest advantage is convenience, as it can be used immediately. The drawback is that it is not easy to mock the logger when testing because it is not parameterized. (Perhaps there are other good solutions?)

package logging

var Logger *log.Logger

func init() {
  Logger = &log.Logger{
    // your setting
  }
}

Next, let's look at the recommendations in this article:

  1. It is recommended to use logrus. logrus is a very useful logging package and perfectly supports the native log, making it easy to switch without pain. With the hook mechanism, you can easily integrate with third-party reporting platforms (like datadog, sentry, etc.).
  2. Use JSON as the log file format: JSON is easy to parse, supported by major languages, and easy to analyze for third-party platforms. It is highly recommended to use it. In logrus, you can directly set the output format with SetFormatter(&logrus.JSONFormatter{}).
  3. Use a unified interface.
  4. Avoid calling the logger in goroutines. Besides considering concurrent issues, the internal implementation of the logger may also involve goroutines, making the entire mechanism difficult to control.
  5. Store logs as local files, even if you may use other platforms. This prevents the loss of certain records due to network issues and ensures that log files can always be found.
  6. If using a cluster architecture, you may need to centralize logs on a server using the syslog method.
  7. If using container services like Docker, you may need to listen to Docker's STDOUT.

Others

log.Fatal() and log.Panic(), where Fatal secretly calls os.Exit(1), and Panic secretly calls panic. Be extra careful when using them, although many people may not have noticed it in the documentation.

Prev

Golang Notes — Type Assertion

Next

Safer Request Headers - Fetch Metadata Request Headers

If you found this article helpful, please consider buy me a drink ☕️ It'll make my ordinary day shine✨

Buy me a coffee