Add a Touch of Randomness to Life

Stochastic Gradient Descent (SGD) is an algorithm that everyone venturing into machine learning or deep learning will inevitably encounter. The goal of this algorithm is to find the local minimum of a function, which is often the loss function.

When I first encountered it, I found it to be quite magical. The usual gradient descent method has a problem with computational workload since training typically involves a large number of samples. Calculating the gradient over all samples in every iteration is not only inefficient but also requires additional memory management.

However, with a slight modification, instead of calculating the gradient over all samples at once, if we randomly pick a single sample to calculate the loss, the precision might not be as high, but it can still find the minimum; or even by randomly picking a few samples to form a mini-batch, it's possible to achieve our goal while balancing performance.

Lately, I've been feeling that life should also incorporate a bit of randomness, which has been a significant factor in my decision to resign.

To be honest, the salary and benefits here are quite good, and without any unforeseen circumstances, I could stay until retirement. But I can't imagine continuing to lock myself in work, living a life that only moves between two points, with my monthly salary becoming a poison.

Of course, it's not that I haven't learned anything in the company. The growth over these years has allowed me to view things from a broader perspective, feeling as if my cognition has leveled up. I can confidently say that the me from five years ago would never have thought beyond a technical role.

Although there's a lot of fear regarding the next step in life, I like to describe it using a line from a recent favorite anime, Chi's Sweet Home: "A life without fear lacks its essence."

A life without fear is lacking its essence.

Fear stems from the unknown, but life is inherently filled with uncertainties. The future might involve starting a business, playing music, opening a store, or even moving to another country. There are many possibilities filled with unknowns. Standing in the thick of it, I can't see the big picture, but what I can do is add a bit of randomness, hoping that gradient descent will lead me to the best solution.