Adding a Little Randomness to Life

Stochastic Gradient Descent (SGD) is an algorithm that anyone learning machine learning or deep learning is bound to encounter. The goal of this algorithm is to find a local minimum of a function, typically the loss function.

When I first learned about it, I found it fascinating. Standard gradient descent faces computational bottlenecks; datasets during training are usually massive, and computing across all samples in every iteration is not only inefficient, but also demands extra memory.

Yet with just a slight tweak—randomly picking a single sample to compute the loss rather than using the entire dataset at once—it might be noisy and take a winding path, but it still finds the minimum. Or, by randomly sampling a small batch (mini-batch), you can achieve your goal while balancing efficiency and performance.

Lately, I’ve come to feel that life should be the same way. Injecting a bit of randomness into my life was a major catalyst for my decision to quit my job.

To be honest, the salary and benefits here were quite good. Barring any surprises, I could have comfortably stayed until retirement. But I really couldn’t imagine keeping myself locked into this job, living a predictable life commuting back and forth between work and home, where the monthly paycheck had become a kind of poison.

Of course, it’s not that I learned nothing at the company. My growth over the past few years has enabled me to view things from a much broader perspective, almost as if my mindset leveled up. One thing is certain: five years ago, I never would have considered anything outside of a purely technical role.

Although I still harbor a lot of fear about the next step in life, I think it can be described by a quote from an anime I’ve really enjoyed recently—Orb: On the Movements of the Earth (チ。-地球の運動について-): “怖くない人生など、その本質を欠く”

A life without fear lacks its very essence.

Fear stems from the unknown, but life is inherently full of unknowns. Who knows? In the future, I might start a business, dive into music, open a shop, or move to another country. There are so many choices, all shrouded in uncertainty. Deep in the mountains, you can’t see the entire landscape. All I can do right now is introduce a little randomness, trusting that gradient descent will guide me to an optimal solution.