There’s always an opportunity cost. Are you going to work or spend time with family? Are you going to play a game or spend time with friends? Are you going to be productive by somebody else’s measure, or your own? What do you lose, what do you gain?
Certainly, you can’t get the time back from working rather than spending quality time with your family, yet granted you can’t sustain a family or yourself if you abdicate from work. You can’t invest energy into one thing without sacrificing the other.
That said, you can grow your own wealth, and there is such a solution as minimizing and maximizing for returns. There is only so much time in life, but that doesn’t mean you have to split it equally. Take Tim Ferriss and his four work week, for example, you do not have to spend every possible second fighting to get ahead, but rather you are able to spend your seconds as wisely and effectively as possible.
There may be an opportunity cost, but it doesn’t have to rule your life, and at all times the cost is subjective. The opportunity cost is what you set it as.