Of course your work matters but it is not the only thing that matters. To write a heart touching poem is work.
But making passionate love is work, but it matters.
To hug my wife and children is not work but it matters.
Our compassion, empathy and our humanity are not working matter.
If you honestly believe that only your work matter, I would feel sorry for you.
All the abstract feelings that have nothing to do with your work that matter a lot. I can see under no definition emotion, and work are correlated