You know, some criminals might use your argument, start blaming their genes heritage, their social circumstances, ... etc., and arguing that they shouldn't be punished

But what about free will?

"The larger grows the island of my knowledge, the
longer stretch the shores of my ignorance."
William Blake