Why F.relu()? – I Think Its Since You Want To Adapt The Solution To A Non Linear Solution Or Get Away From The Trivial Linear Solution