I will start with a power sentence. ”Shit in shit out”. In the machine learning context this means that we are doing things a little bit ?wrong. We start with shit from the inside. From randomly initialized weights. Then by a lot of noise cancellations and filters we get something that looks like the target. But it is often shit out.
I suspect there exist improvements. For instance. I think you can calculate with the truth. That is. Exact solutions to linear systems of equations or single equations. Then you have used truth values.
I don’t know what the start value for the weights should be. That is. The arguments A,b to np.linalg.solve(A,b). But from eliminations of options I think it should come from the target and input values. As they are truth values.
So the idea is to combine exact solution equations like Ax = b in the weights as objects. Then puzzle those truths into a bigger picture.