I think independent invention is really key to the whole issue. If a particular problem is solved in a similar way by multiple independent parties, then the solution on its face should be unpatentable due to obviousness. Independent invention should always be permissible evidence in overturning any extant patents.
The whole idea of a patent is "but for my ingenuity and investment you never could have solved this technological problem. Therefore you must pay me to use my solution."
If it is not the case that nobody could have solved the problem but the patent holder, then the patent is invalid.
That's a very good point, but recall that the standard is the hypothetical person who is presumed to have known the relevant art at the time of the invention. From KSR, we know that it's this person who's the baseline for determining which, if any, inferences or creative steps are obvious.
If a person NOT falling within this definition were to independently arrive at an invention and could have arrived at the same invention prior to the issuance of that invention's patent, then there's an issue. KSR states: "a person of ordinary skill in the art is also a person of ordinary creativity, not an automaton." It follows that a hyper-creative, one-of-a-kind person who proves that he or she independently arrived at the invention would not be this hypothetical person. Although obvious to a genius in the art, it was non-obvious to a person having ordinary skill in the art. That's where I disagree with your reasoning.
EDIT: This is not to say that the genius in the art should be out-of-luck. He or she should be entitled to continue practicing the invention, but the patent itself shouldn't be invalidated. This should be evaluated on a case-by-case basis and treated as an exception to the general rule.
I don't think this provision will work. In practice software developers would then be required to keep the equivalent of lab notebooks and write down everything they do so if there is a lawsuit it can be proved that the invention is independent. While developers are already forbidden by most companies from looking at patents so as to prevent claims of willful infringement I can see legal departments forbidding software from reading papers and using much of the internet so that in case of a lawsuit it can be proved that a developer truly arrived at a solution independently.
Now the novelty claim is a somewhat separate issue. I don't have a good solution to this and there is a lot of ambiguity that arrises here.
It seems to me limiting the duration of a patent is the best solution. There is some precedent for this with the adjustment of how time for pharma patents are measured.
The whole idea of a patent is "but for my ingenuity and investment you never could have solved this technological problem. Therefore you must pay me to use my solution."
If it is not the case that nobody could have solved the problem but the patent holder, then the patent is invalid.