Arguments And Truth
Importance: 6 | # | rationality
The world is not made of arguments. Think not "which of whese arguments, for these two opposing sides, is more compelling? And how reliable is compellingness?" Think instead of the objects the arguments discuss, and let the arguments guide your thoughts about them.
If an argument misleads you, think not "I have learned that similarly compelling arguments are often wrong." Think instead of which step within it was wrong, and adjust your thoughts so that they are not so easily guided down wrong paths.
If you think you have proofs of both A and ¬A, think not "which proof is more persuasive?". Instead, observe that you are mistaken. Either the two statements are not in fact opposed, or one supposed-proof contains a flaw. Don't weigh proofs; seek flaws. So too with arguments.
Another way of breaking loose of 'arguments': Any time somebody manages to persuade you of something via much hard work, do not neglect to remember that you would, if you had been smarter, probably have been persuadable by the empty string.
Or to put it another way: You shouldn't need to read Nanosystems to get that molecular nanotechnology, the 'weaker' arguments in Engines of Creation should suffice because they're still right - but Eric Drexler is who he is, because he was persuadable by the empty string.
"Your strength as a rationalist is the extent to which it takes more Charisma to persuade you of false things, and less Charisma to persuade you of true things," says Planecrash. Rephrasing yet again:
Your strength as a rationalist is the degree to which it takes "very strong and persuasive argumentation" to convince you of false things, and "weak, unpersuasive-sounding argumentation" to convince you of true things; ideally, in the latter case, the empty string should suffice.