(h/t Julia Galef for making me aware of the photo via her excellent TAM talk)
I haven't made "noticing and responding appropriately to confusion" a special explicit focus of my training as a rationalist so far, so I expect there are several things I'm doing wrong that will become obvious quickly upon closer inspection. But I think I just realized a huge mistake I've been making anyway.
When I am confused, I focus on the thing that I am confused about. I know it means I believe something false, and I want to find out what that thing is. My automatic procedure for doing so is, "Investigate the object of confusion for clues to my false belief, then search nearby objects for clues."
In Zen terms, I'm looking at the pointing finger instead of at the moon.
Check out this image. Look at it for a while, if you've never seen it before, before reading on.
I noticed the rock immediately, and I flagged it as confusing. My automatic interpretation was, "Someone threw a rock at that raccoon, and it's about to get hit, oh no!" That felt like an awfully strange, though, so I took a really close look at the rock. I started looking at other elements of the picture. I couldn't find anything strange about it (although I do see a strange thing now that I know what's up with this picture), so I looked around within the picture for other clues. I noticed that the raccoon on the left didn't have the dark markings around its eyes I'm used to raccoons having. I noticed the raccoons are in a slightly unusual environment for the species, since I think of them as preferring to be hidden and in low places.
I did not ask myself, "What do I believe about this picture that makes me confused about the rock? What are other possible interpretations of this picture? If it's not the case that that raccoon is about to be hit by a rock, what else might be going on?"
Upon noticing confusion, I've been going through these mental motions: "Don't ignore or rationalize the confusion. Pay attention to it. Be curious about what false things you believe." It's like I've been putting off examining my own beliefs for errors by examining my observations.
Next time, I'll try this: "If I notice that I am confused, then I will state what I believe about the situation that forbids the confusing thing, and then generate alternative hypotheses. Only then will I examine the situation closely to see which hypothesis best fits my observations."
Correct movement: Notice which of my beliefs forbids the confusing thing.
Incorrect movement: Look really hard at the confusing thing.
I suppose I did manage to look at the moon, though, when my response to having noticed confusion about this picture failed to lead me to the correct answer. "I'm confused about noticing confusion. Am I noticing confusion wrong? How do I actually notice confusion? How else should I maybe be doing it?"
ETA: Apparently it was not obvious to some that I intend for you to work out for yourself what's actually going on with the rock. Here's a hint: Suppose I'd swapped "clouds" for "moon".