First, you select a difficulty. Then when the hats have switched around, you pick one that you think the pearl is under.
Just a silly little game I cooked up. I couldn't get the pearl on hard mode!