Question about season 3 episode 3

In season 3, episode 5, it seems like the sparrows suddenly learn that the old man is Harlen, but it's not explained in the episode. Was this just a random plot hole? Edit: mislabeled the episode