It's used metaphorically to express a difficult search among many possibilities. Her journal had lots of ideas, finding the best one was a needle in a haystack.
Pro Tip 2/3
Hyperbole
The phrase often exaggerates difficulty for effect, but doesn't always mean impossible. With all these keys, locating the right one is like finding a needle in a haystack.
Pro Tip 3/3
Common Usage
Often used in everyday conversation, suggesting a shared understanding of the difficulty. Asking Alex to arrive on time is like looking for a needle in a haystack.