There’s almost never a really good reason to care about what someone else thinks. And the less you know a person, the less reason there is to care.
That secret you’re keeping - are you keeping it for you, or for them? The thing you’re hiding - are you protecting them, or yourself? What are you so worried about?