When do you let your child go into the bathroom alone?

My son is currently 7y. At what age is it fine to let my son go to the public restroom by himself. I walk with him and still make him go to the lady’s side and wait by the door to stall his in. If my husband accompanies him, they go to the men’s side, and the husband waits for the same way. But till what age is it considered that my son can goes to the lady’s side and he has to go by himself in the men’s side when my husband is not around. I’m a bit paranoid about current situations happening. I always have that fear something bad is about to happen to my kids when we are out in public. Sometimes I just get a bad mood having my kids out in public and worrying someone is trying to get them. I’m always aware of who around, or if my kids lose sight of my kids(I have a 3yr old also) for a moment, like hiding behind a rack I have small panics. That’s the reason for my question. Thank you for your time.