As per my knowledge, Number of people going with main sponsor (mehram) to Umrah or Hajj, Should return with him. Unless god forbids if anyone is lost & you have police report.

They will not allow you to exit either by road or by air without family members.

There is special registration of all members after immigration for those entering on Umrah or Hajj Visa. And this is checked before immigration when you exit.

I would suggest you to send your family first to stay then, on the way you can pick and go for umrah by road and come back.