If you have a valid visa and passport and no criminal records, no one can stop you from leaving the country (it is written in our constitution - right to travel - last time I check). If you allow them to stop you from traveling out of the country, its your fault.