There is no passenger Boat services between Qatar and Oman. The only two way to travel to Oman is by Road in your own vehicle and by Plane.

From Dubai you can take a bus to Muscat..

Have a nice time..