Decided to give VPS a try, I picked VPSLink.
I asked their support about differences between Xen and OpenVS they pointed me to this. Decided to give OpenVS a try.
Brought a new domain name at GoDaddy at 10:16pm, paid up VPSLink-2 (5G storage and 128M RAM) for 1 month at $14.95 at 10.24pm, changed DNS information and created 'A' record of my domain name following instruction in Wiki here. I logged into my server using the new domain name at 10:29pm.
That was all very quick and I don't have to remember IP address now.