If the usage is less you can reduce the vCPU, but I would recommend to keep the minimum Operating system recommendation to do their operation well.
You may have to look for downtime as well, most of the Operating system doesn't support Hot CPU removal
-Nithin