When I first set our system up with multiple workers I increase the vCPU count to 16 and the vRAM to 80 GB. I've since backed off some. I really need to go back and see what it really needs and pare this back some more.
With this setup I'm averaging 1 publish per second (86,000 per day). I needed this for a bulk re-publish of over a million objects. 