Any process that modulates the frequency, rate or extent of hemocyte proliferation. Hemocytes are blood cells associated with a hemocoel (the cavity containing most of the major organs of the arthropod body) which are involved in defense and clotting of hemolymph, but not involved in transport of oxygen. An example of this is found in Drosophila melanogaster. [ GOC:mtg_sensu GOC:bf ]