Gypsum or calcium sulfate (CS) or plaster of Paris (POP) is considered as a fast degradable material that usually resorbs before the bone defect area is completely filled by new bone. In this study, the incorporation of CaCO3 hydrogel into POP in different compositions was proposed to enhance the bone biological activity of POP and to decrease its degradability. The mechanical and degradation properties of the various materials were characterized by in vitro analysis. Subsequently, the materials were inserted into cylindrically sized bone defects as created into the femoral condyle of rats and left in situ for 1, 4, and 8 weeks. Histological analysis of the retrieved specimens indicated that the addition of CaCO3 hydrogel into POP increased bone formation, angiogenesis and collagen density and resulted into faster bone formation and maturation. It was also confirmed that the degradation rate of the POP decreased by the addition of CaCO3 hydrogel. The in vivo findings did corroborate with the in vitro analysis. In conclusion, the incorporation of CaCO3 hydrogel provides a promising technology to improve the properties of POP, the oldest biomaterial used for bone grafting.