A 2D/2D BiPO4/g-C3N4-B nano-sheet heterojunction photocatalyst was synthesized via a simple coprecipitation method at room temperature using glacial acetic acid as solvent, which showed excellent activity toward the degradation of rhodamine B (RhB). The heterojunction showed much higher efficiency of separation and transfer of photogenerated carriers compared to that of its constituents. Moreover, the spectral response range of BiPO4 was effectively broadened after the combination of g-C3N4-B and BiPO4. Consequently, a 97.3% degradation of RhB within 25 min by BiPO4/g-C3N4-B heterojunction photocatalyst under visible light irradiation was observed. The difference in work functions of BiPO4 and g-C3N4-B was evident from UPS characterization, which led to the bending of the energy band and the establishment of an internal electric field at the interface of the heterojunction. Therefore, the synthesized direct Z-type BiPO4/g-C3N4-B heterojunction enhanced the oxidation-reduction ability by promoting the effective separation of photogenerated carriers.