The antimicrobial activities of silver-loaded zirconium phosphate (JDG) and silver-loaded phosphate glasses (ZZB) against Escherichia coli were studied. Although the silver content in JDG was higher than that in ZZB, ZZB suspensions showed better antimicrobial property than JDG suspensions, especially at low concentrations. The antimicrobial activity was analyzed using minimum inhibitory concentrations, bacterial inhibition ring tests, and detection of silver ions in the suspensions. Furthermore, the amounts of silver ions in suspensions with/without bacterial cells were analyzed. Results revealed that only a portion of released silver ions could be adsorbed by E. coli cells, which are critical to cell death. The damaged microstructures of E. coli cells observed by transmission electron microscopy may further prove that the adsorbed silver ions play an important role in the antimicrobial process.