Several studies have suggested that silver nanoparticles (AgNPs) have the potential to treat human cancers, including leukemia. However, the detailed cellular mechanisms for AgNPs to inhibit the growth of leukemic cells and their efficacy on clinical isolates of leukemic patients are not elucidated. In this study, the cellular uptake and cytotoxic mechanism of AgNPs in chronic myeloid leukemia (CML) cells were investigated. AgNPs were synthesized with a modified polyol method, which were stable under cell culture conditions with fetal bovine serum (FBS). AgNPs were demonstrated to be able to enter K562 cells (a CML cell line) in a dose-dependent manner and locate in endosomes. Reactive oxygen species (ROS) could be generated upon AgNPs exposure and cause cytotoxicity and apoptosis. It was also found that AgNPs treatment inhibited the viability of cells from CML patients (n = 4). The cell cycle status and several critical regulators were altered upon AgNPs treatment as well. All these cellular and molecular alterations caused by AgNPs exposure could be reversed by the addition of Vitamin C (an antioxidant). These results suggested that proper usage of AgNPs would be of great significance for CML treatment in future.