Oligomers are the key suspects in protein aggregation-linked diseases, such as Alzheimer's and Type II diabetes, and most likely exert their toxicity by interacting with lipid membranes. However, the "which oligomer" question remains an obstacle in understanding the disease mechanism, as the exact identity of the toxic oligomer(s) is not yet known. Oligomers exist as a mixture of species of different sizes (i.e. as different 'n-mers') in a physiological solution, making it difficult to determine the properties of individual species. Here we demonstrate a method based on single-molecule photo-bleaching (smPB) which can provide an answer to the "which oligomer" question, at least as far as membrane affinity is concerned. We calculate the ratio of the oligomer size distribution of human Islet Amyloid Polypeptide (IAPP) in the aqueous phase and that on a coexisting artificial lipid bilayer, and this measures the relative membrane affinity of individual oligomeric species. A problem with smPB measurements is that they can be very sensitive to pre-measurement bleaching. Here we correct for pre-bleaching using a covalently linked multimeric peptide as a bleaching standard. We find that the order of membrane affinity for IAPP n-mers is trimer > dimer > tetramer ≫ monomer. Our results agree well with the average membrane affinity values of oligomeric and monomeric solutions previously measured with Fluorescence Correlation Spectroscopy. The "which oligomer" question, in the context of membrane affinity, can therefore, be solved quantitatively for any membrane-active toxic protein aggregate.