James Carroll was born in England in 1854. He moved to Canada in 1869. Five years later he decided to join the United States Army, in which he remained for the rest of his life. While still a sergeant he graduated with a degree in medicine from the University of Maryland in 1891, after which he decided to study bacteriology at Johns Hopkins. He was eventually hired as Professor of Pathology and Bacteriology at George Washington University. His colleague at that institution was Walter Reed, with whom he travelled to Havana in 1900 to investigate the cause of yellow fever. As part of his studies, he volunteered to submit to the bite of an infected mosquito, from which he contracted disease severe enough to almost kill him. Although he survived, the damage to his heart was significant and he succumbed to heart failure in 1907.
Carroll became Curator of the Army Medical Museum in Washington, D. C. in 1903, which he managed until his death. In 1906, he had several meetings with William MacCallum (physician at Johns Hopkins Hospital) and Maude Abbott (Curator of the McGill University Medical Museum) about the possibility of founding an association of medical museums. They soon agreed on the conditions of such an association and sent letters of invitation to directors of the leading museums throughout the world. The first official meeting of the International Association of Medical Museums was held in Washington, D. C. in 1907. James Carroll was elected its first President; he died only five months later.