Baxter was born on December 16, 1844, in Port Glasgow, Scotland.[1] In 1850, his father Duncan brought the family to America, where they settled in Corning, New York. Baxter worked on a farm and studied at the Corning Academy. He was a clerk in the village post-office when the American Civil War broke out.[2]
Baxter enrolled in the 141st New York Volunteer Infantry in August 1862 and was mustered in as a private in Company E. By September 1862, he was promoted to sergeant, followed by first sergeant in April 1863 and first lieutenant in August 1863. In May 1864, he was wounded during the Battle of Resaca. On January 1, 1865, he was promoted to captain. He was mustered out with his company near Washington, D.C., in June 1865.[3] He left the service as brevet-major of volunteers.[4] He later served as adjutant and lieutenant-colonel in the National Guard of New York.[5]
Baxter returned to Corning after the War. From 1865 to 1872, he worked in the Tioga Railroad Company. In 1874, he was elected clerk of Steuben County. When he moved to Bath to work as clerk, and began studying law under Judge William Rumsey. He then attended Albany Law School, graduating in 1879. After graduating, he moved to Elmira and started practicing law there. He was an active member of and public speaker for the Republican Party.[6]