Born near Mount Sterling, Kentucky, Williams attended the common schools and graduated from Miami University in Oxford, Ohio, in 1839. He studied law, was admitted to the bar in 1840, and commenced practice in Paris, Kentucky. He served in the Mexican–American War, first as a captain of an independent company attached to the 6th U.S. Infantry, and afterward as a colonel of the Fourth Regiment of the Kentucky Volunteers. He received the nickname "Cerro Gordo Williams" for his gallantry at that battle.
With the outbreak of hostilities, Williams travelled to Prestonburg in early 1861 and was commissioned colonel of the 5th Kentucky Infantry. He served initially in the Eastern Theater, initially under Humphrey Marshall in southwestern Virginia. He participated in Marshall's ill-fated invasion of eastern Kentucky in 1862; and fought for Marshall at the victory at the Battle of Princeton Court House. Marshall resigned soon thereafter and was sent to eastern Tennessee. Williams' brigade then spent a short period of time fighting for William W. Loring in the Kanawha Valley Campaign of 1862, skirmishing at Fayetteville and reaching Charleston. Loring was then transferred to the Mississippi, replaced by John Echols. He was promoted to brigadier general in late 1862 and assigned temporary command of the Department of Southwestern Virginia, replacing Echols. He was replaced by General Samuel Jones.
Williams returned home following the war and went on to engage in agricultural pursuits, with his residence in Winchester, Kentucky.
He again became a member of the State House in 1873 and 1875. He ran unsuccessfully for Governor of Kentucky in 1875, and was a presidential elector on the Democratic ticket in 1876. He was elected to the United States Senate in 1879 and served from March 4, 1879, to March 3, 1885. He failed in his reelection bid and returned to his agricultural pursuits.
