Elisha Watson Gardner Sr. was born in South Kingston, Rhode Island. He was educated in the common schools of that State. On April 19, 1801 he married Sarah Pattison, daughter of Sarah (Utter) Pattison (who had been abducted by Indians at the Wyoming Valley Massacre) and Sunderland Pattison, a soldier in the American Revolution who accompanied General Washington on his expedition to Trenton on Christmas Eve 1776. Elisha and Sarah began their married life in Westerloo, Albany Co., NY, working on his father's farm.  In 1814 Elisha moved his family to Farmington, Ontario Co., NY - a trek of nine days by oxen-drawn wagons in persistently rainy weather - where Gardner purchased a farm in which he was to spend the remainder of his days. Soon after settling in Farmington he joined (initially against the wishes of his wife) the Religious Society of Friends (Quakers). All of his eleven children were instructed in that faith. His eldest son, Sunderland P. Gardner, was to become one of the most beloved and remarkable Quaker ministers of the 19th century, spreading a message of peace and tolerance throughout the northern United States and Canada. Another son, Elisha W. Gardner, Jr., was a well-respected attorney. His youngest son, Thomas P. Gardner, left farming to become a veterinary surgeon.

Father of Sunderland P., Hannah M., Sarah P., Harriet P., John W., Mary, Marion, Elizabeth, Amy Ann, Elisha W. Jr., and Thomas P. Gardner.