The first Episcopal parish established in Caldwell County was St. Andrews, founded by the Reverend Robert Johnstone Miller (Parson Miller) in the Granite Falls area about 1821. Although it was short-lived, St. Andrews became the mother of St. James in that some of the families of St. Andrews helped establish St. James after the death of Parson Miller and the decline of their original parish.

In 1844, the Reverend Thomas Mott arrived in the vicinity of Lenoir. Mott held Episcopal services throughout Caldwell County, urging the scattered Episcopalians to build a church. On June 10, 1848, a drive was begun for pledges to build St. James Church. 39 members and subscribers gave a total of $1,564 to build the church on property donated by Major Elisha Perkins Miller, oldest son of Parson Miller. The building was completed in 1851. St. James was accepted as a parish by the Diocese of North Carolina on May 24, 1849. Some present members trace their ancestry to the families who made up this first congregation.

The parish House was built in 1932 under the leadership of the Reverend Boston M. Lackey, who served the church for 28 years - the longest tenure of any rector. The Parish House was renovated and the third floor completed in 1957 under the leadership of the Reverend Floyd W. Finch, Jr.

The Church building has been remodeled four times, the last time being in 1962 when Reverend Frederick E. Taylor was Rector. On this occasion, the chancel and narthex were transposed, so that the Church, then, as now, faces College Avenue. In 1993, during the rectorship of the Reverend Peter D. Ouzts, the Church exterior was replaced; however, throughout all the changes, the original nave was preserved, so that St. James is the oldest church building in Lenoir.

Most recently, during the tenure of the Reverend Ingram C. Parmley, St James embarked upon the monumental task of total Parish House renovation, providing for a more spacious Parish Hall, modern kitchen facilities, classrooms, restrooms and much more.