samuel and jane newman were the first owners of the guest house, which was originally built in the mid-1840s as a one-and-a-half story, greek revival town house. the newmans purchased the corner property in 1844 for $1, 000 and sold it in november 1853 for $9, 000. as originally built and documented in historic photographs, the guest house resembled the nearby john smith house, a smaller brick cottage at the southwest corner of north pearl and jefferson streets. samuel b. newman was the grandson of samuel brooks, the first mayor of natchez. newman was a natchez merchant who founded the business establishment of stockman and newman and also served as sheriff of adams county. in the early 1850s, he relocated to new orleans and established the firm of s. b. newman and company. (the newmans are buried in new orleans.)

jacob and leah ullman acquired the guest house as their residence in 1871 and lived there until their deaths, in 1880 and 1873 respectively. their heirs sold the property in 1884. jacob ullman, was the patriarch of the prominent ullman family in natchez and was one of many jewish immigrants who came to the natchez area before the civil war. he was born in germany in 1805 and came to america from alsace in 1852. he settled first in nearby port gibson, but, in 1870, relocated to natchez, where several of his children had already achieved success in the mercantile world. his son samuel ullman, who moved to natchez in 1865 and established the mercantile house of ullman and laub, was instrumental in the establishment of temple b'nai israel. samuel ullman later moved to birmingham, alabama, where he served as president of the board of education and as a city alderman. he wrote the well known poem, "youth, " which has been published in many languages and was a favorite poem of general douglas macarthur. jacob and leah ullman's son marcus maximillian ullman founded the well known department store, m. m. ullman & co., which operated for more than a century in downtown natchez. (jacob and leah ullman are buried in the natchez city cemetery.)

isaac lowenburg was born in germany in 1837. he served in the commissary department of the union army and arrived in natchez when the union army occupied the city in the summer of 1863. he soon initiated a friendship with merchant john mayer, who had earlier immigrated to natchez from landau, france, in 1841. mayer's son simon was then serving as a major in the confederate army, and mayer's wife and children were not excited about their father bringing a yankee home to visit. lowenburg and his friend henry frank, who also came to natchez with the union army, soon married two of john mayer's daughters, ophelia and melanie respectively, and eventually became two of the city's most successful businessmen. before 1880, lowenburg was elected mayor of natchez, a position he relinquished after two terms due to ill health. he died in 1888 and is buried on jewish hill in the natchez city cemetery.

general william t. martin was born in kentucky in 1823 and immigrated to natchez, where he was admitted to the mississippi bar in 1844. in 1851, he represented the family of african american diarist william johnson as a special prosecutor to try to convict johnson's murderer. in 1855, he and his wife built the mansion monteigne, where he lived until his death in 1910. although a strong unionist whig and opposed to the concept of secession, martin ultimately chose to defend his home state and joined the confederate army. according to martin himself, he gave the last speech against secession in the mississippi legislature and was accused of being "unfaithful to the south and not fit to be trusted." ultimately, martin became natchez's highest ranking confederate soldier, one of five mississippians to achieve the rank of major general. after the civil war, martin returned to natchez, where he resumed both the practice of law and his career in state politics. he also served as president of the natchez, jackson, & columbus railroad. (martin is buried n the natchez city cemetery.)

william johnson was born a slave in natchez in 1809 and was freed by his white owner, who was probably his father. he learned the trade of barbering and, by the 1840s, had become a farmer as well as an urban businessman. johnson was murdered in 1851, and a published eulogy testifies to the high regard in which he was held by the local community. despite public outrage, the legal restrictions of race prevented the conviction of his murderer. under mississippi law, a black man, slave or free, could not testify in court against a white man, and the only witnesses to the crime were black. the defense of the murderer rested solely on his supposed whiteness. in 1951, exactly one hundred years after johnson's death, louisiana state university press published his 2, 000-page diary. his business acumen and the conduct of his life assured his position at the top of his social class, and his personal account of that life secured him an important place in american history. johnson's diary provides the most complete account of the life of a free black in the antebellum south. (william johnson is buried in the natchez city cemetery.)