The relative success of the 1864 Southern Bazaar was in many ways the last hurrah for the Confederate supporters of Abercromby Square, as the final years of the war brought turmoil and financial failure to some of the leading figures in the Southern Club. Throughout the war, the Confederate operation in Europe had been plagued by organizational confusion. The low point came with a letter written by Bulloch to Prioleau protesting a request for a transfer of £40,000 to the Confederacy's near bankrupt treasury department. Prioleau argued the money was “essential for the preservation of the government." However, transferring this money left no budget for the Confederate naval operations in Europe, much to Bulloch’s dismay.
The collapse of Confederate finances was also damaging for the business of Fraser, Trenholm, & Co. In the early months of 1865, Prioleau was beset with demands to repay the company's debts, which were accrued while managing blockade runs on behalf of the Confederacy. Demands for money came from many prominent Liverpudlians. The situation escalated when Liverpool resident George Wigg threatened legal action and a smear campaign in the press for the firm's refusal to pay a debt of £10,000. The Confederacy's financial failure soured the commercial relationships that Prioleau had fostered in Liverpool. An 1866 affidavit reveals that the Confederate Government owed Fraser, Trenholm, & Co. a mammoth £272,758 (roughly equivalent to $147,000,000 in the present day) at the end of the Civil War, demonstrating the degree to which the Confederacy had depended on Prioleau and his business throughout the war.
After the war, Thomas Haines Dudley, American Consul in Liverpool, and William Seward, U.S. Secretary of State, pursued a court case to reclaim funds from Charles Prioleau and Fraser, Trenholm, & Co. Prioleau tried to recover the Confederacy's outstanding debt to his company by selling off cotton and ships, but the value of these assets had fallen drastically at the end of the war. Prioleau attempted a deal with the U.S. government to receive £150,000 for any sales, but Seward blocked it. The United States was in the process of seeking reparations for the damage inflicted by the British-built CSS Alabama during the war, and government officials showed no interest in negotiating with Liverpool businessmen who had supported the losing side.
In addition, rumors circulated in the United States and Great Britain that the Confederacy, and specifically President Jefferson Davis, had funds equaling millions of pounds hidden in Liverpool. Convinced that Prioleau was involved, Dudley pushed forward with litigation. When Fraser, Trenholm, & Co. eventually declared bankruptcy in May 1867, the firm's liabilities were around £1.2 million, including £290,000 of Prioleau's private estate at 19 Abercromby Square. During this time, many incriminating financial records were destroyed. To relieve some of Prioleau's financial troubles, Major Norman Walker, a business colleague, fellow Confederate, and friend of Prioleau's, purchased 19 Abercromby Square in 1870.
Ultimately, the recovery of Britain's cotton industry after the US Civil War was a slow one. It was only in 1873 that the market, and prices of American cotton, returned to pre-war normality. Still, the first few post-war decades were tumultuous for Liverpool cotton merchants, though the Civil War did proved to be profitable for Britain's merchant marine. The disruption to American shipping allowed British merchants, including the many in Liverpool who held interests other than cotton, to step in and reap rewards. The experience of the residents of Abercromby Square demonstrates the varying fortunes that the Civil War bought to the city of Liverpool. Those in a position to take advantage of the commercial opportunities that the conflict presented, such as Charles MacIver or Robert Preston, were able to obtain short-term financial rewards. However, openly supporting the Confederate cause, and therefore American slavery, proved to be a huge personal and financial gamble, which lead to bankruptcy and political ruin for Abercromby residents such as James Spence and Charles Prioleau.