The Charleston Manufacturing Company did not make money during its first two years of operation. C. I. Walker, the president, claimed that this was because of the very small margin between the cost of raw cotton and the price of manufactured goods. Poor planning and international economic forces conspired to chip away profits, not just in Charleston, but in mills across the South. Instead of shutting down the mills, most companies, including the Charleston Manufacturing Company, chose to run on “short time,” reducing shifts, production, and wages. Workers who lived paycheck to paycheck often went hungry and couldn’t pay their rent.

In August 1885, a major hurricane struck Charleston, stripping off the roof of the factory and exposing the machinery and materials. Almost exactly a year later, a devastating earthquake hit the city, the largest ever to strike the east coast of the United States. Unlike much of Charleston, the factory building came through remarkably well. Still, the massive chimney cracked and had to be rebuilt.  Walker praised the building’s resilience in the company’s annual report: “The 'geysers' breaking through our cisterns, the subsidence of the ground level north of the mill, the sinking of the boiler house, and the lateral movement of the picker house from the main building, attest the severity of the action around the mill. The money spent in securing such superior work on our building has well paid for itself.” Eighty-five hands were working in the weave room on August 31, 1886, when the earthquake hit. Only eleven of them showed up for work when the factory reopened.

And then, just weeks after the earthquake, a third disaster struck. A boiler at the factory exploded, killing one worker and scalding another. In the wake of these calamities, many workers quit or simply disappeared, leaving their machines untended. The president of the company admitted that “while our real property escaped any serious injury, the demoralization of the help was so great, that for months afterwards our production was largely reduced, for want of workers."

At the annual stockholders’ meeting in May 1887, President Walker reported that the factory had, for the first time, showed a small profit, but it was not enough to satisfy investors. One of the biggest obstacles to profitability, according to the managers, was the scarcity of white laborers in Charleston. They had hoped to train four hundred hands, yet had succeeded in recruiting and retaining just ninety-eight. As a result, a News and Courier article divulged, the investors were reassessing their business strategies, and “now discussing the advisability of putting up money and trying to run the mill with negro hands.” They did not act on this idea immediately—they decided to sell the Charleston Manufacturing Company. But the possibility of hiring black workers was officially on the table.