The year after Castle was incorporated, production of silver began to dwindle. In 1893, there was a financial panic in the United States, and President Grover Cleveland was convinced that the government's mandatory silver purchase program was the cause of the depression. He called a special session of Congress that summer, and the Sherman Silver Purchase Act was repealed. Silver prices plummeted, and Castle's Cumberland Mine closed down immediately. The Cumberland Boarding House served 135 suppers the last day the mines operated. Three days later, it served six men who remained to dismantle machinery. The town literally emptied out in 72 hours. A few families remained, but for all practical purposes Castle was dead. By 1936, only two people lived in the old town: the 75-year-old self-appointed mayor, Joe Kidd, and the 70-year-old constable, Joe Martino. The snows came early in the winter of 1936­ 1937, and winds drifted the snow in the coulees 40 feet high; deer fell through the crusts of snow and could be found, after the thaw, frozen in the tops of cottonwood trees.

There was one blizzard after another that winter. One day, with supplies running low, Mayor Kidd hitched up a team of horses to a cutter (a light sleigh) and set out for the small ranching town of Lennep, seven miles down the canyon. He made three miles the first night and stayed with some shepherds at their camp. At Lennep the next day he picked up the mail, spent the night at a local ranch, and headed back the next morning. A mile from Castle the horses gave out, so Kidd walked to Martino's house, arriving at 9 p.m. He had a cup of hot coffee and left for his own house. It was only 500 yards away, but the mayor collapsed and died in the snow.

Martino was unable to carry the body, so he skied down to the sheep camp and the shepherds got word to the nearest big town, White Sulphur Springs. The sheriff and coroner skied into Castle and carried Kidd's body out on a toboggan, leaving Joe Martino as the last full-time resident of Castle. His fate is unrecorded, but when he passed on or left, the rodents took over.