In Atlanta, where a record low of 6 degrees hit early Tuesday, fountains froze over, a 200-foot Ferris wheel shut down and Southerners had to dig out winter coats, hats and gloves they almost never have to use. It shouldn't take too long to thaw out, though. The forecast Wednesday was sunny and 42 degrees.
In the Midwest and East, where brutal polar air has lingered over the past few days, temperatures climbed but were still expected to be below freezing.
After more than two days of temperatures well into the negative double digits along with unbearable wind chills, Minnesotans emerged from the indoors in greater numbers Tuesday in anticipation of a slight warming, reports CBS Minneapolis station WCCO-TV. Forecasters said temperatures could approach zero in many parts of the state by Wednesday, followed by a relative heat wave: the teens on Thursday, and balmy 30s by Friday and Saturday.
In Indianapolis, Timolyn Johnson-Fitzgerald returned to her home after spending the night in a shelter with her three children because they lost power to their apartment. The water lines were working, but much of the food she bought in preparation for the storm was ruined from a combination of thawing and then freezing during the outage.
"All my eggs were cracked, the cheese and milk was frozen. And the ice cream had melted and then refroze. It's crazy, but we're just glad to be back home," she said.
On Tuesday, the mercury plunged into the single digits and teens from Boston and New York to Atlanta, Birmingham, Nashville and Little Rock - places where many people don't know the first thing about extreme cold.
"I didn't think the South got this cold," said Marty Williams, a homeless man, originally from Chicago, who took shelter at a church in Atlanta. "That was the main reason for me to come down from up North, from the cold, to get away from all that stuff."
In Chicago, homeowners were warned about the danger icicles could pose to their houses, reports CBS Chicago station WBBM-TV.
The cold turned deadly for some: Authorities reported at least 21 cold-related deaths across the country since Sunday, including seven in Illinois and six in Indiana. At least five people died after collapsing while shoveling snow, while several victims were identified as homeless people who either refused shelter or didn't make it to a warm haven soon enough.
In Missouri on Monday, a 1-year-old boy was killed when the car he was riding in struck a snow plow, and a 20-year-old woman was killed in a separate crash after her car slid on ice and into the path of a tractor-trailer.
In a phenomenon that forecasters said is actually not all that unusual, all 50 states saw freezing temperatures at some point Tuesday. That included Hawaii, where it was 18 degrees atop Mauna Kea, a dormant volcano.
It was 1 degree in Reading, Pa., and 2 in Trenton, N.J. New York City plummeted to 4 degrees; the old record for the date was 6, set in 1896.
"It's brutal out here," said Spunkiy Jon, who took a break from her sanitation job in New York to smoke a cigarette in the cab of a garbage truck. "Your fingers freeze off after three minutes, your cheeks feel as if you're going to get windburn, and you work as quick as you can."
Farther south, Birmingham, Ala., dipped to a low of 7, four degrees colder than the old mark, set in 1970. Huntsville, Ala., dropped to 5, Nashville, Tenn., got down to 2, and Little Rock, Ark., fell to 9. Charlotte, N.C., reached 6 degrees, breaking the 12-degree record that had stood since 1884.
The big chill started in the Midwest over the weekend, caused by a kink in the "polar vortex," the strong winds that circulate around the North Pole. The icy air covered about half the country by Tuesday, but it was moving north, returning more normal and warmer weather to most of the country. This weekend, it was expected to be in the 50s in New York and even higher in places farther south along the Eastern Seaboard.
The deep freeze dragged on in the Midwest. More than 500 Amtrak passengers were stranded overnight on three Chicago-bound trains that were stopped by blowing and drifting snow in Illinois. Food ran low, but the heat stayed on.
On Tuesday, many schools and day care centers across the eastern half of the U.S. were closed and officials opened shelters for the homeless and anyone else who needed a warm place.
With the bitter cold slowing baggage handling and aircraft refueling, airlines canceled more than 2,000 flights in the U.S., bringing the four-day total to more than 11,000.
The Lower 48 states, when averaged out, reached a low of 13.8 degrees overnight Monday, according to calculations by Ryan Maue of Weather Bell Analytics. An estimated 190 million people in the U.S. were subjected to the polar vortex's icy blast.
Still, farmers worried about their crops.
Diane Cordeau of Kai-Kai Farm in Indiantown, Fla., about 90 miles north of Miami, had to pick her squash and tomatoes Monday to beat the freeze but said her leafy vegetables, such as kale, will be sweeter and tastier because of the cold.
"I'm the queen of lettuce around here, so the colder the better," said Cordeau, whose farm serves high-end restaurants that request specific produce or organic vegetables.
PJM Interconnection, which operates the power grid that serves more than 61 million people in the Mid-Atlantic, Midwest and South, asked users to conserve electricity because of the cold, especially in the morning and late afternoon.
Across the South, the Tennessee Valley Authority said power demand in the morning reached the second-highest winter peak in the history of the Depression-era utility. Temperatures averaged 4 degrees across the utility's seven-state region.
In South Carolina, a large utility used 15-minute rolling blackouts to handle demand, but there were no reports of widespread outages in the South.
Natural gas demand in the U.S. set a record Tuesday, eclipsing the mark set a day earlier, according to Jack Weixel, director of energy analysis at Bentek Energy.
© 2014 CBS Interactive Inc. All Rights Reserved. This material may not be published, broadcast, rewritten, or redistributed. The Associated Press contributed to this report.