NEW YORK (CNNMoney) -- Trillions of dollars in looming tax hikes and spending cuts set to take effect next year would likely cause a recession, the Congressional Budget Office said Tuesday.

At issue is the so-called fiscal cliff -- a series of measures set to begin in January that would take more than $500 billion out of the economy in 2013 alone.

Those measures include the expiration of the Bush tax cuts and protection of the middle class from the Alternative Minimum Tax, the onset of $1 trillion in blunt spending cuts, and a reduction in Medicare doctors' pay.

If Congress lets all those policies take effect, inflation-adjusted growth for 2013 would be just 0.5% -- with the economy projected to contract by 1.3% in the first half of the year and to grow by 2.3% in the second half.

"Given the pattern of past recessions as identified by the National Bureau of Economic Research, such a contraction in output in the first half of 2013 would probably be judged to be a recession," the CBO said.

Fiscal cliff: Market sting may come sooner

Overall, taxable income would be lower, unemployment would be higher, tax revenue would fall, and spending on safety net provisions such as jobless benefits would increase, the CBO said.

Federal budget deficits, meanwhile, would be reduced by more than $500 billion.

By contrast, were lawmakers to cancel the onset of the fiscal cliff measures, the CBO estimates that growth in real GDP for calendar year 2013 would come in around 4.4%.

Fiscal cliff: What you need to know

But the agency cautioned that just canceling all the fiscal cliff measures and not imposing "comparable restraint in future years" would hurt the economy in the long run.

Few observers expect that lawmakers would actually let all the tax hikes and spending cuts take effect, when all is said and done.

But when all will be said and done is hard to predict, since the two parties remain in entrenched positions. Everyone says they'd be happy to address the fiscal cliff now, if only the other guy would budge.

This gets you to the video clip: