President Barack Obama is criticizing Congress for refusing to close the military prison at Guantanamo Bay, Cuba, saying there’s “simply no justification beyond politics” to keep it open.

Obama pledged at the start of his Presidency to close the facility in his first year. Lawmakers of both parties blocked the effort.

The President has significantly reduced the number of inmates being held there from several hundred to under 50 in hopes of making a financial case for closing the Bush-era facility.

But that argument has not swayed lawmakers who object to moving inmates to U.S. prisons on the mainland.

In a letter to Congress the day before his term ends, Obama tells lawmakers they have put “politics above the ongoing costs to taxpayers” in keeping the prison open.