The lifting of the sanctions was announced by EU foreign policy chief Federica Mogherini and Iranian Foreign Minister Javad Zarif.
The US citizens released on Saturday are being flown to Switzerland and will be taken to a US base in Germany for medical treatment.
The other three Americans were named as Saeed Abedini, Amir Hekmati and Nosratollah Khosravi-Roodsari.
US citizens released by Iran
The four prisoners are all dual US-Iran citizens.
Jason Rezaian: aged 39, Tehran correspondent for the Washington Post. Detained in Iran for more than a year before his sentencing last November.
Saeed Abedini: aged 35, a Christian pastor who had been imprisoned since July 2012 for organising churches in people's houses.
Amir Hekmati: aged 32, former Marine who spent more than four years in prison on spying charges following his arrest in August 2011 during a visit to see his grandmother.
Nosratollah Khosravi-Roodsari: Little is known about the fourth US citizen.
Iranian citizens released by the US
Of the seven Iranian citizens being released, six are also citizens of the US. The Iranian state news agency listed them as: Nader Modanlo, Bahram Mechanic, Khosrow Afghani, Arash Ghahreman, Tooraj Faridi, Nima Golestaneh and Ali Saboun.
They are all accused or convicted of violating US sanctions on Iran.
A US official said a diplomatic channel was established "with the focus of getting our detained US citizens home".
"Iran has also committed to continue co-operating with the United States to determine the whereabouts of Robert Levinson," the official said.
According to the Associated Press news agency, Mr Levinson is a former FBI agent who disappeared in Iran in 2007 while working for the CIA on an unapproved intelligence mission.
A further 14 Iranians sought by the US would be removed from an Interpol wanted list, the US and Iran said.
A fifth American, writer and student Matthew Trevithick, was also released from jail in Iran on Saturday, but his release was not part of the prisoner exchange.
Analysis: Sebastian Usher, Middle East Editor, BBC World Service
For months after his arrest in 2014, there was uncertainty over what accusations Jason Rezaian was facing. He was held in the notorious Evin prison before finally being found guilty on charges that his lawyer said included espionage.
His newspaper vigorously denied this. President Barack Obama said he would not rest until he was freed. Now, his release has been announced in what Iran calls a prisoner swap - an indication he might have been used as a bargaining chip.
His release removes a shadow that had lain over relations between Tehran and Washington even as they were thawing with the nuclear deal.
The media freedom organisation Reporters Without Borders welcomed Rezaian's release, saying he "was just doing his job as a reporter and was unlucky enough to have dual US and Iranian nationality at a time when the two countries were conducting tense negotiations about Iran's nuclear programme".
"We are thrilled to see Jason finally free, but he should have never been imprisoned in the first place. Jason was innocent. It is outrageous that he has been used as a bargaining chip," said Christophe Deloire, the organisation's secretary general.