WASHINGTON — The U.S. Supreme Court on Thursday ruled unanimously that taxpayers have a right to challenge an Internal Revenue Service summons enforcement action in court when they can show the tax agency might have issued the summons in bad faith.

But the nine justices sent the case back to the 11th U.S. Circuit Court of Appeals and said that court had wrongly decided that IRS agents could be examined at an evidentiary hearing based on a taxpayer’s mere “conclusory allegations.”

Although the Supreme Court scaled back the Atlanta-based appeals court’s taxpayer-friendly ruling from 2013, the high court decision might make it harder for the IRS to avoid summons challenges in district court, tax lawyers said.

“This (decision) is a setback to the government’s attempt to shield IRS agents from being cross-examined about potential IRS abuses,” said Robert Kovacev, a former Justice Department tax lawyer and now a partner with law firm Steptoe & Johnson LLP.

A Florida man, Michael Clarke of West Palm Beach, had accused the IRS of issuing a summons as retribution against him and his business partners for resisting an audit. In its ruling, the Supreme Court said Clarke can seek an evidentiary hearing about the motives of IRS officials.

The IRS argued taxpayers already had opportunities to challenge a summons and that an evidentiary hearing would be an unnecessary “fishing expedition.”

In a nine-page opinion written by Justice Elena Kagan, the Supreme Court said: “A taxpayer has a right to conduct an examination of IRS officials regarding their reasons for issuing a summons when he points to specific facts or circumstances plausibly raising an inference of bad faith.”

But the opinion also said that the appeals court “applied a categorical rule, demanding the examination of IRS agents even when a taxpayer made only conclusory allegations. That was error.”

An IRS spokesman declined to comment on the decision.

“We’re very happy and pleased with the decision,” said Edward Marod, a lawyer with Gunster Yoakley & Stewart PA, who argued the case for Clarke. “We live to fight another day.”

Under the law, the IRS can issue a summons for information when a taxpayer refuses to provide it voluntarily. If the taxpayer ignores the summons, the IRS can then ask the U.S. Department of Justice to seek a court order from a judge.

Judges routinely rubber-stamp requests for court orders to enforce summonses without first holding evidentiary hearings.

In the past, evidentiary hearings have been rarely granted, said Barbara Kaplan, a tax lawyer for Greenberg Traurig LLP.

“The Supreme Court’s opinion is a victory for taxpayers,” she said.