Police were called to Howden House, Union Street, after the man started making threats to set the building alight at around 9.20am.
He was arrested at around 9.30am.
Howden House is a contact point for members of the public with queries about council services.
A South Yorkshire Police spokeswoman said: "It was reported to us just before 9.20am that a man, believed to be his 30s, was in the building in possession of a petrol can and he was making threats to set the building on fire. He was arrested at 9.30am.
"No members of the public were in the building and staff had been evacuated."