A witness told how a man wearing a balaclava and armed with a hammer had jumped over the counter at City Road Service Station and grabbed bags of cash at around 12.20pm.
But police who happened to be pulling in to the shop to buy a drink arrested a man outside after quick-thinking staff pressed the panic button.
Glenn Ashley, who was in the store, opposite the cemetery, at the time, said: "I'd just popped in to get a paper and a loaf of bread when a guy in a balaclava legged it past me.
"At first I thought it must be someone off a motorbike who was in a hurry, but then he jumped over the counter.
"The staff hit the panic button and two police cars happened to be passing.
"He ran out and the officers arrested him. He had a lump hammer and several bags of coins. It was surreal."
Police tweeted following the incident: "One detained for armed robbery, City Rd @SYPSheffield. Just robbed shop as officers went to buy a drink. Disarmed and arrested. Enq ongoing."
A South Yorkshire Police spokesperson said: "Police were called to the Texaco petrol station on City Road, Sheffield at around 12.20pm this afternoon (Wednesday 30 November) after it was reported a man, who is believed to have been armed with a hammer, attempted to take money from the petrol station.
"A 47-year-old man was arrested by police at the scene and has been taken into custody.
"No one was injured in the reported robbery and the petrol station has now reopened."