Pallets and packaging to the rear of Tesco on Ecclesall Road were set alight last night.
Flames were spotted at around 9.10pm and the alarm was raised.
Firefighters alerted to the incident said the blaze was started deliberately.
They spent around 90 minutes at the scene.
Anyone with information should call South Yorkshire Police on 101.