The man, believed to be in his 20s, was shot in Butterthwaite Road, Shiregreen, at around 7.30pm.
One resident said he heard one gunshot, looked out of his window and saw two men pointing a gun at another man before two further shots were fired.
The 45-year-old said the victim slumped to the ground and the two gunmen fled and jumped into a car parked nearby.
Concerned residents raised the alarm and helped the victim until medics arrived.
Police incident Woodhouse Sheffield: Two men and a woman injured after large fight breaks out on Market Square
Sheffield fire: Crews tackle blaze in Wharncliffe Woods near Oughtibridge as heatwave continues
Yorkshire Natural History Museum: New Sheffield museum in Malin Bridge is big success on opening morning
Popple Street Sheffield: 'Several' people arrested as street is cordoned off by police
Sheffield weather: Met Office issues thunderstorm warning as heatwave set to end with heavy rain and lightning
The eye witness, who does not want to be named, said the victim lives on Butterthwaite Road with his partner.
He said: "I was upstairs and heard a shot so I opened my curtains and saw two men wearing black clothes. One was holding a gun and pointing it at another man.
"I saw two further shots."
He said the victim was shot in his chest.
"The men with the gun ran off and got into a car," he added.
Butterthwaite Road is cordoned off this morning and under police guard while officers carry out enquiries at the scene.
Another resident, who lives close to the cordon, said: "It's not unusual for things like this to happen round here.
"I keep my head down and everything is fine.
"It's frightening to think that this was early evening and there could have been children around."
Anyone with information should call South Yorkshire Police on 101.