Residents have been campaigning for months to save Owlthorpe Fields near Crystal Peaks and had collected a 1,000 name petition against developing the site with up to 500 homes.
The first phase was for 200 properties by Avant Homes but councillors overwhelmingly rejected it - against their own planning officers' advice.
Owlthorpe Fields Action Group was backed by Sheffield South East MP Clive Betts, councillors Bob McCann and Douglas Johnson, the Campaign to Protect Rural England, Sheffield and Rotherham Wildlife Trust and Dr Patrick Harrison, a lecturer in biochemistry.
There were concerns about the loss of wildlife and green open space, an increase in traffic and the lack of affordable housing.
The plot was one of three which could be developed but councillors said they wanted to see a holistic plan for the whole area, not just sites developed in a piecemeal way.
The site is designated for housing though and because officers had recommended the plans be approved, the committee will have to hope its reasons for refusal are strong enough to withstand an appeal by Avant.
Beighton Liberal Democrat councillor Bob McCann said because Sheffield doesn't have a Local Plan, it left too much to chance.
"We need this Local Plan so we know exactly what is going to be developed - without one, planning officers are struggling.
"But this campaign by residents shows the strength of feeling, I've had 200 emails this past fortnight objecting.
"This is all down to Owlthorpe Fields Action Group which has been a well coordinated enthusiastic campaign and goes to show what local people can do when they stick together.
"There is still a long way to go but this was a victory. People are well aware there needs to be a development but they are looking for something sympathetic."
Green councillor Douglas Johnson also called for the Local Plan. “Once again, this situation need not have come about if the council had an up-to-date Local Plan where Owlthorpe could have been respected for the greenfield site it is.
“The plans were a poor quality offering that wasted the opportunity for good transport links and would have meant the destruction of land that is highly valuable for biodiversity, science and recreation.
“Ten objectors spoke superbly and were congratulated by the committee for being concise. This shows why it is vital for the public to be able to put their case directly to councillors at a planning meeting.
“It is evident this area means a lot to local people - I received dozens of emails about it. It is absurd to build on greenfield sites like Owlthorpe just because it was set aside for car-driven urban sprawl in the 1960s."
Andy Tickle of the Campaign to Protect Rural England said the committee had made the right decision.
"The councillors asked the right questions and realised that the site deserves much better. We hope the developer can now look constructively at bringing forward a much improved scheme which will create a far more sustainable, low carbon community that provides more homes but with far less negative impact which will be fit for the 21st century."
Claire Plant, speaking on behalf of Avant Homes, said they had included 15 per cent affordable housing and local roads could cope with increased traffic.
She added: "It's going to be really important to support the construction sector in the economy going forward. We have taken extensive ecological surveys and where possible, any impact has been avoided and a plan has been prepared to ensure sensitive long term management of the site."
The history of the area
Owlthorpe Fields should have been developed along the rest of the townships but instead, it was left to its own devices and flourished with greenery and wildlife.
Residents have used it for exercise and as a tranquil haven for 30 years but under council policy, it was brownfield land for housing.
MP Clive Betts said: "I am not opposed to having some housing on this site. The land was first purchased by the council for development when I was chairman of housing in the early 1980s and it has always been earmarked for housing so anyone buying a house in the area for the last 25 years would recognise housing was going to be built here at some point.
"My objection is the council divided the site into three and sold it off in separate parts. They should have had a masterplan for the whole site.
"There are not enough affordable homes in this plan and they are pushed to the back of the site which is not acceptable. Any low income families would look in disbelief at the cost of these shared ownership homes and there is no social rented housing."
Beighton Labour councillor Chris Rosling-Josephs was on the committee and voted against it. He said: "I think we need to develop that site for housing but it has to be affordable with an element of social rented housing."