Ernst and Young is in the final stages of agreeing an audit contract worth about £220,000-a-year, for up to seven years.
The deal is being negotiated with a company called Public Sector Audit Appointments Ltd, owned by the Local Government Association.
Council chiefs hope it is big enough to tempt the Leeds-based firm to move in.
Coun Mazher Iqbal, cabinet member for business and investment, said: “We’d be very happy to work with a national auditor if it wanted to set up a local office.
“We have a great team of people to support businesses looking to move to Sheffield, and some fantastic office space available.
“The public sector audit contract is being arranged by a national organisation called PSAA, which takes the decision about who to appoint.”
A spokeswoman for Ernst and Young said: “We are unable to comment on this matter.
“In terms of opening a Sheffield office - EY has no current plans.”
A report by HS2 published last week, stated EY intends to spend £20m on commercial property outside London by 2018.
News of the audit contract comes a week after PWC announced it was closing its Sheffield office in a drive to save money.
That is seen as a huge opportunity by accountants Grant Thornton which is set to move its entire 146-strong office into the city centre next year.
The firm is taking space in the former NUM offices on Barkers Pool which are undergoing a £5m revamp.
Managing partner Paul Houghton said: “As a company we’re committed to the city and region, because our passion and purpose is to be at the heart of shaping a vibrant local economy.
“To be the firm leading the way on these things requires us not only to be providing services to organisations in the city, but to be part of the fabric of the city. Our planned move to the city centre is all about demonstrating this commitment in bricks and mortar – it’s not just talk but walk.”