After establishing a 20-point lead with under 20 minutes left, Sheffield allowed their hosts to score 16 unanswered points in the last 15 minutes, which almost undid their earlier efforts.
A red card for prop Tyler Dickinson also saw them finish the game with 12 men.
Unconverted tries from wingers Rob Worrincy and James Glover inside the first 20 minutes had got them off to a great start but Oldham soon halved the deficit after full-back Ryan Johnson fumbled a high kick.
Sheffield’s response was perfect, though, and Glover scored their third try before half-time when he ran the length of the field to cross the line and extend their lead again.
Izaac Farrell added the extras to make it 6-14 at the interval.
It was Oldham – themselves struggling in the table and without a win in five – who scored first after the break to make it 12-14 but, again, Sheffield responded well.
Loose-forward Aaron Brown capitalised on a poor kick from the restart to cross the line for the first of three tries in just six minutes.
A brace from Worrincy helped stretch their lead further, with Farrell converting all three efforts to make it 12-32 before Oldham hit back again.
A converted effort reduced the deficit then Dickinson saw red for a late and high tackle with just over 10 minutes left to play.
The hosts scored twice more in the closing stages to move within four points but Sheffield managed to hold on for their first win since beating York City Knights in Round two.
Man of the match Worrincy praised his team-mates after the game.
He tweeted: “Nice to get the plaudits but it was very much a team effort and one the boys have desperately trying to achieve for the last few weeks.
"We go again next week and try to build on the back of that effort and much-needed W.”
The Eagles, who now have eight points from nine matches, take on Widnes Vikings next Sunday (3pm kick-off).