Richard Wood's first-half goal gave the Millers a sixth successive victory and one they fully deserved.
More than 1,000 away fans celebrated wildly at the end as Paul Warne's promotion-chasing side moved to within four points of the top two.
They were clearly the better side and are now just four points away from the automatic-promotion places.
Wood scored just a minute before the break, poking the ball home after Michael Ihiekwe had got his head on an Anthony Forde cross.
Rivelin Valley Road Sheffield: Body found after police search of Rivelin Valley
Hosepipe ban: Reservoir near Sheffield less than 15 per cent full as photos show how water levels have plunged
Sheffield weather: Met Office issues thunderstorm warning as heatwave set to end with heavy rain and lightning
The Long Blondes: Leadmill cancel gig after allegations surface against band creator Dorian Cox
Rotherham missing boy: 13-year-old missing for 10 days as police make appeal
Rotherham were good value for their interval lead and were rarely troubled seriously in the second half as they recorded a famous triumph.
The Millers should have been ahead after five minutes when Semi Ajayi's clever flick sent Michael Smith clean through but the striker shot inches wide.
Ajayi was the culprit himself in the 14th and 36th minutes. First, he completely fluffed his kick in front of goal after David Ball had danced through the home defence and brought a good save from Dean Henderson. Then, he rose unmarked and put a free header from Joe Newell's corner the wrong side of the post.
Marek Rodak saved the visitors after 25 minutes, standing up to Shaun Whalley when the Shrews winger went through one on one, but Forde and Newell fired in shots as Rotherham did most of the attacking.
Rodak was in goal-stopping form again, in the 62nd minute, keeping out Nathan Thomas' shot in stunning style.
However, the Millers could have extended their lead 10 minutes from time when Joe Newell produced a devastating burst down the left only for Smith to put his effort off target.
Whalley fired over late on for the Shrews but by then there was going to be only one winner.
Shrewsbury (4-1-4-1): Henderson; Bolton, Nsiala, Sadler, Beckles; Godfrey; Whalley, Nolan (Payne 68), Ogogo, Thomas (Rodman 84); C Morris (John-Lewis 75). Subs not used: MacGillivray, Lowe, B Morris, Hendrie.
Rotherham (4-4-2): Rodak; Emmanuel, Ajayi, Wood, Ihiekwe; Forde (Williams 63), Towell, Vaulks, Newell (Palmer 90+2); Ball (Lavery 75), Smith: Subs not used: Price, Cummings, Taylor, Yates.
Goals: Wood 44 (Rotherham).
Referee: Carl Boyeson (East Yorkshire).
Attendance: 7,012 (1,136).