Steve Evans kept faith with the Millers side that beat Ipswich last weekend but they ended up without a point at Ewood Park.
It was a disappointment particularly with Rovers winner coming four minutes from the end.
Rovers’ Jordan Rhodes was fouled on the right after five minutes then headed wide from Tom Carney’s cross.
Rhodes threatened again five minutes later but his shot was saved by Collin and the Millers faced more pressure as Jay Spearing homed in on goal.
Rovers lost Jason Lowe to injury and substitute Alex Baptisre was soon to be the subject of a penalty claim for handball by the Millers which was waved aside by the referee.
Rotherham threatened as Derbyshire set up Conor Newton for a shot which was blocked.
But Craig Conley gave the hosts the lead with a cracking goal, a right-footer from 20 yards four minutes before the interval.
Rovers’ Chris Brown had a header saved in the last minute of the first half.
Derbyshire had a shot blocked early in the second half before manager Steve Evans made two substitutions, bringing on Jack Hunt and Adam Hammill for Ben Pringle and Conor Newton.
The pressure began to build on Blackburn, who had Jay Spearing (foul) and Tom Cairney (dive) yellow-carded in a three-minute spell before Hammill had a shot saved for the Millers.
And the Millers got an equaliser with a dream goal for Matt Derbyshire against his boyhood team.
The Millers ace scored his third goal in as many matches with a right-foot shot from the centre of the box following a Paul Green cross.
But that man Rhodes put Blackburn back in front after 86 minutes from close range after Gestede headed on.
Blackburn: Steele, Lowe, Hanley, Kilgallon, Olsson, Cairney, Spearing, Evans, Conway, Brown, Rhodes. Subs: Eastwood, Spurr, King, Marshall, Baptiste, Taylor, Gestede.
Millers; Collin, Richardson, Morgan, Arnason, Fryers, Newton, Green, Smallwood, Pringle, Sammon, Derbyshire.Subs: Thompson, Broadfoot, Hunt, Bowery, Hammill, Ward, Frecklington.