The Valiants were 2-1 winners despite going down to 10 men in first half stoppage time when Pugh went into a challenge with Brown and left his studs showing.
Referee Tony Harrington wasted no time in pulling out his red card, which incensed the home support who felt Brad Barry should have been red carded for a second minute elbow on Luke Hannant.
Aspin addressed the sending off of his midfielder in his post-match press interviews.
“I looked at it at half time, I think it is harsh,” he said.
“I can see why the ref’s made that decision, when I looked at it again, his foot is slightly high.
“But I think the player made a meal of it and obviously that puts pressure on the referee to make the decision.”
Aspin was concerned about being a man down, given Chesterfield’s style of play, but in the end his men dug in and found a late winner to take three hugely important points.
“It is a shame for Danny because he had just scored,” said Aspin.
“We were down to 10 men against a team who are good in possession so they are going to work the ball and try to pull you out of position.
“They are probably one of the teams you don’t want to go down to ten men against because they keep the ball, but the players got the result.”