Mr Parish, who has represented Tiverton and Honiton in Devon since 2010, said the first time he looked at the pornographic website was accidental after looking at a tractor site.
But admitted the second time he looked at it – in the House of Commons while he was sitting waiting to vote – was deliberate.
He claimed it had been a "moment of madness" and was suspended by the Conservative Party on Friday over the allegations.
Two female colleagues claimed they had seen him looking at adult content on his phone while sitting near them.
In an exclusive interview with BBC South West, Mr Parish said: "The situation was that funnily enough, it was tractors I was looking at.
"I did get into another website that had a very similar name and I watched it for a bit, which I shouldn't have done.
"But my crime - biggest crime - is that on another occasion I went in a second time."
He said what he did "was absolutely totally wrong", adding: "I will have to live with this for the rest of my life. I made a huge terrible mistake and I'm here to tell the world."
"I was wrong, I was stupid, I lost sense of mind," he said.
But he denied watching the material in a way where he hoped other people would see it.
He told the BBC: "I make a full apology. A total full apology. It was not my intention to intimidate."