Duffy, who has won two promotions in only three seasons since arriving at Bramall Lane, was again missing as United continued their pre-season preparations with a 5-0 win over Chesterfield.
Speaking after the match, which saw captain Billy Sharp score a hat-trick, Wilder told The Star that Duffy had asked for a new deal before Saturday's visit to Sixfields before being given "the weekend off" to explore other "options ahead of the new Premier League campaign.
"He came to see my on Thursday, I think it was, and takjed about a new contract," Wilder said. "I reminded him that he signed one 18 months back and got a rise to Premier League money.
"Unless it's a special case, my work has to be improving the first team squad by bringing players in. Not looking after players already under contract. He's had a rise.
"He won't dictate to me when he signed a new contract and neither will his agent. It's as simple as that. We told him that and he wanted to pursue different options. He wanted to do that and I gave him the weekend off to do that."
Duffy, who helped United reach the top-flight earlier this year, has yet to speak publicly about the matter.
Asked if he would accept the midfielder back into the group. Wilder responded: "That's a decision I will make. He clearly felt his future was elsewhere.
"If you ask the best manager in the business, if a player isn't happy, well."
With record signing Lys Mousset watching from the stand, Wilder confirmed Osborn's move from the City Ground is close to being completed while United remain "hopeful" Dean Henderson will eventually seal his return to Bramall Lane after spending last season on loan from Manchester United.
"We've been in conversations about Ben," Wilder said. "It's between the two clubs now. The two clubs are talking and that's where it's at."
"Ben can play in a number of positions and we think, with what we're going into, it's important to have thatg competition for places," Wilder added.