On-loan Chelsea youngster Gilmour was singled out by some of the travelling Canaries fans venting their frustrations at Selhurst Park during a 3-0 defeat by Crystal Palace last month.
Smith, though, feels the 20-year-old Scotland international does not deserve to be the focus of their ire.
“I was disappointed with what I heard, not during the actual game but I had family members in the (away) stand,” said Smith.
“It was a minority, Billy knows that. Everybody within the dressing room and the training ground knows the value of Billy.
“Unfortunately a big expectation comes along with Billy and people have to remember he is 20.
“Stuff like that, it will hurt and there is no one I see tried more than Billy in wanting to get the ball and wanting to do well (against Palace).
“He is very well supported here, within the group with the players, the staff, and he will just keep working hard to get better.”
There has been some suggestion that Chelsea may recall Gilmour in the wake of the criticism, with manager Thomas Tuchel expected to make a decision in the coming weeks.
Meanwhile, Smith admits he has no idea if midfielder Todd Cantwell will still be at the club when the January transfer window closes.
Cantwell struggled for opportunities earlier this season under former boss Daniel Farke, and has so far not made the most of a chance to kickstart his campaign following Smith’s arrival.
The 23-year-old is out of contract in the summer, although the club do hold the option of an extension, and has been linked with a switch to Newcastle.
Smith will leave any possible transfer discussions with Norwich’s sporting director Stuart Webber, but is determined to see the best of Cantwell while he remains part of the squad.
“I know there is an awful lot of speculation about Todd at the moment and there always is about good players,” said Smith.
“He has been in a situation where when we arrived he had not been training with the first team for a while, so we had to build up his fitness and he is in a good place now.
“He is contention to play on Sunday (in the FA Cup at Charlton), but I don’t deal with the speculation, Stuart does, so I don’t know is the honest answer.
“Hopefully he will be here still and playing well, because that is what we want from him. We know what we can do at this level and are working hard to get him back to that.”
Smith added: “It has been frustrating, but I think it’s been frustrating for him as well.
“He wasn’t up to speed in the first couple of games we played (since I took over), but he is certainly getting there.
“He is working hard in training, we have had no problems with him in terms of attitude and application at all.
“It is about building his confidence up as well, which had taken a bit of a hit, but there is certainly a talented footballer there.”
Norwich remain bottom of the Premier League and have not played since December 28 following a Covid-19 outbreak which saw the trip to Leicester on New Year’s Day postponed.