Tuncay Sanli's 32nd-minute strike was enough to see off dismal Derby - but Boro finished badly and almost paid when Robert Earnshaw missed a golden chance to equalise five minutes from time.
After his team's 1-0 Barclays Premier League win, Southgate said: "We've seen it through today, and they weren't able to punish us - but we can't play like that.
"They should have scored, and we would have been sitting here frustrated again.
"We've got to be better than we were in the second half. We've got to be bolder and be willing to attack more. At the moment we tend to go into our shells, because of the importance of the points.
"Guts has got them over the line, but we have got to be mentally tougher and braver in the way we finish game.
"I think they are learning as a team and scrapping as a team and grafting for each other."
Boro are now eight points clear of the relegation zone.
But Southgate does not consider his team safe yet.
"If we switch off for five minutes at this place we let ourselves down quickly, so we have to drive ourselves on," he warned.