American Dufner, the 2013 PGA Championship victor, started three shots behind co-leaders Graham DeLaet of Canada and Englishman Luke Donald.
Former US Open champion Webb Simpson and American Kevin Kisner are a shot further back in third, with Donald's former Ryder Cup team-mate Ian Poulter fifth on 10 under.
"I feel like this golf course has always been a good set up for me", Dufner said.
Donald has finished second or third in six of his past eight visits to the RBC Heritage - and put himself in position for another stellar finish this time, too.
Smith sunk birdies on the second, sixth, 11th, 12th and 18th holes with his lone bogey coming on the par-four ninth.
Poulter arrived at Hilton Head needing to earn $US 145, 000 combined in this week's event and next week's Valero Texas Open to be assured of being afforded Full 2017 Tour status after missing almost six months in the later half of previous year due to a foot injury.
Fairmont native William McGirt got off to a blistering start en route to carding a 3-under-par 68 in the first round of the RBC Heritage on Thursday at Harbour Town Golf Links. The last six Heritage winners have come back from two or more strokes behind after 36 holes, since Jim Furyk rode a 36-hole lead to victory in 2010.
"I just knew from a mental standpoint I needed a couple of weeks at home to recharge", DeLaet said.
"There are a lot of guys that have just one PGA Tour victory". On the 15th, the course's final long hole, Dufner settled for birdie, one of three in a row on the back nine that took him into the lead.
The setup around Harbour Town's greens is allowing players to play a variety of shots, which should help DeLaet if his ball-striking wanes. David Frost was second after a bogey-free 66.
Danny Willett, who missed the cut in his defence of the Masters at Augusta, had two double bogeys in a 78 that left him at seven over absent for the weekend for a second tournament in succession.
European - Gregory Havret took the outright lead at the Trophee Hassan II in Morocco as he moved to 6 under par after Friday's second round.
DeLaet followed with birdies on two of the next three holes to reach 10 under. She made six more birdie putts the final round, taking the lead on the 14th hole.