"He caught himself before he told Dwight that when they played the Giants," Archie says.
As the youngest of the three brothers, Eli spent the most time at home without his siblings. Like Cooper, he followed Archie to Mississippi, while Peyton went to Tennessee.
It might seem a given that all three would wind up as football players, but Eli says otherwise.
"Obviously I feel our parents did a great job of raising us, supporting us, and not forcing us into sports or into doing anything," he says. "They told us to find something you are passionate about and give it your all, and that has been great advice."
It also meant Peyton and Eli went the same route as Archie — Cooper was a wide receiver, in part because when he was in high school, a young QB named Peyton came along. And both Harbaugh boys followed Jack into coaching, although Jim also was a highly successful college and pro player. As a quarterback, of course.
And all of them are competitive on and off the field, albeit in a far more friendly way when it doesn't involve football.
Asked if he needs to beat Jim to making the phone call to Jack on Father's Day, John Harbaugh says: "There's always a race. Birthdays? There's always a competition of who can get to the phone call first. Believe me, my dad does keep track."
Both Jack Harbaugh and Archie Manning note how difficult it is to get the entire family together for any occasion. Jack and Jackie did get to celebrate their 50th wedding anniversary as well as Thanksgiving in Baltimore in 2011, mainly because the 49ers were playing at the Ravens that night.
What seems to impress the patriarchs most is the interaction among their children and their children's children — the Mannings have six grandkids, with Eli and Abby expecting a baby next week.
Archie recalls how Peyton reacted to older brother Cooper's career ending early at Ole Miss.
"Peyton was distraught," Archie says. "He said how life is not fair. I think that changed all of our lives, taught all of us to count your blessings."
Jackie Harbaugh, echoing her husband's comments, stresses how her three children and their families "respect what they all do in their lives."
"It's like John says, the best thing you can do in life is raise somebody up," she says.
To which Jack adds: "The fact is we are always there for each other, on Father's Day or any day. That loves is always there."