"It's just been a rough last 24 hours," B.J. Upton said.
Harvey (6-1) fanned six in a row at one point and didn't allow a hit until Jason Heyward's fluke infield single leading off the seventh. The right-hander tired in the eighth as the Braves tried to rally for the second straight game.
Trailing 4-0, Atlanta scored three runs and had the bases loaded before Bobby Parnell, the fourth Mets pitcher of the inning, fanned Chris Johnson to end the threat. Parnell earned his 10th save with a scoreless ninth.
John Buck homered for the Mets.
"Certain days you wake up and you feel good and you can let it go," Harvey said. "Today was one of those days."
The Mets had scored only 18 runs in Harvey's previous 10 starts while he was in the game. Largely because of that, he had eight no-decisions in a stretch of nine appearances before a hard-luck 2-1 loss to the St. Louis Cardinals in his last outing, snapping a stretch of 14 consecutive starts without a loss dating to his final appearance of 2012.
"He has electric stuff," Atlanta's Dan Uggla said. "He was throwing everything for strikes on both sides of the plate."
Atlanta opened the five-game series with its 21st comeback win of the season, a rain-delayed 2-1 victory that ended at 1:22 a.m. — less than 12 hours before the start of the start of the day-night doubleheader.
Dillon Gee took a 1-0 lead to the ninth, but Freddie Freeman won it for the Braves with a two-run homer. Only Kansas City has more come-from-behind victories than the Braves.
But the Mets erased the memory of that stunning loss behind two of baseball's most dynamic young pitchers.
The Braves didn't come close to a hit off Harvey through six innings, their only baserunners on a pair of walks in the third. Finally, Heyward reached safely on perhaps the weakest ball hit off the New York starter all day — a slow dribbler up the first-base line.
Harvey came off the mound to field it and flipped to first base, but there was no one there to catch it. Lucas Duda, making just his second start of the season at first, charged in and left the bag uncovered.
New York padded its lead with two runs in the eighth, just enough to hold off the Braves. In the bottom half, Gerald Laird walked, Uggla reached on a bad-hop single and Andrelton Simmons knocked out Harvey with Atlanta's first clean hit, a sharp single between shortstop and third base.
Jordan Schafer singled off LaTroy Hawkins to bring in two runs, and Heyward's run-scoring double off Scott Rice made it 4-3. After Rice intentionally walked Freeman to load the bases, Parnell came on to strike out Johnson.
Atlanta rookie Alex Wood (0-1) lost in his first career start, lasting only three innings.
NOTES: Harvey eclipsed his previous career high of 12 strikeouts in a May 7 game against the Chicago White Sox. ... Quintanilla snapped an 0-for-22 slump with a seventh-inning single in Game 1. ... New York RHP Scott Atchison, just off the DL, was supposed to take over for Wheeler in Game 2. But the reliever injured his right groin while warming up and left the game without throwing an actual pitch. ... The Mets swept the Braves in a doubleheader for the first time since Sept. 6, 2006, at Shea Stadium.