U.S. retail sales will grow about 4 percent this holiday season, buoyed by a better job market, according to a forecast by Deloitte. Still, the news isn't all good for big retail chains.

Consumers are increasingly buying gifts at niche retailers, potentially pulling spending away from places like Wal-Mart and Macy's, Deloitte said in an annual report released on Wednesday. Upstart companies — think Bonobos or Fabletics — have collectively steered $200 billion in annual sales away from big chains over the last five years, the consulting firm estimates.

"You've got a lot of small startups that are nibbling away a little bit of the volume at a time," said Rod Sides, vice chairman of Deloitte. "Everyone's worried about the Amazon effect and those kinds of players, but at the end of the day, it's the smaller websites that focus on a specific product category and whittle away at market share."

The shift means traditional retailers will have to work harder over the holidays to entice customers. Even so, gains in wages and disposable income will help put more money in shoppers' wallets. Excluding motor and gasoline purchases, sales may exceed $1 trillion in the November-through-January season, New York-based Deloitte said.

Holiday sales by that measure rose 3.6 percent last year, according to the firm. Online purchases will grow much faster than the overall figure, jumping as much as 19 percent to $98 billion during the 2016 holiday.

U.S. shoppers may be feeling more cheery as economic growth improves and the unemployment rate remains low. An increase in average hourly earnings also may boost consumers' willingness to spend this holiday. The U.S. personal savings rate fell to 5.7 percent of disposable personal income in July, according to Commerce Department data, down from 6.2 percent in March.

"People are still saving more, but just not as much as they were in the past," Sides said. "They feel more confident about their employment and their wages."

It's also gotten easier to start a retailer, especially an e-commerce shop, Sides said. To keep consumers coming back, major chains will have to sharpen their price points and offer more compelling products, he said.

To prepare for the holiday rush, retailers are making their annual push for seasonal workers. Target and Kohl's have said they plan to hire the same number of temporary employees as last year, with both adding about 70,000 apiece. Macy's, meanwhile, is reducing its staff to 83,000, down about 2,000 from last year.

While an unexpected spike in gas prices or a dip in employment could hurt spending, one thing retailers don't need to worry about is the presidential election in November, Sides said. The campaigns could disrupt early-season purchasing, but shoppers historically return to spending after the election results are in.

"It really doesn't have a long-term effect," he said.