The 0.5 percent mortgage refi fee is here. What does it mean for you?

The Federal Housing Finance Agency's 0.5% "adverse market" fee on most mortgage refinances is now in place.

If you are in the process of refinancing, it has probably already been priced into your rate. If you are thinking about refinancing, it means your loan will be a little more expensive than it would have a few months ago.

But with mortgage rates still near all-time lows, the fee may not be the end of savings for many borrowers.

Now that it's being implemented, here's everything you need to know.

Who will be charged the fee? The short answer is: almost all refinancers.

The fee will be levied on every mortgage refi valued at $125,000 or more for loans that are ultimately sold to mortgage giants Fannie Mae and Freddie Mac.

Those two companies are responsible for servicing two-thirds of American mortgages, so practically speaking, most lenders will likely price all loans so they are eligible to be sold.

How are lenders applying the fee? For most borrowers, the charge will come in the form of an extra cost at closing or a higher interest rate on the new mortgage.

On a $250,000 loan, the fee would be $1,250. Bankrate's chief financial analyst Greg McBride has recommended that most borrowers pay the fee up front if they can afford to.

Otherwise, it will be charged through a higher interest rate or larger principal loan balance, which will wind up costing you more money over time.

All that makes it even more important to shop around before you settle on a lender for your refi.

Why does the refinance fee exist? Mark Calabria, director of the Federal Housing Finance Agency, said at a conference earlier this fall that the new charge is meant to stabilize Freddie Mac and Fannie Mae's finances after borrowers got $6 billion worth of assistance thanks to coronavirus protections.

Can I avoid the fee? Possibly. If you keep your loan amount below the $125,000 threshold, or above the jumbo mortgage minimums, you won't get charged.

You can also try to find a financial institution, like a portfolio lender, that doesn't plan to sell your loan to Fannie or Freddie, so they won't have to pay the fee.

And, you can look for programs that are exempt from the fee, like the VA and FHA loans mentioned previously.

But, in the grand scheme of things, it may not be worth the hassle. Even with the fee priced in, mortgage rates remain very low, and if you have an existing loan with high interest you still stand to save even after accounting for this extra cost.

The bottom line is that the FHFA refinance fee is here, and it's likely here to stay. Mortgage refinancing is usually a lengthy process with a number of costs on the road to closing.