Workers who take out 401(k) loans risk not having enough saved for retirement because they miss out on growth while the money is borrowed. Some may also reduce their contributions or stop them altogether, research shows.

Internal Revenue Service rules say you can borrow up to $50,000 or 50 percent of the account balance, whichever is greater.

This ability to cash out some portion of your retirement account balance is unique to 401(k) plans. You cannot borrow against an Individual Retirement Account or a pension, for instance.

The problem is with middle-aged workers, who are the heaviest loan users, according data from the Employee Benefit Research Institute. The overall average of loans has hovered between 18 and 20 percent for the last few years; about 27 percent of participants in their 40s had a loan balance in 2013, the last year of EBRI's data. Workers can take out money as withdrawals without penalty after age 59 1/2.

Among developed countries with private retirement systems, the United States is alone in allowing basically unrestricted access to cash without providing proof of a hardship, according to a recent study led by Brigitte Madrian, a professor at Harvard's Kennedy School of Government.

In fact, loans were used to entice workers dependent on pension plans to enroll in 401(k)s when they were introduced in 1981. A study by EBRI Research Director Jack VanDerhei in 2001 showed the loan option made a big difference in how much a person was willing to contribute.

But that was before the financial crisis of 2008 and before the age of auto-enrollment. Today's under-40 generation does not pay much attention to the details of retirement plans they get at work, and it is unlikely that any change would prompt them to start opting out in huge numbers, VanDerhei says.

Huge consequences

While it is alarmingly simple to borrow from your 401(k), borrowers may sometimes have to pay setup fees. The low interest rate charged is actually credited back to your own account as you repay.

The consequences in lost growth, however, can be monumental. Fidelity Investments estimates that a person who takes one loan out — the average balance they see is $9,000 — is set back about 7.6 percent from his or her long-term retirement goal.

Half of Fidelity's borrowers end up with more than one loan. The real-dollar impact is between $180 and $650 a month in retirement.

It is not just the loan balance that affects the retirement account. Of the 20 percent who borrow, Fidelity has found that 25 percent lower their savings rates within five years of taking a loan, and another 15 percent stop saving altogether while the debt is outstanding.

A more serious problem is with those who have an outstanding balance when they lose or change jobs. They must repay their loans immediately or face tax penalties on top of credit problems.

Legislation to change 401(k) loan provisions is unlikely at this point. "It would be easier if you had some companies get rid of the option and show the employees were better off," Harvard's Madrian said. "Absent some more compelling data, it's going to be hard to shift the policy landscape on that front."

Beth Pinsker writes for Reuters.