Nonbank lenders in 2014 took their biggest share of the mortgage market since at least 1995, new data show, as several large banks pulled back on lending to all but the most pristine borrowers.

According to federal government data released Tuesday, nondepository independent mortgage companies in 2014 accounted for 47% of loans to buy homes for owner-occupants and 42% of refinancing loans. Those market shares increased from 43% and 31%, respectively, in 2013.

The home-purchase share last year was 12 percentage points higher than in 2010.

The increasing prominence of nonbank lenders came as the total number of mortgages made in 2014 plummeted 31% from the year before to six million, according to the Federal Financial Institutions Examination Council. The drop was driven by a 55% plunge in refinances, which were stifled by higher interest rates. Loans to buy homes in 2014 rose 4% to 3.24 million.

In the past few years, large banks, such as Wells Fargo & Co., J.P. Morgan Chase & Co., and Bank of America Corp., have ceded chunks of the mortgage market to fast-rising nonbank lenders and community banks, partly because of the severe penalties big banks were slapped with for mistakes that had been made during last decade’s housing boom.

Although nonbanks played a larger part in the subprime housing boom and bust, many of those lenders have since closed or been acquired. Fast-rising nonbank lenders, such as Quicken Loans Inc. and loanDepot LLC, nowadays tend to issue mortgages that qualify for government backing from Fannie MaeFreddie Mac, the Federal Housing Administration or the Veterans Administration.

Still, some worry that the rise of nonbanks could introduce more risks into the mortgage market.

Ted Tozer, president of Ginnie Mae, in a speech on Monday said the increasing role of independent mortgage banks has introduced new risks into the system. Ginnie is a government-owned corporation that guarantees bonds backed by mortgages insured by agencies such as the FHA and VA.

“The risk is a lot higher and business models of our issuers are a lot more complex. Add in sharply higher annual volumes and these risks are amplified many times over,” said Mr. Tozer, who is asking for more government funding to keep up with the need to oversee issuers.

Edward Golding, head of the FHA, has said the agency is neutral on whether banks or nonbanks offer loans through its program, as long as FHA-eligible borrowers are able to get loans.

Still, the FHA as well as Fannie and Freddie have taken steps over the last year to try to ease lenders’ liability concerns.

The FHA, which backs loans to borrowers with little money for a down payment, this summer released new rules it hopes will increase banks’ confidence in making loans through its program. Fannie and Freddie have reached agreements with lenders clarifying the mistakes and associated penalties that lenders could face through its program as well.

While Fannie and Freddie’s efforts have been met with approval from lenders, some banks have said the FHA’s efforts haven’t gone far enough.

The changes might be starting to have an impact on lenders’ credit standards. According to a Fannie Mae survey of lenders, in the third quarter about 11% of lenders said they expected their credit standards for Fannie and Freddie-eligible mortgages to ease over the next three months, versus 4% that expected them to tighten.

On the other hand, 8% of lenders said they expected their credit standards on loans backed by the FHA and Veterans Administration to tighten over the next quarter, versus only 7% who said they would loosen. That was the first time more lenders said they’d tighten such loans than loosen them since 2014.

In 2014, large banks accounted for 32% of mortgages to buy homes, according to the FFIEC, down from 34% in 2013 and 42% in 2010. Large banks saw an even greater drop in their share of refinance mortgages, accounting for 38% of such loans in 2014, down from 49% in 2013.

The FFIEC data also showed that minority groups in 2014 began to garner a larger share of the overall mortgage market. Blacks accounted for 5.2% of loans to buy homes last year, versus 4.8% in 2013. Hispanic-white borrowers represented 7.9% of such loans, up from 7.3% in 2013. Both groups had seen a declining share between 2012 and 2013.