Mayor Bloomberg has said this, and he's been asked to state what he based this on (as others also quote these same figures). Politifact for one asked the mayor's office, and reports the answer on their site. Excerpt:

Bloomberg’s office pointed us to a 1997 study by the National Institute of Justice on who owns guns and how they use them.The researchers estimated that about 40 percent of all firearm sales took place through people other than licensed dealers. They based their conclusion on a random survey of more than 2,500 households.

In 1999, the Bureau of Alcohol, Tobacco, Firearms and Explosives released a report on gun shows. Investigators found that a quarter of the vendors were private sellers, not licensed dealers, and reported that "felons and other prohibited persons who want to avoid Brady Act checks and records of their purchase buy firearms at these shows." They said guns from such shows had been used in drug crimes.

Both of these reports are at least 10 years out of date. We called the ATF and asked if there was anything more recent. They had nothing new to add. We called the National Rifle Association and the National Shooting Sports Foundation, groups that oppose new efforts to track gun transactions. Neither organization responded.

Bloomberg himself has complained about the lack of new research. Josh Horwitz, executive director of the Coalition to Stop Gun Violence, told PolitiFact that no one knows exactly how gun sales break down between the formal and the informal markets. He said there are no data on gun shows, blaming the NRA for opposing regulation of them. Without a paper trail for each transaction, there’s nothing to count.

Legislation briefs posted on one of the NRA’s websites confirm the group’s stand on this point. When New York lawmakers introduced a bill that would have required background checks at gun shows, an NRA columnist warned this would lead to "confiscation of some or all firearms" by creating a more complete list of all gun owners.

The City of New York commissioned an investigation of Internet gun sales. The report said on 10 websites, it found over 25,000 weapons for sale.The report said that over 60 percent of sellers allowed a purchase to move forward even when the alleged buyer said he didn’t believe he would pass a background check. Sellers who used Craigslist were most likely to violate the law, the report said. Our ruling

Mayor Bloomberg said 40 percent of gun sales take place through gun shows or the Internet.

The best information on the informal gun market is based on a survey and is about 15 years old. Current regulations don’t allow direct tallies of sales of this sort. An undercover investigation found a great deal of internet activity, but it was sponsored by a mayor who seeks greater regulation. Groups opposed to greater regulation were asked to rebut the mayor’s assertion and did not respond.