The Biggest Loser Doctor Robert Huizenga Says He ‘Really Took Issue’ with Some of the Show’s Tactics

Dr. Robert Huizenga, a celebrity doctor who appeared regularly on The Biggest Loser, is reflecting on his concerns regarding contestants’ health and safety as the cameras rolled on the NBC reality series for more than a decade.

In Netflix’s three-part docuseries Fit for TV: The Reality of The Biggest Loser, which premiered Friday, August 15, Dr. Huizenga, a former team doctor for the then-Los Angeles Raiders, said his experiences at the intersection of medicine and the professional sports world helped shape the weight-loss reality show.

“The Biggest Loser was totally sculpted after my experience with the Raiders and the only question was is it possible for an overweight, sedentary person to work out, not with the ability of a professional athlete, but with the intensity of a professional athlete?” Dr. Huizenga says on the show.

Still, as the series progressed and showrunners faced pressure to maintain viewers’ attention through 17 seasons, he says he became increasingly worried about the physical challenges contestants faced.

“We had a 99.9% success [rate] in losing weight. That was really an incredible discovery,” Dr. Huizenga recalls. “But as the show really got popular, some things were being done that I really, I really took issue with.”

“It was scary because from season 1, you have women — 200 lbs., men — 300, and it just went up, straight line up. And with that, the ability to exercise, a straight line down. So when they do a physical challenge that puts people in harm’s way, that’s where I have to step in,” he says.

Looking back, Dr. Huizenga said he was not always aware of every aspect of the contestants’ experiences.

“There were times, unfortunately though, that challenges were done that I didn’t see or hear about, which was the main hang-up to making it a much safer environment,” Dr. Huizenga said.

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

In particular, he alluded to differences in perspective between himself and the show’s trainers, Bob Harper and Jillian Michaels.

“Every season, I pretty much said I was going to quit unless I got to talk to the trainers,” he claims. “So they would force them to come in and I’d give my little half an hour speech and maybe some things changed, not as much as I would like.”

PEOPLE reached out to NBC for comment on the claims made in Fit for TV: The Reality of The Biggest Loser, but did not receive a response.

Fit for TV: The Reality of The Biggest Loser is streaming now on Netflix.