But with the series now approaching a $1 billion valuation, Ecclestone was happy enough to eat his words when the pair met for a pre-Christmas lunch.
"Bernie in the beginning said, 'This is never going to work,'" recalled Agag. "When I met him for lunch he could say congratulations, very impressive and he said he's happy that he was wrong and I was right. But he was very negative in the beginning. He said, 'Listen, don't do it, you're going to crash.' Luckily, we didn't."
Five years on from its series opener, Ecclestone has been far more receptive as Spaniard Agag starts again from scratch with Extreme E.
The new electric series, to be launched in 2021, has been likened to "Blue Planet meets the Dakar Rally," and is "the future of off-road racing," according to Agag.
The premise of the series is to take electric SUVs to the most remote and extreme locations on the planet, from the destroyed rainforests of the Amazon to the Himalayas, and from the deserts of Saudi Arabia to the rising oceans affecting Senegal to raise awareness of the damage of climate change.
Agag is confident there is the room for another electric series out there.
"My conviction is that there's a huge appetite for Extreme E," he told CNN from Extreme E's London offices.
"The beauty is that all these challenges are connected. In Greenland, we were seeing the melting of the ice caps then in Senegal on the coast we saw families having to migrate as their homes are being destroyed by the rising seas level. That is the same water we saw melting in the Arctic."
Lewis Hamilton to Formula E
The Spaniard is well aware there will be pitfalls. At one stage, Formula E was down to its last $100,000 in the bank while owing $25 million to its suppliers, a pressure that he admits meant he was not sleeping at night. Now, it is into its sixth season and Agag is targeting even bigger names joining the grid, namely six-time F1 world champion Lewis Hamilton, who has recently been pushing environmental issues.
"It would be great to have Lewis in Formula E but I don't see it happening soon," he said. "I'm sure Lewis wants to win more F1 championships, there's some records for him to beat. He's an outstanding driver and also a great personality, plus I think his statements for the environment are really important. He's someone that helps make the world wake up and that's important."
With Extreme E, there have been no shortage of driver suitors. Six-time World Rally Champion Sebastien Ogier has already signed up as an ambassador, while double amputee Billy Monger and ex-Formula E champion Lucas di Grassi are among a long list of drivers to have expressed an interest.
The racing itself sees groups of six teams compete with the top four going through to the knock-out stage with head-to-heads for a place in the final on a course about six miles long with virtual gates to navigate in often extreme conditions.
The series will have a floating grid, the series and its cars traveling from venue to venue via former Royal Mail ship RMS St Helena. But won't it have its own carbon footprint on the planet?
"Extreme E will have a footprint," he added. "But as we say in Spain, if you want to make an omelette you need to break some eggs. We need to take action and we think staying at home is not the solution. We think the benefits of championships like Extreme E far exceeds the emissions a championship like this will generate."
There are plans to replant forests where they race in a deforested part of the Amazon or else help with agriculture projects for local communities in the Himalayas.
'F1 will have to become electric one day'
Agag says he has been pushing green credentials for more than two decades, harking back to his time in the European Parliament when, as he puts it, "I was putting up questions about sustainability when 21 years ago no one cared about sustainability."
"I follow closely what Greta Thunberg and what Extinction Rebellion are doing, and activism is a key part of waking up and the world needs to wake up," he said.
"That role is very important but it's only one part, you also need people like us, businessmen, people that do things and execute their vision. It all helps find a big solution."
The 49-year-old remains chairman of Formula E but has stepped away from his role as CEO to turn his attention to Extreme E. The former has already put pressure on F1, while the latter looks destined to compete directly with events like the Dakar.
"F1 is an amazing brand - the pinnacle of motorsport and I'm a great fan," he added. "But I also think F1 will have to become electric one day.
"The way to do that... I don't know, maybe we'll merge but something needs to happen because I don't think F1 can live long-term without it.
"And Extreme E is the future of off-road racing. Is that going to affect other rally championships, events like Dakar, I don't know. I think there may also be a space for them in the future, I don't know. What I do know, what I believe, what I hope is that there is a big, big space for Extreme E to showcase these locations and these problems."