13/09/2015 2:04 PM AEST | Updated 15/07/2016 12:51 PM AEST

Peter Dutton Apologises For 'Vulgar' Joke

Stefan Postles via Getty Images
CANBERRA, AUSTRALIA - DECEMBER 23: Minister for Immigration and Border Protection Peter Dutton is sworn in by Governor-General Peter Cosgrove at Government House on December 23, 2014 in Canberra, Australia. Australian Prime Minister Tony Abbott today announced his first ministerial reshuffle to Governor-General Sir Peter Cosgrove. (Photo by Stefan Postles/Getty Images)

Immigration Minister Peter Dutton has apologised "if anyone took offence" at his joke about rising sea levels, a serious threat facing small nations in the Pacific Ocean.

On Friday Mr Dutton was caught laughing with Prime Minister Tony Abbott on a live microphone about a joke he made about a meeting running late, referring to "Cape York time" and that "time doesn't mean anything when you're about to have water lapping at your door."

Mr Abbott had just attended a meeting in Port Moresby with Pacific Island leaders and had lengthy talks about the potentially disastrous effects of climate change on Pacific Nations.

Today Mr Dutton told Sky News that he was sorry for the joke, which has been called "lame" by Tony Abbott, and "vulgar" by Kiribati President Anote Tong.

"Obviously it was a private conversation and I should have realised the mic was there. I didn't. It was directly behind me," Mr Dutton told Sky News.

"I made a mistake and I apologise to anyone who has taken offence to it. It was a light-hearted discussion with the PM and I didn't mean any offence to anyone."

Deputy Labor leader Tanya Plibersek told Barrie Cassidy on ABC's Insiders today: "He apologised for being caught. I don't think that should be the end of the matter at all. He should be apologising to Pacific leaders and to Aboriginal Australians in Cape York. He seems to have insulted millions of people in one go and he hasn't apologised for that."

Opposition Leader Bill Shorten has called for Dutton to be axed from Cabinet.

Yesterday Tony Abbott said the gaffe was a "lame joke."

He was defending Mr Dutton, insisting our national focus should have been on the decision spearheaded by Mr Dutton to take an extra 12,000 refugees from Syria.

While on the campaign trail ahead of the Canning by-election in Western Australia, Mr Abbott told reporters: "It says something about the quality of our national conversation that a lame joke should be of more interest than something which is truly nation building and truly transforming which reflects Australia at its best.

"Here we made a decision which reflects Australia at its best, and then we have a subsequent Twitter storm, which if I may say so reflects Australia at its worst."

Yesterday Gerhardt Pearson, Cape York traditional owner, said in a statement the reference to "Cape York time" was aimed at portraying Aboriginal people as "lazy good-for-nothings".

"We are constantly burdened with the view espoused by the likes of Dutton; it's one of soft bigotry and low expectations, and it continues to dominate policy responses," he said.