Now this was big problem as I had to delete some of these channels and the people who created these channels already left the organisation.
I went through a lot of posts online and it clearly said that there was no way we could open channel setting unless we are channel admin. Even global admin could not bypass this.
Being working with Office 365 for so long, I didn't believe this so thought of trying few things out and this one worked.
First open the channel (where you dont have access)and identify the list ID for the channel. It will be chid parameter in the URL of the channel as shown below:
As I created some channels in past, I was channel admin for some of them so I opened Channel setting page for this channel and in the list ID parameter of the URL, put the chldID identified in previous step.
If you are global admin you can open this new URL you have created and change the Channel settings.
Before trying to delete the channel, add yourself as a channel admin else you may get error messages.
Hope this article helps Office 365 admins.