I have followed tutorial on how ISPConfig3 can be installed on debian8. I tried this twice with google cloud machine but I could not access the ISPConfig. I don't really know what is the problem but the only thing i am suspecting is that machine hostname can not be domain name as suggested in the tutorial i.e server1.example.com. but instead i used my registered domain name.
did you assign a external IP for that machine? see https://cloud.google.com/compute/docs/instances-and-network
Can you access other services like ssh on the external IP? If yes, did you open port 8080 tcp in the google firewall? A wrong hostname can not affect the ispconfig login, it will affect only email. You can use any domain as hostname, but it should be a subdomain like server1.yourdomain.tld and not just yourdomain.tld. But as mentioned before, even if you set it wrong then this is not the reason for your current issue and you can correct the hostname later.
You might want to read the following google documentations https://cloud.google.com/compute/docs/troubleshooting#communicatewithinternet https://cloud.google.com/compute/docs/networking#firewalls https://cloud.google.com/compute/docs/networking#addingafirewall