All Platform-as-a-Service offerings have issues. In this article, the Stack Exchange team detail some of their issues with Azure.
There are still plenty of teething issues left, for example: on the SQL Azure side we can't run cross database queries, add full-text indexes or backup our dbs using the BACKUP command. I am sure these will eventually be worked out. There's also the 30 minute deploy cycle. Found a typo on the website? Correcting it is going to take 30 minutes, minimum.
It's a shame the article doesn't go into more detail about the issues. These would have been instructive for startup founders evaluating Azure as a platform.
It is important to note that these issues are by no means specific to Azure; similar teething issues affect other Platform-As-A-Service providers such as Google App Engine and Heroku. When you are using a PAAS you are giving up a lot of control to the service provider. The service provider chooses which applications you can run and imposes a series of restrictions.
Google App Engine certainly suffers from its share of restrictions.