Error with Azure service SSL in Development Fabric
I'm running into a problem with getting SSL to work in the Development Fabric. I'm running a clean install of Windows 8 Pro with Visual Studio 2012 Ultimate and the October 2012 Azure SDK for .NET. IIS8 is not installed, only IIS Express, which claims to support HTTPS so I'm hoping that's not the issue.
Running VS 12 as administrator, I've created a blank VS solution, added a new (.NET 4.5) cloud service with a new ASP.NET MVC 4 Internet web application project, and hit F5. Everything works fine. Then, when I add an SSL certificate to the web role and replace the HTTP endpoint (port 80) with an HTTPS endpoint (port 443, with the certificate), hitting F5 produces the following error message:
Windows Azure Tools for Microsoft Visual Studio
There was an error attaching the debugger to the role instance 'deployment18(32).WindowsAzureCloudService.Mvc4WebRole_IN_0' with Process Id: 4892'. Unable to attach. Access is denied.
Note, the last part ("Access is denied") comes in a few variations, a particularly pleasant one being "Catastrophic failure". :)
The only message in the VS Output window ('General' output) is:
Windows Azure Tools: Warning: Remapping private port 443 to 444 in role 'Mvc4WebRole' to avoid conflict during emulation.
The Compute Emulator UI is not much help; just before the instance disappears, this is the only console output that I get consistently (sometimes other messages appear, but sporadically every few runs; I'm not sure how to capture these):
[fabric] Role Instance: deployment18(33).WindowsAzureCloudService.Mvc4WebRole.0
[fabric] Role state Unknown
[fabric] Role state Suspended
[fabric] Role state Busy
[fabric] Role state Unhealthy
[fabric] Role state Stopped
The certificate was obtained from a CA and properly imported into the Local Machine/Personal/Certificates store as a .pfx with private key, extended properties, and marked as exportable, for what it's worth.
When I attempt to publish the service to Azure, I get one build (validation) warning about the database connection string (which I assume is irrelevant):
The connection string 'DefaultConnection' is using a local database '(LocalDb)\v11.0' in project 'Mvc4WebRole'. This connection string will not work when you run this application in Windows Azure. To access a different database, you should update the connection string in the web.config file.
Probably more important, the deployment actually fails with the following history in the Windows Azure Activity Log window:
9:00:25 AM - Warning: There are package validation warnings.
9:00:25 AM - Preparing deployment for WindowsAzureCloudService - 1/3/2013 8:59:55 AM with Subscription ID '<...>' using Service Management URL 'https://management.core.windows.net/'...
9:00:25 AM - Connecting...
9:00:26 AM - Object reference not set to an instance of an object.
9:00:26 AM - Deployment failed with a fatal error
Can someone help me troubleshoot this issue? I've rebooted a few times. ;)
Thanks in advance!
EDIT (Jan. 3, 4:44 PM): I have a few ideas that might help me make progress, but some are pretty drastic so any advice would be appreciated:
- Is there a way to capture all the output from the Compute Emulator (Dev Fabric) to a log file so I can review it? (System.Diagnostic.Trace calls from my service won't help, since I don't even get as far as the RoleEntryPoint when using HTTPS!) I figured this out; see next edit.
- That null pointer exception during the Azure deployment has me worried. Is it worthwhile to try reinstalling the Azure SDK, and if so, how should I go about doing a clean install of it?
- Has anyone seen a problem of this sort disappear when switching to using full IIS for the emulator? (That seems unlikely since IIS vs. IIS Express should have no relevance to the Azure deployment.)
EDIT (Jan. 4, 10:15 AM): Bad news: I tried the suggestion to grant Read access to the certificates, but it didn't help in my case. Good news: I managed to capture one of those sporadic messages in the Compute Emulator UI before it shut down; it was a bit of info from some diagnostics. Not helpful in and of itself, but it revealed where the Development Fabric was storing its temporary files:
[Diagnostics] Information: C:\Users\Lars\AppData\Local\dftmp\Resources\0005155d-4592-40f4-812e-18793b26576c\directory\DiagnosticStore\Monitor
The GUID portion gets recreated for every deployment, and it is deleted when the deployment goes away (as it always does in my case). But in the parent directory ('dftmp'), there are a few helpful directories that I then monitored during a new deployment: DevFCLogs, DFAgentLogs, and IISConfiguratorLogs. I guess that answers the first question I had yesterday! :)
DFAgentLogs\DFAgent.log: (41KB) No useful information. A bunch of "Failure to read pipe" messages and failures to get the role/deployment instance ID, which I assume are just noise.
DevFCLogs\DevFabric--2013.01.04--<...>.log: (510 KB) No useful information. I skimmed the file and also searched for 'error', 'failure', 'not found', 'certificate', and 'Mvc4WebRole_IN_0'; none of those showed any hints of what was going on.
IISConfiguratorLogs\IISConfigurator.log: (6 KB) Now we're making progress!! :) Can someone tell me what this means? (In the meantime, I'm off ILSpy-hunting... fun fun...)
IISConfigurator Information: 0 : [00006356:00000005, 2013/01/04 16:07:08.915] Using IIS Express appdomain
IISConfigurator Information: 0 : [00006356:00000005, 2013/01/04 16:07:08.936] Adding binding 127.255.0.0:444: to site deployment18(40).WindowsAzureCloudService.Mvc4WebRole_IN_0_Web
IISConfigurator Information: 0 : [00006356:00000005, 2013/01/04 16:07:10.484] Caught exception
IISConfigurator Information: 0 : [00006356:00000005, 2013/01/04 16:07:10.487] Exception:System.Runtime.InteropServices.COMException (0x800401F3): Invalid class string (Exception from HRESULT: 0x800401F3 (CO_E_CLASSSTRING))
Server stack trace:
at Microsoft.Web.Administration.ConfigurationElement.GetPropertyValue(IAppHostProperty property)
at Microsoft.Web.Administration.BindingCollection.Add(Binding binding)
at Microsoft.WindowsAzure.ServiceRuntime.IISConfigurator.WasManager.DeploySite(String roleId, WASite roleSite, String appPoolName, String sitePath, String iisLogsRootFolder, String failedRequestLogsRootFolder, List1 bindings, List1 protocols, FileManager fileManager, WAAppPool defaultAppPoolSettings, String roleGuid, String& appPoolSid, List`1 appPoolsAdded, String configPath)
EDIT (Jan. 4, 11 AM): ILSpy wasn't much help; the exception is being thrown at an interop point (we knew that already) while trying to get the hash of a certificate in order to set up the binding (we knew that too). Does anyone know what COM object would need to be registered in order to get a certificate hash for a binding in Microsoft.Web.Administration? Or how I could intercept the interop call to find out? Bonus points if you can tell me why this is happening in the first place. :)
When running locally with a private key cert for SSL, you'll need to give the user the emulator app is running under access to the private key. Open mmc.exe and add the Certificates >> Local Computer Snap-In to view your certificate. Right Click on the certificate, then All Tasks >> Manage Private Keys - then add IUSR and Network Service with at least read access.
For deployment to azure, you'll need to upload the certificate to the Cloud Service and make sure the certificate is valid for the domain.
Follow step 11 from http://www.microsoft.com/en-us/download/details.aspx?id=35448. From this SO post
I've had similar problem on two computers. On both cases installing IIS solved the problem.
It seems to be enough to just install the IIS (via add/remove Windows components). You don't need to start using it. The installation changes something and after that my IIS Express started working again with HTTPS from Visual Studio.
There is a discussion on similar issue on MSDN Social: http://social.msdn.microsoft.com/Forums/nl-NL/windowsazuredevelopment/thread/ad362016-16f6-459a-8022-9307aa5f910e And the issue has been also raised on Microsoft connect: https://connect.microsoft.com/VisualStudio/feedback/details/758533
In my case the error in the log files was:
IISConfigurator Information: 0 : [00007644:00000007, 2013.01.17 00:39:18.523] Exception:System.Runtime.InteropServices.COMException (0x800401F3): Invalid class string (Exception from HRESULT: 0x800401F3 (CO_E_CLASSSTRING))
I found the log files from C:\Users\\AppData\Local\dftmp\IISConfiguratorLogs directory.