SCCM 1602 Application Catalog is broken after Windows Server Inplace Upgrade

25 בMay 2016

as you are probably aware, according to TechNet, since SCCM Current Branch version 1602 was published, it is supported to upgrade the site server from Windows Server 2008 R2 to Windows Server 2012 R2 by using in-place upgrade installation.

after upgrading SCCM primary site to Current Branch version and running the in-console upgrade to 1602, I started with the Windows Server in-place upgrade.

when eventually succeeded with the Windows Server upgrade and was finally on the latest version of all components I’ve performed tests to verify everything stayed where I left it before the upgrade.

one component after another I was proudly marked as a success, but when I started to check the Application Catalog, the errors were everywhere.

Cannot connect to the application server.

the website cannot communicate with the server. this might be a temporary problem. try again later to see if the problem has been corrected. if this problem continues. contact your help desk.

Application Catalog error

Cannot connect to the application server

When looking in the Site Component node, I could see that the SMS_AWEBSVC_CONTROL_MANAGER possess a warning sign which was in fact a several error notifications with the following description:

Application Web Service Control Manager detected AWEBSVC is not responding to HTTP requests.  The http status code and text is 500, Internal Server Error.

Possible cause: Internet Information Services (IIS) isn’t configured to listen on the ports over which AWEBSVC is configured to communicate.
Solution: Verify that the designated Web Site is configured to use the same ports which AWEBSVC is configured to use.

Possible cause: The designated Web Site is disabled in IIS.
Solution: Verify that the designated Web Site is enabled, and functioning properly.

awebsctl.log

awebsctl.log

in the  awebsctl.log i witnessed the following errors:

Call to HttpSendRequestSync failed for port 80 with status code 500, text: Internal Server Error    SMS_AWEBSVC_CONTROL_MANAGER    24-May-16 20:57:19    3840 (0x0F00)
AWEBSVCs http check returned hr=0, bFailed=1    SMS_AWEBSVC_CONTROL_MANAGER    24-May-16 20:57:19    3840 (0x0F00)
AWEBSVC’s previous status was 1 (0 = Online, 1 = Failed, 4 = Undefined)    SMS_AWEBSVC_CONTROL_MANAGER    24-May-16 20:57:19    3840 (0x0F00)
Health check request failed, status code is 500, ‘Internal Server Error’.    SMS_AWEBSVC_CONTROL_MANAGER    24-May-16 20:57:19    3840 (0x0F00)
STATMSG: ID=8100 SEV=E LEV=M SOURCE=”SMS Server” COMP=”SMS_AWEBSVC_CONTROL_MANAGER” SYS=SCCM.U-BTech.com SITE=SIL PID=2316 TID=3840 GMTDATE=Tue May 24 17:57:19.437 2016 ISTR0=”500″ ISTR1=”Internal Server Error” ISTR2=”” ISTR3=”” ISTR4=”” ISTR5=”” ISTR6=”” ISTR7=”” ISTR8=”” ISTR9=”” NUMATTRS=0    SMS_AWEBSVC_CONTROL_MANAGER    24-May-16 20:57:19    3840 (0x0F00)

Beginning my journey through the web, I ‘ve found a lot of suggestions to reinstall the roles, IIS and .Net but these didn’t provide the solution.

after few hours of trying to solve in multiple paths I’ve bumped into this post that pointed me to see what is happening when I try to access the web service itself via the web.

on the primary site (which holds both roles of Application Catalog) I’ve navigated to  http://localhost/CMApplicationCatalogSvc/applicationofferService.svc

and got an error web page that ended with a problem description:

Server Error in ‘/CMApplicationCatalogSvc’ Application.

Could not load type ‘System.Runtime.Diagnostics.ITraceSourceStringProvider’ from assembly ‘System.ServiceModel.Internals, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’.
  Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.TypeLoadException: Could not load type ‘System.Runtime.Diagnostics.ITraceSourceStringProvider’ from assembly ‘System.ServiceModel.Internals, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. 

Stack Trace:

[TypeLoadException: Could not load type ‘System.Runtime.Diagnostics.ITraceSourceStringProvider’ from assembly ‘System.ServiceModel.Internals, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’.]
   System.ServiceModel.Channels.TextMessageEncoderFactory..ctor(MessageVersion version, Encoding writeEncoding, Int32 maxReadPoolSize, Int32 maxWritePoolSize, XmlDictionaryReaderQuotas quotas) +0
   System.ServiceModel.Channels.HttpChannelListener..ctor(HttpTransportBindingElement bindingElement, BindingContext context) +164
   System.ServiceModel.Channels.HttpChannelListener`1..ctor(HttpTransportBindingElement bindingElement, BindingContext context) +30
   System.ServiceModel.Channels.HttpTransportBindingElement.BuildChannelListener(BindingContext context) +173
   System.ServiceModel.Channels.DatagramChannelDemuxer`2..ctor(BindingContext context) +110
   System.ServiceModel.Channels.ChannelDemuxer.GetTypedDemuxer(Type channelType, BindingContext context) +304
   System.ServiceModel.Channels.ChannelDemuxer.BuildChannelListener(BindingContext context, ChannelDemuxerFilter filter) +61
   System.ServiceModel.Channels.ChannelBuilder.BuildChannelListener() +73
   System.ServiceModel.Channels.ChannelBuilder.BuildChannelListener(MessageFilter filter, Int32 priority) +119
   System.ServiceModel.Security.SecuritySessionServerSettings.CreateInnerChannelListener() +297
   System.ServiceModel.Channels.SecurityChannelListener`1.InitializeListener(ChannelBuilder channelBuilder) +103
   System.ServiceModel.Channels.SymmetricSecurityBindingElement.BuildChannelListenerCore(BindingContext context) +1863
   System.ServiceModel.Channels.SecurityBindingElement.BuildChannelListener(BindingContext context) +336
   System.ServiceModel.Channels.Binding.BuildChannelListener(Uri listenUriBaseAddress, String listenUriRelativeAddress, ListenUriMode listenUriMode, BindingParameterCollection parameters) +177
   System.ServiceModel.Description.DispatcherBuilder.MaybeCreateListener(Boolean actuallyCreate, Type[] supportedChannels, Binding binding, BindingParameterCollection parameters, Uri listenUriBaseAddress, String listenUriRelativeAddress, ListenUriMode listenUriMode, ServiceThrottle throttle, IChannelListener& result, Boolean supportContextSession) +16929042
   System.ServiceModel.Description.DispatcherBuilder.BuildChannelListener(StuffPerListenUriInfo stuff, ServiceHostBase serviceHost, Uri listenUri, ListenUriMode listenUriMode, Boolean supportContextSession, IChannelListener& result) +572
   System.ServiceModel.Description.DispatcherBuilder.InitializeServiceHost(ServiceDescription description, ServiceHostBase serviceHost) +1908
   System.ServiceModel.ServiceHostBase.InitializeRuntime() +90
   System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout) +175
   System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout) +740
   System.ServiceModel.HostingManager.ActivateService(ServiceActivationInfo serviceActivationInfo, EventTraceActivity eventTraceActivity) +125
   System.ServiceModel.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity) +901

[ServiceActivationException: The service ‘/CMApplicationCatalogSvc/ApplicationOfferService.svc’ cannot be activated due to an exception during compilation.  The exception message is: Could not load type ‘System.Runtime.Diagnostics.ITraceSourceStringProvider’ from assembly ‘System.ServiceModel.Internals, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’..]
   System.Runtime.AsyncResult.End(IAsyncResult result) +622882
   System.ServiceModel.Activation.HostedHttpRequestAsyncResult.End(IAsyncResult result) +196075
   System.Web.AsyncEventExecutionStep.OnAsyncEventCompletion(IAsyncResult ar) +166

Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.0.30319.34009

From that point, I searched for the error: “System.Runtime.Diagnostics.ITraceSourceStringProvider’ from assembly ‘System.ServiceModel.Internals” and that led me to this KB article by Symantec  which pointing out that installing .Net  4.5.2 will fix this specific error message. I wasn’t satisfied and searched for another clue and found this blog post that mentioned a broken .net version after in-place upgrade to Server 2012 R2.

And that’s the clue I was seeking to so much because I know that SCCM 1602 installs .Net 4.5.2 during it’s upgrade (that happened on the Windows Server 2008 R2 version)

After installing .Net framework 4.5.2,  rebooting the server I still had that same error message.

Again, browsing to  http://localhost/CMApplicationCatalogSvc/applicationofferService.svc resulted in providing me a lead to an error in a line (which can be a result of one of my troubleshooting steps before writing these words) fixing a line in the file: “C:\Program Files\SMS_CCM\CMApplicationCatalogSvc\Web.config”

from:

<serviceHostingEnvironment>
<baseAddressPrefixFilters>
<add prefix=”HTTP://SCCM.U-BTech.COM:80“/></baseAddressPrefixFilters>
</serviceHostingEnvironment>

To:

<serviceHostingEnvironment>
<baseAddressPrefixFilters><add prefix=”HTTP://SCCM.U-BTech.COM:80″/></baseAddressPrefixFilters>
</serviceHostingEnvironment>

And performing IISReset was the last piece in that puzzle that fortunatly solved the problem

It is important to mention the as part of my troubleshooting steps i reinstalled  .net features by the command:  dism /online /enable-feature /featurename:IIS-ASPNET45

why not  “%windir%\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis.exe -i –enable”?

because  this command is not supported on Windows Server 2012 R2

Hope it will help the rest of us in-place upgraders and belivers

Regards,

Ori Husyt

System Center MVP

Windows  Deployment & Management Solutions Specialist @ U-BTech Solutions

הוסף תגובה
facebook linkedin twitter email

Leave a Reply

Your email address will not be published. Required fields are marked *