Moving IIS7+ configuration to another IIS7+ server

We have had to move config from one server to another, and we have discovered a general procedure for achieving this. We were just trying to copy over the applicationhost.config, but found sometimes the application pools would fall over.

Now as you might be aware, the majority of IIS7 website config is held in the applicationhost.config file, usually located in the

<windows folder>\system32\inetsrv\config

folder. There are instances where the config will be configured in web.config files, in the root of the website\virtual directory physical folders. But I won’t go into that here.

You can just copy over the applicationhost.config file to the new server, but you will have problems with passwords as the encrypted machine keys will not match. We need to export from the original sever and import into the new server. You can do this.

aspnet_regiis.exe found here

c:\windows\Microsoft.NET\Framework64\v2.0.50727\

Exporting

aspnet_regiis.exe -px "iisConfigurationKey" C:\temp\iisConfKey.xml -pri
aspnet_regiis.exe -px "iisWasKey" C:\temp\iisWasKey.xml -pri

Copy files to new server.

Importing (use -exp so the keys can be exported again)

aspnet_regiis.exe -pi "iisConfigurationKey" c:\temp\iisconfkey.xml -exp
aspnet_regiis.exe -pi "iisWasKey" c:\temp\iisWasKey.xml -exp

This should allow the application pools to properly function.

Moving IIS7+ configuration to another IIS7+ server

IIS7 and tomcat– basic setup

Every now and again I have to setup IIS and tomcat, this usually involves a gap of a year or so, so for my own sanity I have made some notes on how it sits together. Most of the issues involve jumping IIS versions.

I am assuming that the tomcat service is configured correctly, and I will only detail the tomcat to IIS setup.

We must download the ISAPI_rediect.dll ISAPI filter for IIS. ensure you have the correct version for your architecture (x86/x64).

http://apache.mirror.aussiehq.net.au//tomcat/tomcat-connectors/jk/binaries/

Extract the files into a location on the webserver, you will pointing a virtual directory at this location further on.

I love diagrams as it helps me visualise what is going on. So I have drawn a diagram to aid this

IIS

To get tomcat and IIS communicating we have to load an isapi filter into IIS that will handle the communications.

In my example I will be applying the isapi filter to a specific website in IIS, however you could if you wish add it at the server level and it will handle all sites within IIS.

First we have to tell IIS to allow the ISAPI filter. This is down at the server level within the ISAPI and CGI Restrictions.

Ok so now we go the site we want to allow the ISAPI filter to run on. And go to the ISAPI filters.

While still in the website in IIS, we need to add a virtual directory called Jakarta and point it to the path of isapi_redirect.dll.

Now click on the virtual directory and go to Handler Mappings,  click “Edit Feature Permissions” and check “execute”

Ok so we have configured IIS. We now have to configure the properties files that are used by the dll.

Now the redirector can utilise the registry or configuration files. I will cover the config files as this allows you to run multiple redirectors with various configs.

Now in the same folder there should be a file called isapi_redirect.properties. This file provides the filter with its initial config.

isapi_redirect.properties
It gives the virtual directory location of the DLL withiin the website.
extension_uri

logfile location and log level
# Full path to the log file for the ISAPI Redirector
log_file=<physical path>\logs\isapi_redirect.log

# Log level (debug, info, warn, error or trace)
log_level=info

And the path to a couple of other properties files

# Full path to the workers.properties file
worker_file=<physical path>\conf\workers.properties.minimal

# Full path to the uriworkermap.properties file
worker_mount_file=<physical path>\conf\uriworkermap.properties
rewrite_rule_file=<physical path>\conf\rewrites.properties

As you can see I have placed the config files into a subfolder conf and the logs in logs folder, but they can exist anyway really.

So the dll uses this initial config file to provide further setup details.

rewrites.properties
I will ignore the rewrites.properties file as I don’t use it, but it basically allows you to rewrite the url.

uriworkermap.properties
This file tells the extension (isapi_redirect.dll) what constitues a valid request to pass to tomcat. In my case I want everything sent to the website

/*=worker1

So the wildcard means send everything to worker1. No whats worker1?

workers.properties.minimal

The worker is the process that actually communicates to the tomcat process. The workers.properties.minimal file contains the setup for the worker processes. I have only one called worker1 and it set up to call tomcat using ajp13 on port 8009 (usual port).

worker.worker1.type=ajp13
worker.worker1.host=localhost
worker.worker1.port=8009

The worker will send the request to tomcat and then handle the response back to IIS.

Once these have all been configured, ensure the tomcat process is running. Restart the website in IIS and everything should work.

I reference these blogs/pages which provide further excellent info.

http://jspors.blogspot.com/2009/01/setting-up-64-bit-tomcat-6-on-iis7.html

http://tomcat.apache.org/connectors-doc/reference/iis.html

http://apache.mirror.aussiehq.net.au//tomcat/tomcat-connectors/jk/binaries/

IIS7 and tomcat– basic setup

Where has my memory gone? or locating that leak…

A few days ago I had to deal with an ColdFusion server that had stopped server page requests. The server could be contacted and using RDP it could be controlled however any request for a ColdFusion page was met with nothing.

After looking around and finding nothing obvious the CF service was restarted. This solved the problem, however I did notice that the amount of used memory had not reduced fully. The machine itself has Windows 2008 R2 with 32GB of RAM with CF set to take up 20GB, usually on a CF restart the amount of used memory would drop to a couple of gig. However used memory was still at 17Gigs. I started task manager and looked at the running processes, nothing was taking up 17Gigs worth of memory. Where had this memory gone? Then I noticed the non paged pool memory setting, nothing I usually look at but lo and behold it was sitting at around 17Gigs.

So what was this and how do I fix it. To cut a long story short, after some research I eventually established that this type of behaviour was indicative of a memory leak (memory being used by a process/driver but not released).

So while the restart of the CF server had rectified the issue in the short term, with 17Gig still assigned and unusable CF was going to run into the same problem again (and if the leak kept on growing each time CF restarted there would be  less and less time in between each restart).

The machine was rebooted, this gave us back 17Gigs, and of course gave us longer. In the meantime I set about trying to track down the memory leak.

The screen shots below shows the NonPaged pool size (bottom left 3088MB), not with the 17Gig but with 3Gigs (it was sometime after the reboot).

So how to track down what was causing the memory leak. Now this is when I came across the excellent Mark Russinovich’s Blog entry

http://blogs.technet.com/b/markrussinovich/archive/2009/03/26/3211216.aspx

Mark Russinovich is a god among Windows people, his sysinternals tools are legendary and since joining Microsoft he has kept up this good work.

The article is a good read and I recommend readin it all, but the section we are interested in is Tracking Pool Leaks.

This article led me onto poolmon, it was a pain to get as I had to download the Windows Driver Kit from Microsoft, although I only needed the poolmon tool. Why microsoft don’t allow this to be downloaded seperatly I don’t know. ANyway I downloaded and installed the WDK locally and then copied the poolmon.exe to the offending server.

On running poolmon you get the following

Ok, now you can go and read up about all the bits and pieces here, as I am not going to go into (as I don’t fully understand it all). But there are a number of ways to filter and order the list.

Press B to order list by Bytes

Press D to order the list by Diff

Press F to order list by Frees

Press A to order the list by Allocations

Press P to toggle between listing Paged, NonPaged and Paged & Non Paged

OK, now with the above commands I was able to track down my memory leak to a specific driver. I listed only Nonpaged (pressing P a few times) and then ordered by Bytes. Now a memory leak is identified by a lot of Allocation and a free value that doesnot correlate (ie. not a lot of frees compared to allocations, memory is allocated but not freed). As you can see from my list the Tag BLFP stands out instantly, now as I understand your memory leak may or may not be as obvious as this, and if thats the case I believe you need to track figures this is covered in Marks Russinovich’s article above.

Now with the TAG code identified I did a look on the net, as you will see from Mark’s article you can search the system for the offending driver which I was going to do, but thought I would give google a go, and for once I was lucky and stumbled across someone who had gone through the same issue.

The fault lied with the Broadcom Virtual Network Adapter used for teaming the NICs together on A Dell R710. So in the end all I needed to do was to download the updated version of the Broadcom Management Application that would up date this virtual adapter driver.

I hope that makes sense, I have tried to summarise, simplify and explain what I did. I took a shortcut in not tracking down the driver and googling, so you may have to go and do some work here but hopefully this will get some people started.

Update: To get the update for the Virtual Adapter you don’t want to download the physical adapter drivers. You need to download the Broadcom Management Application Installer, this contains the virtual adapter driver. Link below is shortcut to download page (skip past the physical adapter downloads, unless of course you want to update those to.)

http://www.broadcom.com/support/ethernet_nic/netxtremeii.php

Cheers

Where has my memory gone? or locating that leak…

ShrewSoft / Cisco VPN Client– periodic disconnection

I recently had to setup the shrewsoft vpn connection for work, this was maninly due to the fact they were rolling out Windows X64 and the cisco vpn client did not support the x64 environment.

Now the shrewsoft vpn client is great, and all I did was import the cisco *.pcf files from the 32bit vpn client. And connectivity was established.

However of the two connections I setup I found that while they initially connected, after a set amount of time one the connections would disconnect with the error

“Connection terminated by gateway”

This was a tad annoying, and I could see no cause. However I eventually managed to establish that the timeout matched exactly to the setting on the firewall.

After trawling the internet, not really understanding what was going on, although pretty sure this setting was the culprit as the times matched (1000 seconds, 16 min 40 seconds), I stumbled across this.

http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a00807e0aca.shtml#solution06

What it eventually came down to was a mismatch in values between the firewall and the client. Now as far as I am aware this disconnection will only occur if the lifetime value on the client is greater than the firewall. (I don’t really understand why it connects once and then craps out).

Now the solution is to set the lifetime value in the client to the same value or less than on the firewall. This then means when the period is reached the client and the firewall can negotiate a new key.

Here is where the setting is in the Shrewsoft vpn client.

here is the setting in the cisco firewall

isakmp policy 1 lifetime 1000

ShrewSoft / Cisco VPN Client– periodic disconnection