Sometimes you need to remove a content database from the Farm. Either to move it to another web application or database server or when dealing with an elaborate restore scenario. Prior to running the deletecontentdb command you need to run preparetomove on the database you are going to detach from SharePoint.

This ensures that the relationship between the database and the SSP will be severed and ensures that you can cleanly reattatch it. Failing to run the preparetomove command may result in the deletion of user membership metadata including the selections made in the Privacy and Grouping section on the Edit Profile page for My Site memberships.

STSADM –o preparetomove –contentdb <database server name:database name> -site <site url>

You can also undo the last preparetomove action by using the command undo, this will revert the changes and connect the database to the SSP again.

STSADM –o preparetomove –contentdb <database server name:database name> -site <site url> -undo

Note that the deletecontentdb does not actually delete the database nor does it detach the database in SQL Server, it only detaches the database from the SharePoint Farm Configuration.

STSADM -o deletecontentdb –url <site url> -databasename <database name> -databaseserver <database server name>

Then after you have moved the databse in/from SQL Server you can add it to the Web Application or a new Web Application by using the following command.

STSADM –o addcontentdb –url <site url> -databasename <database name> -databaseserver <database server name>

When moving databases in SQL always detach them from SharePoint with the deletecontentdb command and always use the preparetomove command prior to using the deletecontentb command (or shut down the entire farm). 

It has been a while since i have been posting on here. This was mainliy due to a busy schedule and vacation time hitting. However something curious has happened when I was installing Windows 2008 Server on a system for a client.

The server had given the client some problems and it was reinstalled a few times prior to my involvement. And when i tried to install the 64bit version of Server 2008 it would not recognize any of the disks attached to the system. At first i thought it was a problem with the drivers for the controller and downloaded the latest (still old mind you) drivers for it from the vendor site.

These however could not be loaded at all, the system gave a message about an invalid certificate. And i suspect that the certificate used to sign those drivers has expired and an update had not been posted to the vendor’s site. When testing the 32bit version of the OS it would run the installer flawlessly. I guess due to the ability to install unsigned drivers and would thus even accept the expired certificate.

Somewhere between calling the vendor outright (on a saturday?) and demanding a new driver, and opening up the WinPE environment on the setup disk and disabeling the requirement for signed drivers when installing 64bit OSses i got a peek at the disk configuration. And saw that for some reason there was a software mirror definded on the disks (the system had a good RAID controller so why this would be configured in Windows is beyond me but there it was). 

When the mirror configuration and partition information was removed the Windows 2008 Setup PE environment was able to see the disks as normal and would install without problems and without the need for the driver. So for next time when installing a system, i would reccomend to check for software RAID configurations on the disks and remove those when this problem pops up.

We had a Windows 2008 Server Network Load balancing setup  consisting of two Windows 2008 Servers wich had two NIC’s each. The Load Balancing was configured to use the one of the two Interfaces for dedicated Load balancing, the other interfaces were configured for normal network traffic as shown in the picture below.

NLB hardware Configuration

The virtual cluster IP address responded if you send it a ping from another host on the same subnet that the cluster was installed on. However, if you were on another segment of the network  it didn’t respond.

Originally we were send off track due to an issue with the vendor’s NIC Teaming solution that was enabled by default and had proved difficult to remove.

This however proved not be the cause of our problem with the NLB configuration not responding to off-subnet network locations.

What did prove to be the case is that in Windows 2008 Server NLB by default IP Forwarding is not enabled. This is the feature of Windows networking that, in the context of NLB, allows responses to requests sent to one NIC to be routed out the other. It can be enabled by using a netsh.exe command.

“netsh interface ipv4 set int “[name of the NIC]” forwarding=enabled”

The command will respond with an “Ok.” and no reboot is required for the configuration to apply. This command has to be run on both machines and only for the interface configured for normal network traffic.

In this case the NLB became active immediately  after running the commands on both machines. This little command is something that is good to remember when installing Windows Server 2008 NLB solutions and I for one will add it to my checklist.

When installing a Windows 2008 Failover Cluster and for whatever reason the creation of the Virtual Computer Objects, Cluster Name Object and the dns records fail you will have to create these manually.

For example you have Server-A and Server-B and create a two node Failover Cluster with these named Cluster-A and you create a Virtual Computer Object named Application-A.

In Active Directory you will need to create a Computer Account for the Cluster-A CNO (Cluster Name Object) and another for the Virtual Computer Object Application-A. Furthermore you will have to give the CNO (Cluster-A$) Full Control permissions on the ACL on the Application-A VCO.

The same has to be done for the dns entries, where the CNO (Computer Account in Active Directory) must have permissions on the dns entries. It might seem strange since this account is a Computer Object but without these permissions and records the cluster will not start the Service. 

Since Microsoft has deprecated the “RequireDNS” option, this would otherwise prevent you from serving your clustered application. Note that the private property “RequireDNS” still exists on a Windows Server 2008 Failover Cluster for backward compatibility, however changing it’s setting has no effect.  In Windows Server 2008 R2 the property has been removed completely.

Some things change between versions, especially with Microsoft products the new management features often are accompanied by a new way of ‘getting things done’. One of these is the disappearance of the cluster.log logfile in Windows 2008 Server. When troubleshooting a Failover Cluster this file used to be the best source of information on the cluster and what would be ailing the configuration. In Windows 2008 Server Failover Cluster you are however unlikely to find this useful file on the system.

The cluster.log has been replaced by a more sophisticated event based tracing system, based of the event model used in Windows 2008 Server and Vista. This system has also replaced the old Eventlog system used in previous versions of the OS.

So how does one get to the cluster.log file in Windows 2008 Server? .

Open a Command Prompt (cmd.exe) and type the following command:

Cluster /Cluster:yourclustername log /gen /copy “C:tmp”

The files containing the log information will be stored in c:tmp or any other directory you choose to create these in, they are in clear text and can be opened via notepad for easy access. To me this beats the GUI by a long shot as personally i dislike the slow to respond MMC 3.0 Event Tracing GUI, and prefer to get my information quick and dirty from the Command Line.

I hope this information will help some of you out tehre who might have wondered what happened to the cluster.log logfile.

The other day the normal routine was disturbed by several users reporting that they were unable to log on to their Terminal Server Environment.  The following event was recorded on the Windows 2003 Terminal Server.

Event Type: Error
Event Source: Winlogon
Event ID: 1219

Description:
Logon rejected for DOMAINUsername. Unable to obtain Terminal Server User Configuration. Error: Access is denied.

There were thirty five users who were generating this event when log-in into the server, on the user end the error was displayed as : Access is Denied. and they were unable to log-in to the Terminal Server via RDP.

Usualy this event is logged when DNS or Domain connection problems prevent communication between the Server and a Domain controller, however in this case that is not the problem. Furthermore a small group of users was able to log in and run applications on the server, this seemed to be random.

To investigate the problem further and see what would happen if a user would be able to log in despite the existing problem we changed the IgnoreRegUserConfigErrors setting in the Registry to 1.

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlTerminal Server]
“IgnoreRegUserConfigErrors”=dword:1

After a reboot we had a test user log in and checked the status of the account, what happened was that the log-in was permitted but the properties of the object in Active Directory were being ignored. The home drive was not mapped, and the application that was supposed to start after the user had logged in as specified in the Environment tab on the Active Directory Object never ran.

The test user was also unable to log on to any other Terminal Server in the domain, and the same event was logged on those servers as well. It was not long after this that we got word that a project group had been migrating Exchange Mailboxes and we soon identified that all accounts that were having problems were the accounts that were migrated. Lucky for us this was limited to just thirty five accounts. The group of people that was able to work on this server had not been migrated due to their accounts residing in a different Organisational Unit in the Active Directory.

Investigation into the account Object in Active Directory revealed that the migration process had changed properties on the account Object, something that was not supposed to happen. At first these looked to be very innocent an additional x.500 address for example would not have caused the failure to log-on. But we also found that the security settings had been changed and ‘SELF’ was removed all permissions on the migrated account’s.

Changing these back resolved the issue of the users who were unable to log-in to the Terminal Server. This problem and the resulting event description are generated because the Winlogon process runs under the user’s credentials when logging on and passing the account properties from the Active Directory Object to the Terminal Server Service. The absence of  the ‘SELF’ permissions generated an Access Denied on the query for these properties resulting in the event and error shown to the user.

The second lesson that one can learn from stories like these is; Test before even thinking of running your script’s and or assumed safe to install and use applications on any production environment.  Make it part of your procedures and best practises to always test every line of code that runs against your production environment.  Communicate what and when it will be implemented and preferably test it on a copy of the production environment.

Using DTAP (OTAP for us Dutchies) and enforcing the testing of  code before releasing it ohe next level goes a long way, and in these times where the buzz is all about vitalization there really no longer is an excuse not to test your code/applications before they hit production and perhaps wreak havoc on your existing infrastructure. Costing you time and money in the process.

As the saying goes:  “A failure to plan, usually leads to a plan to failure”.
A little bit of information about Windows Terminal Server 2003 and Winlogon in this document: http://technet.microsoft.com/en-us/library/cc755399.aspx

Logon and logoff information is found in the Event IDs 528 and 540 for a successful logon and  event ID 538 for a logoff, there are more event’s where this information can be seen but these will give you a general idea where the accounts log on and off. If you are troubleshooting why an account is failing or for example who or what is locking out that user that call’s in every day to have his account unlocked you may want to look at the following Events.
Logon Types in Event ID’s 528, 540 and 538:

Logon Type 2 – Interactive
This type indicates that the user was either logged on either physical or via some sort of KVM (iLo for HP Blades for example) solution.

Logon Type 3 – Network
Typically indicated that the user was logged in via a SMB connection (File Share, Printers), IIS Windows Integrated or Kerberos authentication also logged the logontype as type 3.

Logon Type 4 – Batch
This indicates that the account was logged in from a scheduled job. Also happens when using at.exe to run a job immediately.

Logon Type 5 – Service
This indicates the account that logged in is configured in a Service.

Logon Type 7 – Unlock
When a user unlocks the system after the desktop was locked type 7 is logged in the eventlog.

Logon Type 8 – NetworkCleartext
Typicaly seen when using basic authentication in IIS, unless encryption is being ofered at a lower level (SSL for example) this would be very bad.

Logon Type 9 – NewCredentials
This indicates that a user used the runas /netonly to authenticate to a remote source on the network.

Logon Type 10 – RemoteInteractive
This indicates that the user logged in interactively via RDP (Terminal Server, Citrix Desktop). Users and Administrators logging in via the RDP protocol will

show up as type 10 in the event viewer.

Logon Type 11 – CachedInteractive
This is usualy seen on laptops where an user logges in on a notebook that is not connected to the network, Windows caches the cridentials if the user was previously logged on successfully on the network and this enabeles people to still be able to use their notebooks when not directly connected to the network. This may also indicate a problem in the LAN/WAN connections if this is logged frequently within desktops in the domain.

The indicator that a logon attempt was made when the account was logged off is the registration of Event ID 539. This event will not give a reason why the account was locked, it serves solely as an indicator to the fact that it is locked. Event ID’s 675 and 681 give a lot of information on what went wrong with the logon process. The following list shows the failiure codes in binary and hex and their descriptions for Event ID 675:

1 0x1 Client’s entry in database has expired  
2 0x2 Server’s entry in database has expired  
3 0x3 Requested protocol version # not supported  
4 0x4 Client’s key encrypted in old master key  
5 0x5 Server’s key encrypted in old master key  
6 0x6 Client not found in Kerberos database Bad user name, or new account has not replicated to DC yet
7 0x7 Server not found in Kerberos database  New computer account has not replicated yet or computer is pre-w2k
8 0x8 Multiple principal entries in database  
9 0x9 The client or server has a null key  administrator should reset the password on the account
10 0xA Ticket not eligible for postdating  
11 0xB Requested start time is later than end time  
12 0xC KDC policy rejects request Workstation/logon time restriction
13 0xD KDC cannot accommodate requested option  
14 0xE KDC has no support for encryption type  
15 0xF KDC has no support for checksum type  
16 0x10 KDC has no support for padata type  
17 0x11 KDC has no support for transited type  
18 0x12 Clients credentials have been revoked Account disabled, expired, or locked out.
19 0x13 Credentials for server have been revoked  
20 0x14 TGT has been revoked  
21 0x15 Client not yet valid – try again later  
22 0x16 Server not yet valid – try again later  
23 0x17 Password has expired The user’s password has expired.
24 0x18 Pre-authentication information was invalid Usually means bad password
25 0x19 Additional pre-authentication required*  
31 0x1F Integrity check on decrypted field failed  
32 0x20 Ticket expired Frequently logged by computer accounts
33 0x21 Ticket not yet valid  
33 0x21 Ticket not yet valid  
34 0x22 Request is a replay  
35 0x23 The ticket isn’t for us  
36 0x24 Ticket and authenticator don’t match  
37 0x25 Clock skew too great Workstation’s clock too far out of sync with the DC’s
38 0x26 Incorrect net address  IP address change?
39 0x27 Protocol version mismatch  
40 0x28 Invalid msg type  
41 0x29 Message stream modified  
42 0x2A Message out of order  
44 0x2C Specified version of key is not available  
45 0x2D Service key not available  
46 0x2E Mutual authentication failed  may be a memory allocation failure
47 0x2F Incorrect message direction  
48 0x30 Alternative authentication method required*  
49 0x31 Incorrect sequence number in message  
50 0x32 Inappropriate type of checksum in message  
60 0x3C Generic error (description in e-text)  
61 0x3D Field is too long for this implementation  

The list of Event ID 681 error codes:

3221225572 The user name doesn’t exist.
3221225578 The user name is correct, but the password is wrong.
3221226036 The user is currently locked out.
3221225586 The account is currently disabled.
3221225583 The user tried to log on outside the user’s time-of-day restrictions.
3221225584 The user tried to log on outside the user’s workstation restrictions.
3221225875 The user account has expired.
3221225585 The user tried to log on with an expired password.
3221226020 The user tried to log on with an account on which the administrator has selected the User must change password at next logon option.

The only bad thing with this type of investigation is that you definitely need to collect the events from all Domain Controllers, and in some cases even from the Workstations and servers where the account is logging in.

Last Friday i had the pleasure to attend the “Meet the Experts” seminar sponsored by Quest Software.  The planned speakers were Joel Oleson, Robin Meure, Mike Watson and Daniel McPherson. 

It was a great day and the sessions were educational and fun to attend and there were a lot of great SharePoint people who took the opportunity to show up and “Meet the Experts”.

The First session was presented By Joel Oleson, who gave his presentation of 10 steps to SharePoint Deployment Success. The slides from this presentation are available on his website at http://www.sharepointjoel.com/Presentations/. A great session and presented with great enthusiasm by Joel.  One which i might add may not be solely for just SharePoint but also other deployments might benefit from his tips.

The seccond Session was presented by Daniel McPherson, he presented the social media aspect of SharePoint through customised applications and features he developed. Think facebook and twitter for the business, one of his points was not to name it “social” but “business” to emphasis the added  business value in these applications.

Joel co-presented another session together with Robin Meure a well known SharePoint expert from The Netherlands. The presentation: SharePoint Logical and Physical Infrastructure Fundamentals witch was very educational even if some of the information was a repeat on what we already (should) know.

The last session before the Experts Q&A was Backup Demystified by Mike Watson. He has some surprising things to say about backing up SharePoint and I am still trying to wrap my head around some of the issues he raised to come up with solutions both working and supported that is 😉

The Q&A was awesome, the audience had some good deep technical questions and even though not all of them could be answered immediately the Experts panel was able to refer them to other experts in the field or give some basic direction to the troubleshooting process.

I also attended the SharePint in the evening, and had a blast there a twitterfeed was put up and Joel stole Robin’s phone to take some pictures of the event. You can find the tweets via by searching for #sp020  on twitter.

All in all i had a great day, returned home way too late and dead tired, but had learned a great deal and had even more to ponder and investigate further.

On another note, Friday i cleared the 070-648 exam. Granting me two cleared exams towards my MCTIP:Enterprise Administrator cridentials. (640 and 642)

I allready had the MCTS cridential from when i took the SharePoint Exam in December 2008, so this did not give me any new abreviations to put on my businesscards or anything. But its always a nice step to check yet another box on the endless list of exams us IT people have to get through.

Personaly i am usualy a little behind on the actual certs and try to be ahead in experience, but i am going to mark some more exams towards the MCTIP certification this year (in the summer). After that I will probably update some of the non-Microsoft checkmarks like ITIL3 and other technology certifications.

For MCTIP: Enterprise Administrator the count is now 2 down, 3 to go.

When you need information on installed or running drivers on a Windows 2008 Server you can build a WMI query to get the information you want or you can use the build in tool driverquery.exe. This utility provides information about drivers and their status.

Just typing driverquery.exe will give you a list of all drivers the OS has installed. A more useful view will be provided if you add the argument /v (verbose). This will among other things provide you with the Driver Type, State, Status and  also with the path and file name of the driver.

This information could be usefull in various ways, providing you with forensic information on malicious installed drivers (that would be bad) or just making sure that the right drivers are installed and started.

And best of all, the utility can be used remotely and even has an argument for providing a domainuser name and password. The report may be formatted by comma or tab separated columns or just as a list.

The difference of this list with the one from pnputil is that it also shows the non 3rd party drivers even more the verbose list will provide you with detailed info on the validity and state of the driver and to me that was very useful.

 The command line options of this utility are :

DRIVERQUERY [/S system [/U username [/P [password]]]]
              [/FO format] [/NH] [/SI] [/V]
Description:
    Enables an administrator to display a list of
    installed device drivers.

Parameter List:
      /S     system           Specifies the remote system to connect to.

      /U     [domain]user    Specifies the user context
                              under which the command should execute.

      /P     [password]       Specify the password for the given
                              user context.

      /FO    format           Specifies the type of output to display.
                              Valid values to be passed with the
                              switch are “TABLE”, “LIST”, “CSV”.

      /NH                     Specifies that the “Column Header”
                              should not be displayed. Valid for
                              “TABLE” and “CSV” format only.

      /SI                     Provides information about signed drivers.

      /V                      Displays verbose output. Not valid
                              for signed drivers.

      /?                      Displays this help message.

Examples:
    DRIVERQUERY
    DRIVERQUERY /FO CSV /SI
    DRIVERQUERY /NH
    DRIVERQUERY /S ipaddress /U user /V
    DRIVERQUERY /S system /U domainuser /P password /FO LIST