Running Virtual Machines? Keep Your Clocks Synchronized!
Using VMs is IT reality nowadays. Hyper-V, VMware, XEN or whatever virtualization platform you're using - keeping clocks in sync is underestimated.
Unless you've been hiding under a rock for the past decade, you probably have already migrated, or switched entirely, to virtual servers and perhaps even desktops. A Virtual machine infrastructure has become somewhat of a standard these days. Whether you're using Hyper-V, VMware, Xen or any other hypervisor platform, there's some misconception and lack of attention around a specific issue: Time synchronization.
Quick background: Computers use an internal clock hardware chip (RTC) to supply the system with an accurate clock. Many encryption and authentication methods also rely on the computer's clock to make sure issued tokens, certificates and passwords are indeed valid and not expired and that's just one example on how important that little clock at the bottom of your screen is. You can read more about that on Wikipedia - Real-Time Clock and also System Time.
Now back to our main topic: When virtual machines become a part of your IT environment they must be synced with one another's clock and any other participating servers or desktops. To make it easy, each hypervisor vendor has provided a bridging technology to pass the clock cycle from the physical server to the virtual machines it runs. That way it will be easy to sync the host once, and then all VM machines will follow and get synced together. A great method indeed, but could potentially cause havoc in your environment, just like any clock that does not get tuned or adjusted when needed. Yes, clocks can go bad in computers as well. That's why there is an official protocol and solution to sync time all around the world - the atomic clocks and NTP servers.
In a typical IT environment, it's crucial that any virtual machine's physical host server will be synced to an NTP server, and any other physical server or desktop should sync to that same NTP server as well. As with any Microsoft based environment utilizing Active Directory, the default configuration of any domain joined server and computer is to use NTP data from the Active Directory PDC domain controller. Anyone using Microsoft's Active Directory should already know the above statement, but still some fail to notice out a very important detail - what if that Domain Controller is virtualized? There could be a potential situation where the physical host NTP server goes "bad" or worse, the host was never configured to use an NTP server in the first place, and that physical host clock will get shifted.
The solution to prevent that potential nightmare is easy and can be done in a few simple steps:
- Designate an authoritative NTP time source for your environment - http://www.pool.ntp.org/en/.
- Make sure that your PDC Domain Controller is synced with that same time source - http://support.microsoft.com/kb/816042.
- Make sure any physical host of virtual machines IS configured to use an NTP server - and point it to the PDC Domain Controller - http://support.microsoft.com/kb/816042 or http://kb.vmware.com/kb/2012069.
- Make sure ANY virtual domain controller is configured to IGNORE the virtual machine clock and always get synced with the PDC - http://blogs.msdn.com/b/virtual_pc_guy/archive/2010/11/19/time-synchronization-in-hyper-v.aspx.
- Keep time and clock information as one of your top priorities and procedures. When performing disaster recovery or maintenance work on your environments, start servers in an orderly fashion and pay attention to the time information. This way will reduce your risk to failures.
Stay synced!comments powered by Disqus