Important IT Information
An email to all occupants of MRI Buildings (MRI, MRL, RUA) from Bob Cornwall and the IT Team.
Compromised/Infected Computers
Over the past year, the number of computers connected to MRI’s network becoming compromised and/or infected with malware has dramatically increased. University regulations require that we take possession of the compromised machine and scan it for personally identifiable information in the form of Social Security and credit card numbers. After scanning, and hopefully confirming that no sensitive information was “leaked,” we are required to reformat the hard drive and reinstall the operating system. For personally owned machines, we are required to scan and return the computer to the owner who then must reformat and reinstall the operating system before the computer is allowed network access through Penn State.
While dealing with all of these issues we are not able to move forward with ongoing strategic and tactical plans; for example installing new conference room computers which we have had since the fall and server upgrades which have been pending since last October. Other than the obvious inconvenience for the user and their data, there are several other issues we are trying to deal with during this process that are detailed below and relate to you.
University Required Scanning of Machines on the PS Network
There are several tools that PSU uses to monitor network activity. Compromised and infected machines are identified by hardware that monitors outgoing network activity and when a spike is observed, particularly one to known malicious IPs, our IT staff is notified to take the compromised computer offline and scan it. Additionally, we have been required by the provost to begin actively scanning all machines on our network for personally identifiable information which will start this week. This process runs on computers and looks for patterns that could be social security numbers or credit card numbers. A report is sent to our network administrators and they are responsible for reviewing these reports and tracking down any suspicious information on machines connected to our network and certifying its removal.
MRI-Managed Computers
We are in the process of joining all university-owned computers based in our buildings to a common Domain. What this means is central management of every machine connected to the wired network. Central management does not mean there is any interest in monitoring or tracking what people do – we don’t have the time, the inclination, nor are we allowed to by university policy. It simply provides a mechanism for IT staff to ensure that all computers are configured to receive regular system software updates and that they have virus and malware-blocking software installed with up-to-date, active definitions. In addition, as part of domain participation your documents can be securely stored and are backed up on a central server. If your computer ever crashes or needs to be rebuilt for security reasons, all of your documents are readily available. We utilize Windows XP as our standard operating system and anticipate migrating to Windows 7 after it becomes available and is proven to be stable. Macs can also be added to the domain if they are capable of running OS 10.5 (Leopard).
Operating Computers as Administrator
In the past, it has been customary to operate the computers in our offices and labs with an “administrator” account. If we had up-to-date anti-virus software and avoided suspicious e-mail attachments and Web links that was sufficient for preventing infections. Unfortunately, today’s malware exploits administrator accounts by using their elevated status to invisibly install software that compromises the computer, and this happens without the user ever doing anything “wrong” or even knowing that something has been installed. We have worked hard to find a solution to this problem that allows the flexibility needed to enable researchers to do their work while avoiding malware attacks. On the Windows side, what we found is an open source program called Sudo for Windows. It allows one to work on a computer with a “user” level account, but when needed, elevate that status to administrator, for tasks such as installing software. As computers are joined to the MRI Domain, we will verify the need for administrator accounts and if so, install Sudo for Windows. Aside from those cases where administrator access is required and Sudo for Windows is installed, all of us will regularly operate our PCs with “user” accounts. This is the case even for the IT staff. For more information on this university initiative, please visit http://www.ipas.psu.edu/phase2/supportingdocs2.html
Along with Penn State’s Security and Operations Services (SOS), we are looking at security risks on the Macintosh platform and searching for appropriate yet workable solutions for combating malware. To date, we have not had a compromised Mac on our network and we hope to keep it that way.
Older Machines and Operating Systems
For those running operating systems prior to Windows XP, it is often a laborious and time consuming process, sometimes taking 8 hours or more to re-install the operating system software, security updates, and all of the associated hardware drivers for that machine in the event of a hardware failure or compromise. These machines are typically past their expected lifespan and out-of-warranty. For these machines we are recommending where possible to update to a newer computer running at a minimum Windows 2000, but even better Windows XP. Microsoft no longer supports or provides security patches for operating systems prior to Windows 2000, thus we cannot allow network connectivity for computers running those operating systems. It is not enough to disconnect the older machines from the network and use removable media such as a USB stick to retrieve files. We have had multiple compromises result from infected USB drives used between machines. Older/unsecured machines need to be replaced as soon as possible. For XP Service Pack 3 and later machines, that the IT group has configured, we maintain a base build that can be re-installed on that computer in a relatively short amount of time.
Because of the increased amount of time required to repair or rebuild older computers, we must now charge a standard fee of $35/hour for troubleshooting and repair work on computers that are out-of-warranty. Fees cannot be charged to federal budgets.
Buying New Computers
Purchasing new computers is often an individual decision. However, this flexibility has created a very difficult network to support and manage with each machine having a different configuration, hardware drivers, warranties and inherent software problems. For instance, several years ago, we were recommending IBM/Lenovo computers. Over the course of the last year we have run into issues with software updates to these computers and it seems that since IBM spun-off their PC Division to Lenovo, their support has diminished.
As a result we have switched to Dell business class computers for all desktop and laptop PC purchases. While we understand that there may be less expensive computers available, they often come with inferior components and do not offer the warranty that Dell does to Penn State. Also, our base builds are configured around the Dell Systems, so it takes the IT group significantly less time to troubleshoot and rebuild machines. Please, when purchasing computers, work with Joel and/or Paul. They have access to the best pricing/warranty/component combinations available to Penn State from Dell and Apple, and uniformity makes a tremendous difference in network operations. Dell is currently running its bulk-buy special pricing program between now and the end of July.
Non University-Owned Computers
Some graduate, undergraduates and visiting scientists use their non university-owned machines for research purposes in our buildings. One of the luxuries they have been afforded in the past was a wired connection. For security reasons and because of the increasing problems with malware we have experienced, we have been transitioning these machines to wireless access through the Penn State network. In this case, one’s Penn State user ID is required to gain network access, and that access is granted outside of MRI’s network systems. Needless to say, wireless in our buildings is not as robust as wired access and also comes with limitations associated with printing and speed. We are searching for a better solution, but because we cannot manage these machines, they are often running file sharing software of questionable nature, and are not properly maintained with adequate malware protection and current operating system updates. We cannot allow them on our wired network any longer. We are offering a printing solution in that will be set up in our copy rooms with the networked printers. Anyone with an MRI Domain account will be able to take a USB stick to these machines and print.
In addition, the MRI IT staff can no longer support user issues on these machines. We currently have more than 600 machines connected to our network in MRI buildings. Securing and maintaining university owned machines within the MRI domain is our number one priority. We recommend that for support on personally owned machines or machines owned by other organizations that the owner go to the supplier, their organization, or to the university help web site http://css.its.psu.edu/ Any support MRI staff are asked to provide, will be billable to a university budget at $35/hr. Fees cannot be charged to federal budgets.
Rebuilding Machines and Software Licenses
This is not an experience anyone wants to go through with any frequency. If your files are not backed up regularly, you can very well lose valuable data in the event that your computer completely crashes. Also, all of your personal settings take time and effort to reset. Often times, the most reliable and expedient way to repair a crashed or crashing computer is to reformat the hard drive and re-install the operating system. One of the problems we experience with this is software for which the user no longer has the license key. Under Penn State policy, we cannot allow unlicensed copies of software to be installed on university machines. If the key is unavailable, a new license will need to be purchased. The frequency with which this occurs suggests that there is actually a good bit of pirated software, and it is the IT staff’s responsibility to be good stewards for the university in this regard. I encourage you all to maintain up to date software licenses within your research groups.
Closing Remarks
It was my intent that this message educates you on some of the concerns we have in network and PC security so you better understand the decisions that are being made. If there are any questions or issues that you would like to discuss, please let me or anyone in the IT team know.
Bob
Robert Cornwall
Managing Director
Material Research Institue
814-863-8735
