smashedbug.com, tech music and geek news

Ok.. I’m fed up. I want to notate some music. I need to wear my headphones. My headphone jack does not work. Why does it not work? I don’t know.

A while back out of the blue my audio driver starting having an “EAccessViolation” error. I have no idea what this is. In anycase, since starting to get this error, my headphone jack does not work.

I googled this error and came up with various forms of black magic to get around the error (most people reinstalling Windows).

Why is it that something as simple as an audio driver or audio control panel cannot be resolved without completely destroying the entire working computer with a full reinstall?

Why can’t Windows developers provide better error reporting than “EAccessViolation”? What DOES THAT MEAN??!! So frustrating.

I seriously yearn for actual error reporting. I think any software that returned something as cryptic as “EAccessViolation” would be shunned and run out of town on any of my Unix systems. Why is it so hard for a Windows application to say “hey you know what? I was trying to access XYZ file and it was either not there, wrong version or something and as a result, I cannot run”?  What is so friggin hard about that? On a Unix system, I’ll get messages like that and if I don’t know what the file is, I can query what package installed that file on my system so I know EXACTLY what application would be causing the issue (but as an aside, I rarely have these issues so its a moot point).

Yes this was a rant, but I just want to use my computer and not troubleshoot some stupid “EAccessViolation” error. I’m about 3 minutes away from buying a new speaker setup that has a headphone jack so I can just bypass this annoyance. Thats seriously messed up.

Note: If you try Backblaze, please use one of the links in this post to access and download the software. This will help fund my site. Thanks!
How safe is your data? Do you actively backup? How do you backup? How often do you check your backup to make sure it is working? I found a cool service called Backblaze that should be considered as part of your backup strategy.

Backblaze is an online backup service. From my experience, it provides the following features:

• Backblaze is easy to use. Download and install.. and thats it. No massive configuration, no scheduling backup times, no full/incremental backups, no rotating of tapes/CDs/drives or other media, etc. Up and running in minutes.
• Backblaze is always backing up. I put some test files on my computer and within an hour, it was safely backed up to the Backblaze servers. I didn’t have to think about the backup process.
• Backblaze backs up everything. By default it backs up all my documents and settings. It (by default) excludes certain system files, temporary files, cache files, program fiels, etc which should never need to be restored. You can exclude other files if desired (ie if there is data you don’t feel comfortable uploading to a remote service or perhaps data that doesn’t matter if it is destroyed).
• Backblaze is affordable. $50 per year to backup an unlimited amount of data on your computer. This is less than $1 per week (15 cents per day) to have the peace of mind that your data is being backed up.

So why am I so excited about this? Well lets look at the alternatives:

• Tape backup. Requires special software, scheduling, rotating of tapes and manual intervention. Typically only has a backup once per day and “done right” requires some type of off-site location to store the tapes. On top of all of this, its expensive. A 160GB drive is $680 and the tapes to rotate adds $279 for a total of $959 (most likely another $50 for software). $1000 .. that equates to 20 years of Backblaze service. Of course, tapes don’t last 20 years, so there will be additional tape costs and many sites use a safety deposit box as their off site location so there is the related monthly fees for that as well.
• Secondary hard drive. A popular option for many people. The cost of the hard drive ends up being around $100 (about the cost of two years of Backblaze service) and most drives come with some type of backup software (or could be utilized with software such as Syncback). There are several drawbacks to this backup method

• prone to physical compromise (usually the external drive stays connected to the computer so if the computer is stolen, a fire or flood occurs or possibly a power spike, it could take out both the computer and backup)
• generally this backup is just a sync or a copy so it doesn’t provide history (Syncback provides up to a month of file changes)
• it stays connected to the computer, it could be suspectable to a malware or virus infection
• If the drives are rotated (similar to tapes), because of the mechanical nature of the drive, it could reduce the longevity of the drive
• Requires manual intervention

Of course, there are other backup methods, but most require significantly more intervention, checking and maintenance to keep up-to-date. Unfortunately, I find so many of these methods simply fall to the wayside and backups do not happen on a regular basis.

So whats the downside to Backblaze?

• Distance of the data. If you have a significant amount of data backed up to the Backblaze servers and need to restore all of it (ie computer died or was stolen, etc) it is not instantly available. You can download it (which depending on your internet connection could take several days or more) or have the Backblaze people ship you an external drive with your data (few days to receive at a cost of $189 — but that does include the drive)
• Relying on a third party to store your data. Depending on your data, this may or may not be an issue.
• If your data changes more than you can transfer to the Backblaze servers in a given day, you may not be able to use the service (ie if you process photos, video or other large files on a daily basis). For most office or home uses, this should not be an issue.
• This is not a full system backup (ie similar to Norton Ghost or other disk mirroring software). If you need to restore a full system, you will need to manually reinstall your applications .. this backup is only your data.

So far, I find the service to be fantastic. Its easy to install, its low cost and the “just work” nature of the service is exceptional. I am quickly finding the service to be my #1 pick for home and personal computer users needing a reliable backup solution.

Website: www.backblaze.com

I’ve ran into a situation over the past few months that servers I maintain started having their hard drives fail. While I maintain backups of the data, I found that instead of being a “leisurely” replacement, I found myself in a stressful situation having to act quickly to acquire new hardware, load the system, load the software & restore the backup. While the use of rsnapshot has been a great tool for these situations, the need for unscheduled downtime (usually two to three hours) and unfortunately usually during business hours was not ideal.

As a result, I started looking into utilizing RAID to address the situation. I found most of my servers were in the 320GB to 500GB disk range (usually storing much less data than the disk provided) and as a result, a new drive today only cost around $50-$55. Definitely worth it for the piece of mind.

In addition to the physical hard drive, I looked at using the FreeBSD gmirror software to build and maintain the RAID. For testing purposes, I setup a FreeBSD box in VirtualBox and worked with the guide on OnLamp to get my initial RAID up and going. Fortunately the system allows for preexisting data to be on the drive (no need to setup the RAID before the OS installation). All in all, the RAID setup took only a few minutes to configure (basic steps is to tell gmirror to make a RAID, adjust the fstab to point to the new “mirror” file system points and load the gmirror/geom system on boot).

While the initial setup was straight forward, I was curious on the performance of the RAID in some “real world” scenarios.

Scenario 1: Failed Drive: atacontrol detach

To emulate this situation, I pulled a drive and booted the system. Pulling either the primary drive or the secondary drive did not matter. The system still booted and showed the RAID in “degraded” mode.

I also used “atacontrol detach” command to stop a controller (and hense a drive) during operations. Once again, the system continued functioning without any indication the drive failed (except for a console warning & degraded status). I had various disk operations running at the time and everything continued as expected. Attaching the drive using “atacontrol attach” had the drive rebuilding in the background while existing server operations continued, unaffected (ie I’d imagine if my servers had hot swap, no need for reboot or other interruptions to operation).

Scenario 2: Failed Drive: Replaced with bigger drive

My next test was to pretend a drive was failed and replace it with a different, larger drive. Instead of an auto-rebuild, the system required two commands to remove the failed drive from the mirror and add the new drive to the mirror:

• gmirror forget gm0 (gm0 is the RAID device)
• gmirror insert gm0 ad4 (ad4 is the new disk device)

After this, the RAID started to rebuild itself in the background and I was back up and running. Using the instructions per the OnLamp article, any additional space on the larger drive was not usable (there are other ways to configure gmirror to allow slices or partitions to be mirrored instead of the entire drive.. but I did not test this functionality).

Scenario 3: Failed Drive: Smaller drive

As I am mirroring the entire drive, I was curious what would happen if I only had a slightly smaller drive as the replacement. Of course, as expected, gmirror did not like the smaller drive. However, a workaround I was able to to was slice & partition the new drive, add a boot manager and then use the dump/restore commands to copy the data from the larger drive to the smaller drive. I modified the fstab back to a standard single drive configuration and then erased the existing gmirror RAID information (gmirror stop gm0) and then simply created a new mirror, using the smaller disk as my initial disk for the mirror.

This was a bit more time intensive but in theory, it is possible to dump a live system and then use rsync to sync any changes in data that changed after the dump occurred (I’d recommend doing an rsync in single user mode so there is more control over data changes).

Once this was done, the system and gmirror was back up and running without issue. This method does require two additional reboots (one to switch the system over to the new drive & one during the initial gmirror setup on the new drive (which *might* not be necessary .. more testing required)).

Scenario 4: Power Failure

Simply put .. I had various disk activities going on and then killed the power to the box. On bootup, fsck ran to check the disk and afterwards the gmirror sync’d the disks back to a working state.

Monitoring & Conclusion

Overall my initial tests have been very positive with gmirror. It is very straight forward and easy to use. I didn’t do any benchmarking given the virtual environment of my tests, but from my understanding, disk reads can be up to twice as fast, so adding that second hard drive does provide additional performance benefits as well.

Use of some simple scripting and the “gmirror status” command will allow me to monitor the status of my RAIDs and hopefully address any issues with the systems before users are aware of a problem. The “gmirror status” returns either a “DEGRADED” or “COMPLETE” status. “DEGRADED” will either indicate a drive failed or the system is rebuilding. “COMPLETE” indicates a functional system. Use of some scripting & cron can email change status to you or this could be tied into a monitoring system such as Nagios (probably more ideal particularly if Nagios is already inplace for other monitoring tasks).

While gmirror is not a particularly new technology, based on my testing and reports from production use of this tool, it appears to be a solid and very worthwhile addition to any server configuration (and quite possibly most desktop configurations).

It has been a few years since I touched on software management and a recent lively dialog prompted me to write this article to explain some of the benefits of software management (usually called package management). The discussion was a belief that Microsoft Windows needed software management/package management like Linux. I was quite surprised at the number of people that disagreed with this and it seemed that many didn’t know a lot of hidden advantages of package management. This article is designed to highlight some of the features.

What is software management? In the world of Linux and BSD, there is the concept of software repositories and package managers. Essentially a wide selection of software is centralized and standardized in the method it is installed. For example, FreeBSD has the ports system where with a simple “make” command, I can install one of thousands of applications for the system (At the time of this writing, over 19500 applications).In Windows, this is  similar to going online, downloading a file and running the setup application.

However, unlike Windows, by default the ports system utilizes hashs (MD5 and  SHA256) to verify the validity of the application (for example, if a web server gets hacked and a virus laden version of the software is loaded, when FreeBSD downloads the software to install, it will notice it failed the checksum and not install the potentially vulnerable software). This does assume that malicious code was not part of the original software and that at the time the port was created/updated, the software was clean (ie not infected as to have an invalid checksum). I am unaware of a situation where this has been a problem. As far as I know, there is no equivalent system for Windows software installation.

In addition, the ports system ties into a security audit system. As a result, when the port is installed, it is checked against a security database to make sure the version of the software being installed does not have any outstanding, known security issues. If it does, this requires a manual override to install the software. Most of the time, the solution is to update the ports tree to get the latest version of the software (generally software is patched within days of the vulnerability being found). As far as I know, there is no equivalent system for Windows software installation.

At this point, your able to easily install software with a single command and have it automatically checked to make sure the file is valid and has no known reported security issues. Perfect. However, one other HUGE advantage is available — recursive installation. Certain software requires other software to be installed inorder to operate correctly. For example, on Windows, an application written in Visual Basic may require the Visual Basic runtimes. Same for applications written in ASP or the .NET framework. When software is installed, the user is usually prompted that this software is required and it is up to the user to navigate the web to find the software, download and install and hope that it is the correct version. While this is not too difficult for perhaps one dependency, if the software has many dependencies (for example, lets say your wanting to install a web application on a fresh Windows installation — this may require manually installing a web server, database server, latest version of the ASP.NET framework, potentially other libraries that the web application uses, etc..). Needless to say, this is a lot of additional locating of software, downloading and installing.

In the FreeBSD ports world, the original “make” command will notice certain software is required and automatically install it. Infact, if it starts installing software and notices THAT software dependecy requries other software, it can install those applications as well. To top it off, its possible to tell the “make” command to configure all the software up-front so the system can install all of the software in the background while your able to attend to more important tasks instead of babysitting the installation. As far as I know, there is no equivalent system for Windows software installation.

So at this point, with one command, I could theoretically install a wide range of software. Here is where the fun really begins with software management.

On FreeBSD, with the “pkg_info” command, I get a list of all the software installed on the system. That is quite nice to know exactly what software is installed (I can backup this list along with my configuration files & user data and recreate the installation without too much issue).

If I am not sure why a certain piece of software is installed, I can query to determine what software (if any) depends on that particular piece of software. On Windows, have you ever uninstalled an application and it had the prompt “This file may be used by another application, do you want to remove?” and was unsure what to do? With software management, it will let you know exactly what other software depends on that particular file and allow you to make an informed decision (not a “well lets try it and hope nothing breaks!”).

Perhaps you need to know the opposite — you see a file on the computer and are unsure what it is for .. once again, you can query the software management system and it can let you know what package that particular file was installed by.If all of the applications you use are managed by the software management system, you can easily weed out potentially harmful files from the system (or simply files that are taking up resources but not being used).

The ongoing maintenance of your software is equally as easy. FreeBSD has a nightly security audit process that checks all installed software against a centralized security database to notify of any security threats for your software. One simple email report that can be reviewed quickly versus reviewing these issues from multiple sources as with Windows. In addition, when the port tree is updated (again, a process that could be setup on a nightly basis) you can easily check to see which software has updates, centrally and scriptable. FreeBSD offers up an “UPDATING” document (which is sync’d when the port tree is updated) that offers up step-by-step instructions for major software updates.

All of the above is fine for a single server, laptop or desktop computer. What can a software management system do for managing clusters, large networks or server farms? Once an application is configured and installed, it is possible to tell FreeBSD to create your own custom packages of this software (pkg_create). From here, you can point other computers to your own central software repository and install the same exact build of the software across as many computers as you manage. Given the flexibility of well defined, simple commands (make, pkg_info, pkg_create, pkg_delete), it is possible to script these in a variety of ways to have software management largely automated.

Perhaps your managing a server farm with mail servers, web servers and file servers. In addition, you might have various desktops (administrative, design, development). It would be possible to create system profiles so a computer could (via a nightly cron process or perhaps through a push out via SSH) be assigned classifications — ie, “Server 01 is a web server and file server” and then assign a software stack that should be loaded on that computer. The system could pull down the custom packages and install them and run any necessary cleanup/configuration scripts to maintain a centralized method of software installation and updates.

Utilizing the security functionality built into the software management tools, it could be possible to have individual systems self-audit their software stacks against the central repository and report any deviation (or simply eliminate inconsistencies).

As a result of these tools, your able to maintain up-to-date software on the computers and have a high degree of confidence that the software on the systems is valid and un-compromised. In the event a system fails, use of packages, text output of the ports installed or a central repository would allow for a new system to be installed and online within a very minimal amount of time (infact, in larger networks, it could be possible to have hot-standby systems so if a system does become unavailable, the stand-by is instantly loaded with the software and during the software install, data from the backup is restored to the stand-by computer to minimize the outage — automatically.).

Yankee Group released a summary of their 2007 server reliability report. Among the findings:

• UNIX systems continue to be near bullet-proof with over 99.99% uptime (IBM AIX lead the survey with 99.994% reliability .. about 30 minutes downtime per year).
• Windows 2003 Server was less reliable, about 25% less reliable .. average downtime over 9 hours per year (99.9% uptime)
• Linux on average was more reliable .. about 75% more reliable than last year averaging about 1 hour per year of downtime (99.99% uptime). This included both enterprise (paid) and community (free) variants of Linux. This was on-par with Solaris and HP UX.

Except for Windows 2003, the summary did not specify exactly what caused these outages. The summary specified many security updates requiring administrators to take down their Windows 2003 servers to apply the patches (its a shame you still have to take an entire server offline and reboot to apply a simple security patch!). It would be interesting to see if the outages for Linux were hardware related or software related. Most Linux installations that I am aware of are on consumer grade equipment which generally doesn’t have nice features to increase uptime numbers (ie hot-swap drives, RAID, redundant power supplies, etc) versus most mid-range and high-end Unix systems.