The Perils of Purging

Posted by Kevin Way Sat, 20 Jan 2007 00:46:00 GMT

If you’ve done a source upgrade on FreeBSD, you probably saw the following after you did your post-installworld “make delete-old”.

make delete-old
>>> Removing old files (only deletes safe to delete libs)
>>> Old files removed
>>> Removing old directories
>>> Old directories removed
To remove old libraries run 'make delete-old-libs'.

But why would you want to remove the old libraries? You know enough about Unix to know that doing so can break dependencies, and that it is often hard to figure out what all depended on a particular object.

Well, I wanted to because I don’t like having obviously outdated and unmaintained code sitting on my systems.

If you decide to run “make delete-old-libs”, you’ll receive this dire warning.

make delete-old-libs
>>> Removing old libraries
Please be sure no application still uses those libraries, else you
can not start such an application. Consult UPDATING for more
information regarding how to cope with the removal/revision bump
of a specific library.
remove /lib/libalias.so.4?

And now what? Do any of your apps require /lib/libalias.so.4? I sure don’t know, and I don’t want to deal with unexpected breakage, so I came up with the attached script.

It is written to function as a nagios plugin, but it is perfectly usable from the command line as well. The only thing to note is that you will want to add a “-v” or two, in order to get detail, if it does say that anything is wrong.

What it does is relatively simple:

  1. It compiles a list of all the spots where executables and libraries are likely to live. It does this by grabbing a list of all the directories in your PATH, and in your ldconfig’s hints file.
  2. It finds all of the dynamically linked ELF objects in those directories. (or to say that in English, it finds all your applications and libraries.)
  3. It checks each of these for two things:
    1. That the library it depends on exists
    2. That the library it depends on is not listed in /usr/src/ObsoleteFiles.inc
  4. Then it generates a report, that varies in length depending on whether you used the “-v” option 0, 1 or 2 times.

We then distributed this script across all of our servers, and setup Nagios to automatically alert us if we have any unexpected library dependency failures (or if any of the libraries we built our applications against have become obsolete.)

Hopefully this can save you some trouble, and allow you to run “make delete-old-libs” fearlessly, to keep your computing environment neat, tidy, safe and secure!


-----------------------------

#!/usr/bin/env ruby
#
# Checks a FreeBSD system for incorrectly linked libraries

require 'optparse'
require 'timeout'

Version='1.0.1'
OPTIONS = {
  :verbose => 0,
  :obsoletefiles => '/usr/src/ObsoleteFiles.inc',
  :warning => 0,
  :critical => 0,
  :timeout => 0,
  :obsolete => true
}

OptionParser.new do |opts|
  script_name = File.basename($0)
  opts.banner = "Usage: #{script_name} [options]" 

  opts.on("-o", "--[no-]obsolete", "Warn if system is using libs listed in /usr/src/ObsoleteFiles.inc.  Default: #{OPTIONS[:obsolete]}") do |o|
    OPTIONS[:obsolete] = o
  end

  opts.on("-w", "--warning N", Integer, "Warning threshold.  Default: #{OPTIONS[:warning]}") do |n|
    OPTIONS[:warning] = n
  end

  opts.on("-c", "--critical N", Integer, "Critical threshold.  Default: #{OPTIONS[:critical]}") do |n|
    OPTIONS[:critical] = n
  end

  opts.on("-t", "--timeout N", Integer, "Timeout (in seconds.  0 means infinite.).  Default: #{OPTIONS[:timeout]}") do |n|
    OPTIONS[:timeout] = n
  end

  opts.on("-v", "--[no-]verbose", "Run verbosely.  If specified more than once, get more verbose.") do |v|
    if v
      OPTIONS[:verbose] += 1
    else
      OPTIONS[:verbose] = 0
    end
  end

  opts.on_tail("-h", "--help", "Show this message") do
    puts opts
    exit
  end

  opts.on_tail("-V", "--version", "Show version") do
    puts Version
    exit
  end
end.parse!

#
# track status with these
sys_error=false
error_cnt=0
warn_cnt=0
error_files=''
error_long=''

# wrap this whole fucker in a timeout loop, since nagios plugins
# are supposed to have a timeout option, by spec
begin
  Timeout::timeout(OPTIONS[:timeout]) {

    #
    # Check the major executable paths...
    dirlist=`/usr/bin/printenv PATH`.split(':')

    # And now get all the major lib paths
    `/sbin/ldconfig -r`.each_line{ |ldconfout|
      if ldconfout =~ /search directories: (.*)/
    dirlist << $1.split(':')
    break
      end
    }

    # And now read in the ObsoleteFiles OLD_LIBS list, to
    # help find things that will go away after a make delete-old-libs

    obsolete = Array.new
    if OPTIONS[:obsolete] 
      begin
    File.open(OPTIONS[:obsoletefiles]).each_line { |line|
      next if line !~ /^OLD_LIBS\+=(.*)/
      obsolete << "/#{$1}" 
    }
      rescue
    sys_error=true
    error_files="#{OPTIONS[:obsoletefiles]} not found." 
    error_long="#{OPTIONS[:obsoletefiles]} not found. Unable to check for obsolete libs." 
      end
    end

    dirlist.each { |dir|
      # Make sure the directory exists
      next if ! File.exists?(dir.to_s)

      # Get every file type
      `/usr/bin/file -N #{dir}/*`.each_line { |fileout|
    error_flag=false  # flag var
    warn_flag=false
    error_libs=''
    warn_libs=''

    # If it isn't an ELF object, we aren't bothering to check it
    # same goes if it is statically linked
    name, type = fileout.split(': ')
    next if type !~ /^ELF/
    next if type =~ /statically linked/

    # check the ELFs
    `/usr/bin/ldd -a -f '%o:\t%p\n' #{name}`.each_line { |lddout|
      lib, loc = lddout.split(":\t")

      # If loc is nil, move on
      next if ! loc.kind_of?(String)

      # Strip the symbol addresses and trailing crap
      loc.strip!.gsub!(/ \(.*/,'')

      # spew an error if a lib is unfindable
      if loc =~ /not found/
        error_libs << "\tmissing object: #{lib}\n" 
        error_flag=true
      # check if it is obsolete, but present
      elsif obsolete.index(loc)
        error_libs << "\tobsolete object: #{lib}\n" 
        warn_flag=true
      end
    }

    # Something hit the fan.
    if error_flag
      error_cnt+=1
      error_files << "#{name} " 

      pinfo=`pkg_info -W #{name}`
      if pinfo =~ /was installed by package (.*)/
        error_long << "#{name} from package #{$1} depends on missing libs.\n" 
      else
        error_long << "#{name} from unknown package depends on missing libs.\n" 
      end
      error_long << error_libs
    end

    # Something will hit the fan if you run make delete-old-libs
    if warn_flag
      warn_cnt+=1
      error_files << "#{name} " 

      pinfo=`pkg_info -W #{name}`
      if pinfo =~ /was installed by package (.*)/
        error_long << "#{name} from package #{$1} depends on obsolete libs.\n" 
      else
        error_long << "#{name} from unknown package depends on obsolete libs.\n" 
      end
      error_long << error_libs
    end
      }
    }
  }
rescue Timeout::Error
  sys_error=true
  error_files="Search timed out after #{OPTIONS[:timeout]} seconds" 
  error_long="Search timed out after #{OPTIONS[:timeout]} seconds" 
end

# Tests are over, time to say what was wrong
# 
if sys_error
  error_status="Unknown: " 
  rc=3
else
  case error_cnt + warn_cnt
    when 0 
      error_status="OK: " 
      rc=0
    when 0...OPTIONS[:warning] 
      error_status="OK: " 
      rc=0
    when OPTIONS[:warning]..OPTIONS[:critical]  
      error_status="Warning: " 
      rc=1
    else
      error_status="Critical: " 
      rc=2
  end
end

# Nagios specifies the following verbosity levels
#   0 - Single line, minimal output
#   1 - single line, additional information
#   2 - multi-line, debug output
#   3 - lots of detail
# We will implement 0-2 for the moment.
case OPTIONS[:verbose]
  when 0 
    error_status << "#{error_cnt + warn_cnt} problems." 
  when 1 
    error_status << "#{error_cnt + warn_cnt} problems - #{error_files}" 
  else
    error_status << "#{error_cnt + warn_cnt} problems\n#{error_long}" 
end

puts error_status
exit rc

Comments

Leave a response

Comments