Re: MD5 checksum changed

From: Tim Haynes (usenet-20040827_at_stirfried.vegetable.org.uk)
Date: 08/27/04


Date: Fri, 27 Aug 2004 17:38:02 +0100

spam@softluck.plus.com (Jonathan L Cunningham) writes:

>>> Replace the package withe the --force option.
>>
>>No no no no NO!!!!
>>
>>Destroying evidence like this is sheer lunacy. Investigate until you find
>>the answer.
>
> In an ideal world, I could replace that machine with another, copy all
> the binaries off it, reinstall everything again from the original
> installation disks, copy all those off somewhere else, then do a diff on
> all the files.
>
> But what would that tell me that would justify my time? (That's a serious
> question: I'd *like* to know what the cause was, but if, say, I could
> just install the latest version of RedHat instead of using a (few months
> old) version of Mandrake, mere curiosity is not sufficient
> justification.)

It would tell you how secure the box was - a clean installation off new
trusted media (CDs) with audited data only, uptodate, firewalled, only
running what it needs, with IDS and nIDS and regular log-watching - this is
well-defined.
Something that's been running a while and you've lost the certainty of
integrity ain't the same. And, more to the point, if you go around
replacing bits of the machine willy-nilly without checking for or tackling
underlying (potential) problems, that certainty is *never* going to come
back.

If you find the box to be clean, all to the good; if you find it's been
cracked, you might have a clue for things to look out for in the future
(firewalling, minimalism, whatever config-changes enhance the box's
security to avoid the class of problem that lead to a crack). Them's the
benefits.

> Meanwhile, we still need to send and receive e-mail, and it would be
> nice if the web-site remained up.

Sure. The logistics are up to you - it may be time to gain certainty with a
clean installation and do it properly; it may be there's a simple
explanation (prelink or HD corruption), if you're lucky. Either way you
should resolve the changed binary, find *why* it happened or otherwise
guarantee that the machine is clean.

>>Here's two more ideas to consider: a) hardware cockup leading to
>>filesystem corruption manifesting itself as a screwup in that binary -
>>run a fsck on the partition in quesiton in single-user mode;
>
> Yes, that seems like a good idea: and it won't take so long I need to
> install a "spare" machine while I'm doing it.

Hope so :)

>>b) Are you running `prelink'? See
>><http://www.google.com/groups?hl=en&lr=&ie=UTF-8&threadm=86k6yjoyfr.fsf%40potato.vegetable.org.uk&rnum=1&prev=/groups%3Fq%3D%2B%2522Paul%2BBlack%2522%2B-moonlight%2Bgroup:uk.comp.os.linux%2Bauthor:Haynes%26hl%3Den%26lr%3D%26ie%3DUTF-8%26scoring%3Dd%26selm%3D86k6yjoyfr.fsf%2540potato.vegetable.org.uk%26rnum%3D1>
>
> Sorry, I followed this link and I couldn't make see what you are
> referring to.

Ah, it was an interesting thread on uk.comp.os.linux a while ago; Paul had
a box where tripwire started saying a binary had changed, transpired he was
running this prelink thing (<http://freshmeat.net/projects/prelink/>) which
modified it...

> I see in your other reply in this thread that you mentioned iptables
> there, too. I'm not running iptables on this machine, but have been
> relying on the separate hardware firewall.

A simple iptables firewall is so easy to stick on a box, I'd say use one
network-wide thing *and* per-machine.

> I assume there is some additional benefit I would get? But you'd need to
> explain v-e-r-y s-l-o-w-l-y in words of one syllable what I'd need to do.

Firewalling a machine individually? My method:

* Ascertain how the existing firewall system works - whether it uses a raw
script or an iptables-save output. Look in /etc/init.d/ and trace it back
from there, what config files the appropriate service script references.

* Grab my firewall script -
<http://spodzone.org.uk/packages/secure/iptables.sh> - and edit it. There
are sections for handling return packets for things we've asked for, and
for services we provide, and then the rest is dropped. (If you want to run
ssh, leave that line in; if you run mail and web-servers, clone twice and
replace 22 with 25 and 80, etc.)

* Run it, check that it works - you should get different output in `iptables
-nL' that looks vaguely like the rules, packets to various ports should be
permitted or dropped, from various places. Test a few.

* If appropriate, maybe try
  iptables-save > /var/lib/iptables/rules-save
(on Gentoo, at least). Either that or make your version of my script run on
startup by fair meansa or foul.

That's, erm, it. :)

> We're not big enough to hire a full-time security consultant, so I have
> to try and fit this stuff into all my other work. (Anyone who explains
> that therefore we deserve to go out of business will be politely
> ignored.)

You do need to dedicate some time or resource to it - read-up on linux
security (<http://www.linuxsecurity.com/> being a possible start) and
firewalling (<http://www.netfilter.org/> also). If you don't have the time,
hire a temporary consultant.
Try to persuade your boss that if you don't invest the time now in a nice
solid lump (day or two, maybe a new machine) and get a regular routine
(apply package updates and read logfiles for the first 10mins of every
day), then the future upstream costs *when* a crack really happens and have
have to do damage assessment and limitation and repairs will take over a
week of your time, being caught with your knickers down with no spare
machine having to do a reinstallation while your web & mail servers are
offline as you sweat your guts up.

~Tim

-- 
  16:58:06 up 10 days, 32 min,  4 users,  load average: 0.05, 0.04, 0.06
piglet@stirfried.vegetable.org.uk |The light of the world keeps shining,
http://spodzone.org.uk/cesspit/   |Bright in the primal glow


Relevant Pages

  • iptables script migration from Red Hat 9 To Fedora Core 4
    ... I'm working towards replacing an old firewall box with a new one. ... old box runs Red Hat Linux 9. ... I have an iptables script that works well ...
    (Fedora)
  • Firewall Rules Summary
    ... Subject: Firewall Rules Summary ... This script is provided "as is" with no implied warranty. ... this came from various howtos and articles on iptables that existed around ... #specific port denies>1024 tcp ...
    (Focus-Linux)
  • Re: [opensuse] IP tables - DIY
    ... At that time there was little YaST for the firewall, and I worked with the config file directly. ... Not knowing how IP-Filters work, it was not clear to me what exactly is happening in my firewall. ... I learnt to list the kernel IP-Table (iptables -nL), and was totally clueless what I saw there and how all this works. ... A friend came over and gave me 2-3 hours of tutoring on kernel IP-Filtering while we wrote a complete script for my network with a VPN included in the rules. ...
    (SuSE)
  • Re: some reality about iptables, please
    ... >>the script which can only be run by a root user. ... but it could re-inforce the fact that maybe running your iptables ... "I'm a packet filtering interface not a firewall tool." ... Generally Debian systems run at init runlevel 3 (this is a change if ...
    (Debian-User)
  • Re: some reality about iptables, please
    ... First post folks so I'm unsure if we top post or not round here but everyone ... friend before I even got close to understanding iptables. ... >great scripts for platform hardening but I prefer shorewall's firewall ... >script can be used to restore a saved configuration at boot time. ...
    (Debian-User)