securing a unix system - suggestions on checking for intrusions

[Robin Paulson]

this was posted on /. earlier, in the wake of the kernel.org intrusion
last month.

http://article.gmane.org/gmane.linux.kernel/1197924

it shows some interesting capabilities rpm has which deb doesn't

i'm going to try those i can on my ubuntu system. any suggestions for
anything else that might be useful to check?

or any comments on the usefulness of their suggestions?

cheers,
--
robin

http://bumblepuppy.org/blog/?p=237 - government bill to remove basic
human rights in NZ
http://universitywithoutconditions.ac.nz - a free university for
Auckland

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Nick Rout]

this was posted on /. earlier, in the wake of the kernel.org intrusion last
month.
http://article.gmane.org/gmane.linux.kernel/1197924
it shows some interesting capabilities rpm has which deb doesn't

I didn't see them, there was just a more laborious checking script for
debs as opposed to the built in option for rpms

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Robin Paulson]

On Thu, Oct 6, 2011 at 10:22 AM, Robin Paulson

Post by Robin Paulson
this was posted on /. earlier, in the wake of the kernel.org
intrusion last
month.
http://article.gmane.org/gmane.linux.kernel/1197924
it shows some interesting capabilities rpm has which deb doesn't

I didn't see them, there was just a more laborious checking script for
debs as opposed to the built in option for rpms

"3. Verify that your packages are really signed with the distro's keys.

Here's a bash snippet that can do this on a rpm based system to
verify that the packages are signed with any key, not necessarily
your distro's key. That exercise is left for the reader:

for package in `rpm -qa`; do
sig=`rpm -q --qf '%{SIGPGP:pgpsig}\n' $package`
if [ -z "$sig" ] ; then
# check if there is a GPG key, not a PGP one
sig=`rpm -q --qf '%{SIGGPG:pgpsig}\n' $package`
if [ -z "$sig" ] ; then
echo "$package does not have a signature!!!"
fi
fi
done

Unfortunately there is no known way of verifying this on
Debian-based
systems."

ok, plural was wrong

[Robin Sheat]

Op donderdag 06-10-2011 om 12:53 uur [tijdzone +1300], schreef Robin

Post by Robin Paulson
Unfortunately there is no known way of verifying this on
Debian-based
systems."

Apt checks the sigs of the repo and the packages when you install from
it, and whinges mightily if they don't match a known key.

I'm not totally sure what that snippet is doing, but it looks like it's
seeing that there's sig on packages that are already installed, which I
don't see the point of.

Robin.

[Tomislav]

Post by Robin Sheat
Op donderdag 06-10-2011 om 12:53 uur [tijdzone +1300], schreef Robin

Post by Robin Paulson
Unfortunately there is no known way of verifying this on
Debian-based
systems."

Apt checks the sigs of the repo and the packages when you install from
it, and whinges mightily if they don't match a known key.
I'm not totally sure what that snippet is doing, but it looks like it's
seeing that there's sig on packages that are already installed, which I
don't see the point of.
Robin.

It is useful if the repo key has been changed and you need to reinstall
everything that uses the old key for verification. But if you are that
worried a clean install is in order anyway.

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Robin Paulson]

Post by Robin Sheat
Op donderdag 06-10-2011 om 12:53 uur [tijdzone +1300], schreef Robin

Post by Robin Paulson
Unfortunately there is no known way of verifying this on
Debian-based
systems."

Apt checks the sigs of the repo and the packages when you install from
it, and whinges mightily if they don't match a known key.
I'm not totally sure what that snippet is doing, but it looks like it's
seeing that there's sig on packages that are already installed, which I
don't see the point of.

i would guess to check for suspicious files that have replaced files
from the repo? i.e. ensuring you've still got the exact binary installed
that you want/should have?

[Robin Sheat]

Op donderdag 06-10-2011 om 13:43 uur [tijdzone +1300], schreef Robin

Post by Robin Paulson
i would guess to check for suspicious files that have replaced files
from the repo? i.e. ensuring you've still got the exact binary installed
that you want/should have?

But, is it checking the sigs on the binaries, or just that the RPM you
installed was fine?

Actually, if every manifest contains the hash of each installed file,
and that manifest is signed, then it would be useful to verify the
integrity of those aspects of the system.

But I don't think it's doing that.

Robin.

[Robin Paulson]

Post by Robin Sheat

Post by Robin Paulson
i would guess to check for suspicious files that have replaced files
from the repo? i.e. ensuring you've still got the exact binary installed
that you want/should have?

But, is it checking the sigs on the binaries, or just that the RPM you
installed was fine?

i think the latter

Post by Robin Sheat
Actually, if every manifest contains the hash of each installed file,
and that manifest is signed, then it would be useful to verify the
integrity of those aspects of the system.
But I don't think it's doing that.

me neither. any suggestions on software/script to do the former, i.e.
check every file?

[Bryan Baldwin]

Post by Robin Paulson

Post by Robin Sheat

Post by Robin Paulson
i would guess to check for suspicious files that have replaced files
from the repo? i.e. ensuring you've still got the exact binary installed
that you want/should have?

But, is it checking the sigs on the binaries, or just that the RPM you
installed was fine?

i think the latter

Post by Robin Sheat
Actually, if every manifest contains the hash of each installed file,
and that manifest is signed, then it would be useful to verify the
integrity of those aspects of the system.
But I don't think it's doing that.

me neither. any suggestions on software/script to do the former, i.e.
check every file?

Something like this could help.

http://www.tripwire.org/

http://en.wikipedia.org/wiki/Open_Source_Tripwire

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Robin Paulson]

Post by Bryan Baldwin

Post by Robin Paulson
me neither. any suggestions on software/script to do the former, i.e.
check every file?

Something like this could help.
http://www.tripwire.org/
http://en.wikipedia.org/wiki/Open_Source_Tripwire

excellent, i'll give it a whirl. but it brings up this question:
is there anything to check the tripwire database, to ensure it hasn't
been tampered with?

[r***@webworxshop.com]

Post by Robin Paulson

Post by Bryan Baldwin

Post by Robin Paulson
me neither. any suggestions on software/script to do the former, i.e.
check every file?

Something like this could help.
http://www.tripwire.org/
http://en.wikipedia.org/wiki/Open_Source_Tripwire

is there anything to check the tripwire database, to ensure it hasn't
been tampered with?

Sounds like a Russian doll problem to me. Any check you can do will
need a check on itself and so on...

You could be reasonably sure by taking a sha1 hash of the tripwire
database when your system is in a known good state and storing it on
another system (encrypted if necessary).

Rob

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Robin Paulson]

Post by r***@webworxshop.com

Post by Robin Paulson

Post by Bryan Baldwin
Something like this could help.
http://www.tripwire.org/
http://en.wikipedia.org/wiki/Open_Source_Tripwire

is there anything to check the tripwire database, to ensure it hasn't
been tampered with?

Sounds like a Russian doll problem to me. Any check you can do will
need a check on itself and so on...
You could be reasonably sure by taking a sha1 hash of the tripwire
database when your system is in a known good state and storing it on
another system (encrypted if necessary).

sorry, i should confess i was kind of having fun. we are always inside
the matrix, etc.

thanks for the suggestion, though. that is a solution, i could always
email it somewhere else each time the hash is created. or if the
database is not too large, email that somewhere

[Bryan Baldwin]

Post by Robin Paulson
thanks for the suggestion, though. that is a solution, i could always
email it somewhere else each time the hash is created. or if the
database is not too large, email that somewhere

Better encrypt the email. And don't tell anyone your email encryption
keys. Hash your email encryption and email that, too!

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Robin Sheat]

Op donderdag 06-10-2011 om 15:24 uur [tijdzone +1300], schreef Robin

Post by Robin Paulson
is there anything to check the tripwire database, to ensure it hasn't
been tampered with?

It gets manually generated, so when you do that, burn it onto a CD-R
(not -RW :) and leave that in the drive.

It's by no means a foolproof system, but it's better'n nothing.

Robin.

[Andrew Simpson]

On Thu, 06 Oct 2011 17:33:47 +1300

Post by Robin Sheat
Op donderdag 06-10-2011 om 15:24 uur [tijdzone +1300], schreef Robin

Post by Robin Paulson
is there anything to check the tripwire database, to ensure it hasn't
been tampered with?

It gets manually generated, so when you do that, burn it onto a CD-R
(not -RW :) and leave that in the drive.
It's by no means a foolproof system, but it's better'n nothing.

Take an MD5 sum (or SHA1 if you prefer) and write down on a piece of
paper.

I've yet to see malware that can change writing on paper...

Andrew

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Robin Sheat]

Op donderdag 06-10-2011 om 18:01 uur [tijdzone +1300], schreef Andrew

Post by Andrew Simpson
Take an MD5 sum (or SHA1 if you prefer) and write down on a piece of
paper.
I've yet to see malware that can change writing on paper...

Don't have to. Just need to change md5sum/sha1sum so they output the
original value when run against that database :)

Robin.

[Cliff Pratt]

Post by Bryan Baldwin

Post by Robin Paulson

Post by Robin Sheat

Post by Robin Paulson
i would guess to check for suspicious files that have replaced files
from the repo? i.e. ensuring you've still got the exact binary installed
that you want/should have?

But, is it checking the sigs on the binaries, or just that the RPM you
installed was fine?

i think the latter

Post by Robin Sheat
Actually, if every manifest contains the hash of each installed file,
and that manifest is signed, then it would be useful to verify the
integrity of those aspects of the system.
But I don't think it's doing that.

me neither. any suggestions on software/script to do the former, i.e.
check every file?

Something like this could help.
http://www.tripwire.org/
http://en.wikipedia.org/wiki/Open_Source_Tripwire

I don't see the point of tripwire and the like. It's shutting the stable
door after the horse has bolted. It's far better to stop the files being
changed in the first place.

Cheers,

Cliff






_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Jim Cheetham]

Post by Cliff Pratt
I don't see the point of tripwire and the like. It's shutting the stable
door after the horse has bolted. It's far better to stop the files being
changed in the first place.

You cannot provide 'perfect' security on a useable system, therefore
you have to accept the possibility that someone will exploit a
weakness that you did not know about.

So, given that an attacker exists, would you like to have some chance
of detecting whether or not they have been attacking you (i.e. log
analysis) or have actually succeeded in modifying your system (i.e.
tripwire & friends)?

An alert from tripwire (assume it is 100% correct here) tells you that
your system has been compromised, and therefore before putting any
*new* important data on that system, you will probably choose to fix
it.

So tripwire and the like have a very good purpose. Where they can go
wrong is where the attacker knows that you are using tripwire, and
attacks tripwire as well. This doesn't work if you run tripwire
'properly' (i.e. from read-only media, on a snapshot of the system
disk rather than the live machine).

IDS systems like tripwire have a lot of value, but the cost of
implementing them needs to be compared to the losses from an attack.

-jim

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Bruce Kingsbury]

Fairly sure deb can verify all the binaries installed by a package .. I
think I had to do it once...

--
sent from my Ideos

Post by Robin Sheat
Op donderdag 06-10-2011 om 13:43 uur [tijdzone +1300], schreef Robin

Post by Robin Paulson
i would guess to check for suspicious files that have replaced files
from the repo? i.e. ensuring you've still got the exact binary installed
that you want/should have?

But, is it checking the sigs on the binaries, or just that the RPM you
installed was fine?
Actually, if every manifest contains the hash of each installed file,
and that manifest is signed, then it would be useful to verify the
integrity of those aspects of the system.
But I don't think it's doing that.
Robin.

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Volker Kuhlmann]

Post by Robin Sheat
But, is it checking the sigs on the binaries, or just that the RPM you
installed was fine?

For rpm that is the same thing. Before installing, it checks for signed
packages whether the signatures matches a key on the rpm keyring, so
there is a point in making sure all packages are signed. During
installation it creates a database containing metadata incl hashes of
all files, and all that metadata can be checked again afterwards. I
don't know how difficult it would be to tamper with the database, but
backing that up is trivial.

Volker

[Jaco]

Post by Robin Paulson
i would guess to check for suspicious files that have replaced files
from the repo? i.e. ensuring you've still got the exact binary
installed that you want/should have?

FYI - if the repo's poisoned, you're borked anyway:
http://www.theregister.co.uk/2011/07/05/ftp_backdoor_shenanigans

The "many eyes" theory does not guarantee security

- J

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Volker Kuhlmann]

Post by Jaco
http://www.theregister.co.uk/2011/07/05/ftp_backdoor_shenanigans

That's just nonsense. It depends on whether the package was signed with
a secure key. If so, you're fine. I remember the time at about 1997 when
downloads cost $5/MB (uni, 3x that in town) from offshore but nothing
onshore, and I could fetch distro packages from any odd ftp server dodgy
or not, it didn't matter. At that time deb hadn't even heard of package
signing, or surely hadn't done anything about it.

Volker

[Andrew Simpson]

On Thu, 06 Oct 2011 12:30:27 +1300

On Thu, Oct 6, 2011 at 10:22 AM, Robin Paulson

Post by Robin Paulson
this was posted on /. earlier, in the wake of the kernel.org
intrusion last month.
http://article.gmane.org/gmane.linux.kernel/1197924
it shows some interesting capabilities rpm has which deb doesn't

I didn't see them, there was just a more laborious checking script for
debs as opposed to the built in option for rpms

Actually the command 'rpm --verify --all', which is at the top of the
article, does much, much more than apt can do in verifying packages. [1]

The rpm keeps a database of md5sum, permissions, size, date, etc on each
installed file and each package. Hence this can very useful in checking
what's changed in a system - providing you can trust the database.[2]

I've used this when I used Redhat 5, 6, 7, 8 & 9. It does tend to
spit out a lot of false positives - config files that are manually
changed, files in /dev, etc.

Since changing to apt based distributions, I've found no equivalent.
I guess you would have to install something like Tripwire.

Andrew

[1] http://www.rpm.org/max-rpm/ch-rpm-verify.html
[2] http://www.sans.org/security-resources/idfaq/rpm.php

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Jim Cheetham]

On Thu, Oct 6, 2011 at 6:17 PM, Andrew Simpson

Post by Andrew Simpson
The rpm keeps a database of md5sum, permissions, size, date, etc on each
installed file and each package. Hence this can very useful in checking
what's changed in a system - providing you can trust the database.[2]
...
Since changing to apt based distributions, I've found no equivalent.

debsums might give you some of that. It looks in
/var/lib/dpkg/info/*.md5sums and verifies your system files against
that.

Of course a smart attacker will change the checksum after they patch
the associated binary; to prevent this the checksum files need to be
signed with the same keys as the packages, which isn't happening, so
the results of this are potentially less useful than the rpm
equivalent (which I haven't investigated).

However, it is useful for less malicious damage ... here's an example
where I have deliberately altered /usr/bin/sup-mail from the sup-mail
package ...

***@maru:/var/lib/dpkg/info$ debsums sup-mail
/usr/bin/sup-add OK
/usr/bin/sup-cmd OK
/usr/bin/sup-config OK
/usr/bin/sup-dump OK
/usr/bin/sup-mail FAILED

Once I accidentally caused the deletion of some files in /lib ... we
were able to restore these by using debsums to check every package
that expected to put anything into /lib, and then re-install from the
specific .debs still sitting in /var/cache/apt/archives ...

-jim
-jim

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Robin Sheat]

Op vrijdag 07-10-2011 om 10:37 uur [tijdzone +1300], schreef Jim

Post by Jim Cheetham
Of course a smart attacker will change the checksum after they patch
the associated binary; to prevent this the checksum files need to be
signed with the same keys as the packages, which isn't happening, so
the results of this are potentially less useful than the rpm
equivalent (which I haven't investigated).

Were I said attacker, I'd put my key into the 'trusted keys' store and
resign the things I changed with that.

Robin.