SPF records and DKIM signatures on ubuntu/postfix/mailman

[Robin Paulson]

i setup a public mailman discussion list recently, and the people who
are subscribed via gmail are getting the following warning when they
open messages sent via the list:
"This message may not have been sent by: <subscriber>@gmail.com Learn
more Report phishing"

the warning includes a link to this page, which gives a fairly
comprehensive explanation:
https://mail.google.com/support/bin/answer.py?hl=en-GB&ctx=mail&answer=185812

however, i'd like to get some further views on the matter.
is this a major enough problem to warrant doing anything about?
are users likely to unsubscribe when they see this message?
any suggestions on the best way to implement it, on a postfix/ubuntu
lucid system?
anything else which is relevant?

cheers,

[Jaco]

Just looking at this quickly (I've seen such cases in the past), this
sort of issue crops up when the sending IP & the domain-lane to not
quite match up.

SPF is a way to "bless" specific IP's with the authority to send mail
for a particular domain, and is a value assigned to the TXT attribute of
your domain's DNS.
In the past I've used openspf.org to do a quick, back-of-the-envelope
calculation that would pass muster, but have not used it in a while.
In most cases "v=spf1 mx a ~all" (I think) is good enough.

Ensuring your rDNS is set up correctly would also help to vastly improve
your "IP reputation"

- Jaco

Post by Robin Paulson
i setup a public mailman discussion list recently, and the people who
are subscribed via gmail are getting the following warning when they
more Report phishing"
the warning includes a link to this page, which gives a fairly
https://mail.google.com/support/bin/answer.py?hl=en-GB&ctx=mail&answer=185812
however, i'd like to get some further views on the matter.
is this a major enough problem to warrant doing anything about?
are users likely to unsubscribe when they see this message?
any suggestions on the best way to implement it, on a postfix/ubuntu
lucid system?
anything else which is relevant?
cheers,

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[David Pando]

Post by Robin Paulson
https://mail.google.com/support/bin/answer.py?hl=en-GB&ctx=mail&answer=185812

I don't think Google and many other domains are picky about SPF
records or DKIM as not many domains implement them properly , although
they are good additions to prevent your emails from hitting the spam
can.

Having a correct rdns record is quite import though, as most remote
mail servers will discard incoming emails on the spot if you mail
server does not have a proper record.To check yours you can use
http://postmaster.aol.com/cgi-bin/plugh/rdns.pl and enter the server
IP.

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Mark Foster]

Post by David Pando

Post by Robin Paulson
https://mail.google.com/support/bin/answer.py?hl=en-GB&ctx=mail&answer=185812

I don't think Google and many other domains are picky about SPF
records or DKIM as not many domains implement them properly , although
they are good additions to prevent your emails from hitting the spam
can.
Having a correct rdns record is quite import though, as most remote
mail servers will discard incoming emails on the spot if you mail
server does not have a proper record.To check yours you can use
http://postmaster.aol.com/cgi-bin/plugh/rdns.pl and enter the server
IP.

... or simply use dig -x.

The key with rDNS is to ensure that you have a matching forward and
reverse pair. The rDNS doesn't have to pertain directly to the host
you're operating Mailman under.

I was going to use the linux.net.nz host as an example, but I can't as
it doesn't have rDNS itself. I will point this out to someone.
Meanwhile my own machine:

***@hawkeye:~$ dig lists.nzoss.org.nz A +short
121.73.240.178
***@hawkeye:~$ dig -x 121.73.240.178 +short
121-73-240-178.broadband.telstraclear.net.
***@hawkeye:~$ dig 121-73-240-178.broadband.telstraclear.net A +short
121.73.240.178
***@hawkeye:~$


There's a matching pair, so it's 'cleaner' than a machine that doesn't
have it.

Mark.




_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Volker Kuhlmann]

Post by Mark Foster
... or simply use dig -x.

No, not good enough unless you use a DNS server *other* than your ISP's.

E.g. my Telstra cable IP gives a valid response (albeit a .in-addr.arpa
one) in the normal situation (where the ISP's DNS is configured), but
returns NXDOMAIN when using opendns.

Volker

[Mark Foster]

Post by Volker Kuhlmann

Post by Mark Foster
... or simply use dig -x.

No, not good enough unless you use a DNS server *other* than your ISP's.
E.g. my Telstra cable IP gives a valid response (albeit a .in-addr.arpa
one) in the normal situation (where the ISP's DNS is configured), but
returns NXDOMAIN when using opendns.

As far as the world is concerned the IP you're on does _not_ have a
valid forward/reverse pair, based on the headers of the last message you
sent to the list anyway.

The fact it works inside of TCL only is actually broken, and I'd suggest
that you raise it with TCL.
For comparison, the earlier forward/reverse pair I posted are also on
TCL and work fine.

Checking your DNS works outside of the authoritive/hosting server is
something that you should always do anyway, if the means presents. Many
ISPs are now blocking external lookups for security reasons, swapping
shells with someone on a different ISP can be very useful for diagnostic
purposes.

Cheers
Mark.



A response from within .in-addr.arpa usually means the PTR zone is
subdelegated below the /24 which require

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Volker Kuhlmann]

Post by Mark Foster
As far as the world is concerned the IP you're on does _not_ have a
valid forward/reverse pair, based on the headers of the last message you
sent to the list anyway.

Yes, so I figured. TCL is going steadily downhill. Beats me how they
manage to be ISP of the year all the time.

Post by Mark Foster
The fact it works inside of TCL only is actually broken, and I'd suggest
that you raise it with TCL.

Do they still care? I'd put more effort in if it had a tangible benefit
for me, but sending email directly without relaying through the ISP
stopped working years ago.

Post by Mark Foster
Checking your DNS works outside of the authoritive/hosting server is
something that you should always do anyway, if the means presents. Many
ISPs are now blocking external lookups for security reasons,

For servers that send mail - certainly, for domestic cable Internet use
it's not so critical. What do you mean by "blocking external lookups"?
External DNS lookups? Blocked, or redirected to their own DNS resolver?
I have no indication that that is the case with TCL. Web cache yes to
varying levels of transparency, but DNS?

Volker

[Mark Foster]

Post by Volker Kuhlmann

Post by Mark Foster
As far as the world is concerned the IP you're on does _not_ have a
valid forward/reverse pair, based on the headers of the last message you
sent to the list anyway.

Yes, so I figured. TCL is going steadily downhill. Beats me how they
manage to be ISP of the year all the time.

+1. Was glad to move away from them to be honest. I had cable when I was
in Wellington for a while.

Post by Volker Kuhlmann

Post by Mark Foster
The fact it works inside of TCL only is actually broken, and I'd suggest
that you raise it with TCL.

Do they still care? I'd put more effort in if it had a tangible benefit
for me, but sending email directly without relaying through the ISP
stopped working years ago.

If you raise it with them, it's their job to care. Failure to have a
matching pair can make connections to FTP and SSH servers take a while
to validate you as they try to verify your DNS, for example.

Post by Volker Kuhlmann

Post by Mark Foster
Checking your DNS works outside of the authoritive/hosting server is
something that you should always do anyway, if the means presents. Many
ISPs are now blocking external lookups for security reasons,

For servers that send mail - certainly, for domestic cable Internet use
it's not so critical. What do you mean by "blocking external lookups"?
External DNS lookups? Blocked, or redirected to their own DNS resolver?
I have no indication that that is the case with TCL. Web cache yes to
varying levels of transparency, but DNS?

Think of it like SMTP.
There's essentially two types of DNS servers

1) Resolvers of ISP X that're set up so that clients of ISP X can point
DNS lookups at them; they then reach out to the world for information on
the domains you wish to connect to.

2) Authoritive DNS servers which host domain names.

Best practise is not to have 1) and 2) on the same machines, but it's
been known to happen.

As an ISP it's quite appropriate for you to set up the configuration of
1) above so that if the DNS lookup comes from outside of your own
local/trusted IP space, the lookup fails. This was commonly done when
the major DNS response vulnerability came up a coupla years ago[1], to
reduce the threat vector. It also reduces the load on ISP DNS servers
from people who aren't paying customers. (How many people hardcoded
Xtra's DNS servers into their workstations, then changed ISP's and
continued to use Xtra's machines for lookups?)

Obviously you can't restrict the Authoritive DNS servers - as then
people wont be able to look up DNS for stuff you host. But there's no
reason that an ISP needs to operate resolvers that're open to folks
other than their own customers.

I mentioned this because these policies can make it hard to externally
test DNS. For example I know Orcon's DNS platform rejects queries from
outside:

***@hawkeye:~$ dig blakjak.net mx @ns1.orcon.net.nz

; <<>> DiG 9.7.1-P2 <<>> blakjak.net mx @ns1.orcon.net.nz
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 50276
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

(i'm not an Orcon customer).

Mark.

[1]
http://www.secureworks.com/research/articles/other_articles/dns-cache-poisoning/

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Volker Kuhlmann]

Post by Mark Foster
+1. Was glad to move away from them to be honest. I had cable when I was
in Wellington for a while.

It's all relative to the available alternatives, which are unfortunately
lacking on the better side. Plenty on the worse side esp when it comes
to latency (really matters for ssh, couldn't care less about games) and
the cows will come home before I use xtra.

Post by Mark Foster
Think of it like SMTP.
There's essentially two types of DNS servers

I see. You were thinking of serving recursive DNS requests from external
to the ISP, I was thinking internal to the ISP so the security aspect
didn't make sense.

Post by Mark Foster
from people who aren't paying customers. (How many people hardcoded
Xtra's DNS servers into their workstations, then changed ISP's and
continued to use Xtra's machines for lookups?)

Well the reverse would hold too, but who in their right mind would
*choose* to use xtra DNS? I had heaps of problems with pages not loading
and resulting in a 404, but an immediate reload would load them fine.
Problems went away when switching from xtra DNS to another one. (pfsense
with DNS forwarder.)

Post by Mark Foster
I mentioned this because these policies can make it hard to externally
test DNS. For example I know Orcon's DNS platform rejects queries from

Having a shell available somewhere else is very useful for investigating
lots of things, but for querying other DNS servers it's not needed. Sure
xtra, orcon, clear and a few smaller ones block requests from
non-customers for domains they don't host, but there are plenty left in
NZ who don't. For checking from overseas you can use the big public DNS
servers from opendns (just spot their ad server and replace that with
NXDOMAIN), cisco or google.

The reason for blocking requests from non-customers has more to do with
"why do something for the competition" than any security. At least xtra
was doing it much earlier than the dns security issues you mention.

Volker

[Mark Foster]

Post by Volker Kuhlmann

Post by Mark Foster
+1. Was glad to move away from them to be honest. I had cable when I was
in Wellington for a while.

It's all relative to the available alternatives, which are unfortunately
lacking on the better side. Plenty on the worse side esp when it comes
to latency (really matters for ssh, couldn't care less about games) and
the cows will come home before I use xtra.

I still have some resources hosted with a friend in Wellington who's on
Cable. I'm relatively happy that we don't need to engage their tech
support often, and that it 'just works' most of the time.

Post by Volker Kuhlmann

Post by Mark Foster
from people who aren't paying customers. (How many people hardcoded
Xtra's DNS servers into their workstations, then changed ISP's and
continued to use Xtra's machines for lookups?)

Well the reverse would hold too, but who in their right mind would
*choose* to use xtra DNS? I had heaps of problems with pages not loading
and resulting in a 404, but an immediate reload would load them fine.
Problems went away when switching from xtra DNS to another one. (pfsense
with DNS forwarder.)

Plenty of people threw Alien and Terminator (or one or the other) into
their systems in order to present some sort of redundancy, or simply
because they knew what their IP's were off the top of their head.
I saw this many times when working for both Xtra and for other ISPs in
support roles in the early 2000s.

Post by Volker Kuhlmann

Post by Mark Foster
I mentioned this because these policies can make it hard to externally
test DNS. For example I know Orcon's DNS platform rejects queries from

Having a shell available somewhere else is very useful for investigating
lots of things, but for querying other DNS servers it's not needed. Sure
xtra, orcon, clear and a few smaller ones block requests from
non-customers for domains they don't host, but there are plenty left in
NZ who don't. For checking from overseas you can use the big public DNS
servers from opendns (just spot their ad server and replace that with
NXDOMAIN), cisco or google.

Yeah there's a few options, but in particular I found that a complaint
from an Orcon customer that (xyz website doesn't resolve) was very hard
to troubleshoot without being able to do my own DNS queries against
their servers.
One of the reasons that blocking external resolver lookups causes
problems for support techs.

Post by Volker Kuhlmann
The reason for blocking requests from non-customers has more to do with
"why do something for the competition" than any security. At least xtra
was doing it much earlier than the dns security issues you mention.

Actually I don't believe so; for a long time Alien and Terminator were
the 'only' DNS servers operated by Xtra; both hosting and
resolver-purposed. This meant the machines had to remain wide-open to
cater for queries against the hosted domains.
It was relatively recently that they changed the auto-assigned-DNS
servers for customers on Dialup and DSL to try to share the load, as the
machines were struggling (and it was a very obvious SPOF when the
machines fell over) - and it also causes problems when hosted domains
are transferred and stale DNS entries remain...

Mark.

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Steve Holdoway]

On Sat, 2011-10-15 at 11:39 +1300, Volker Kuhlmann wrote:
[snip]

Post by Volker Kuhlmann
The reason for blocking requests from non-customers has more to do with
"why do something for the competition" than any security. At least xtra
was doing it much earlier than the dns security issues you mention.
Volker

Sadly this is no longer true. With the increasing acceptance of DNSSEC,
what used to be a trivial amount of UDP data passed back and forth is
now a lot, lot larger, pushing a lot of the traffic over to TCP as well,
with all that added overhead to content with as well.

And, with increasing reliance on 'cloud computing' - whatever your
definition - something like DNSSEC just has to be in place.

So, the days where I ran DNS for the EMEA for a major international
courier service off a pair of very old ( HP equivalent of ) i386's is
long, long gone.

Steve

[Martin D Kealey]

Post by Volker Kuhlmann
The reason for blocking requests from non-customers has more to do with
"why do something for the competition" than any security.

There are exploits in the wild that can poison a DNS cache, based on
guessing (read "brute-forcing") request ID numbers.

A good way to guard against your ISPs resolver cache getting poisoned is
simply not to use it: run your own resolver instead. But most people won't
do that.

Restricting requests to "just customers" will cut by three orders of
magnitude the number of 'bots that can attack you. (Or at least, it will if
you're a "small" ISP, being one whose customers comprise less than 0.1% of
worldwide internet users, which includes every ISP in NZ including Xtra.)

Another way to reduce the chances of an attack succeeding is to use
different outgoing IP addresses for incoming and outgoing queries. Running
many small DNS caches behind a load balancer (rather than just a few large
ones) also helps.

-Martin

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Robin Sheat]

Post by Volker Kuhlmann
Beats me how they
manage to be ISP of the year all the time.

I think it's because every time that rolls around, they email all their
subscribers (of which I'm one) asking to vote for them. I laugh and
delete it, but I guess many people don't.

Robin.

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Craig Box]

Post by Robin Paulson

Post by Robin Paulson
the warning includes a link to this page, which gives a fairly

https://mail.google.com/support/bin/answer.py?hl=en-GB&ctx=mail&answer=185812
I don't think Google and many other domains are picky about SPF
records or DKIM as not many domains implement them properly , although
they are good additions to prevent your emails from hitting the spam
can.

This message may not have been sent by: ***@gmail.com

Don't trust his advice! :-)
_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Bruce Clement]

No solution, just another data point for you.

i setup a public mailman discussion list recently, and the people who are
subscribed via gmail are getting the following warning when they open
more Report phishing"

This email address forwards to my gmail account. I see this message quite
regularly on mails to this list and the Auckland Lug list. For example, it's
showing on David Pando's email in this thread.

I don't know if it's because I'm forwarding email from my server to Gmail
when my server isn't listed as a valid sender in Gmail's SPF record or just
because gmail is "unexpectedly" receiving mail from outside Gmail that
claims to come from Gmail.

Cheers

Bruce
--
Bruce Clement

Home: http://www.clement.co.nz/
Twitter: http://twitter.com/Bruce_Clement
Directory: http://www.searchme.co.nz/

"Before attempting to create something new, it is vital to have a good
appreciation of everything that already exists in this field." Mikhail
Kalashnikov
_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Mark Foster]

Post by Bruce Clement
No solution, just another data point for you.

i setup a public mailman discussion list recently, and the people who are
subscribed via gmail are getting the following warning when they open
more Report phishing"

This email address forwards to my gmail account. I see this message quite
regularly on mails to this list and the Auckland Lug list. For example, it's
showing on David Pando's email in this thread.
I don't know if it's because I'm forwarding email from my server to Gmail
when my server isn't listed as a valid sender in Gmail's SPF record or just
because gmail is "unexpectedly" receiving mail from outside Gmail that
claims to come from Gmail.

https://mail.google.com/support/bin/answer.py?answer=185812

Appears to be both SPF and DKIM based.

Mark.

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Jaco]

Post by Mark Foster
Appears to be both SPF and DKIM based.

I've encountered 2 general ways of implementing such measures, depending
how stringent you mean to get: either in a pass/fail condition, of as
part of a scoring chain.

Generally I'll set up hosts to only simply fail IP's from a DNSBL, but
other counter-measures would add or remove to a total score & be
rejected at a given threshold.

Not perfect, but deals with like 99% of the rubbish

- J

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

[Dagan]

Post by Robin Paulson
i setup a public mailman discussion list recently, and the people who
are subscribed via gmail are getting the following warning when they
more Report phishing"
the warning includes a link to this page, which gives a fairly
https://mail.google.com/support/bin/answer.py?hl=en-GB&ctx=mail&answer=185812
however, i'd like to get some further views on the matter.
is this a major enough problem to warrant doing anything about?
are users likely to unsubscribe when they see this message?
any suggestions on the best way to implement it, on a postfix/ubuntu
lucid system?
anything else which is relevant?

I had these sort of questions when I set-up my completely new mail host
and domain, and wondered if I would get spam-canned by the likes of
Google etc when I tried sending email.

A lot of documentation suggests a minimum of rDNS entry (extremely
important for a new mail server), and SPF record. I did the rDNS, and
noticed a few places that grey-list new domains, but never had any
issues with lack of SPF.

Having said that, SPF seems more prominent at this stage than DKIM, so
I would expect as situations change, SPF will become more mandatory as a
set-up item.

I haven't had any reports of anyone seeing errors about my emails being
spam, but that's not to say some random block list won't get a report at
some stage.

In this particular instance however, the error talks about relating to
email that appear to a user as originating from GMail, when they are
missing authentication records.

So I would start by confirming your mailman set-up matches the sending
domain (not the sending user), so that GMail doesn't suddently
black-list your mailman set-up.

Dagan


_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug