[IP] 4 more on we are all blacklisted

[Dave Farber <dave@farber.net>]

> Date: Fri, 05 Sep 2003 10:07:52 -0400
> From: "David P. Reed" <dpreed@reed.com>
>
>
> Ed gerck makes a great point.  They use us against ourselves.   Here's a 
> bigger view that he inspired me to jot down.
>
> Spam and virus/worm attacks are evolutionary systems.   We are the 
> evolutionary landscape, and we co-evolve with them.
>
> From a systems point-of-view, their goal transcends ours.   We focus on 
> "securing email" against certain kinds of attacks.  But our focus on 
> specific means is their advantage.  The spammer's goal is to appear in 
> front of eyeballs on a mass scale at low cost - not specifically to 
> exploit exploitable holes like relays or anonymous sourcing.   So we end 
> up patching all the holes while the boat is sinking from new kinds of 
> holes.  Better to install some flotation foam, and quick, to stymie their 
> real goal of permeating our ship with water.  The virus/worm guy is 
> focused on "gentile terrorism" - all they want to do is to annoy/destroy 
> at minimal risk to themselves.  Again they enlist our help in destroying 
> ourselves by tricking us into blocking legitimate communications.
>
> We help both sets in their evolutionary race by fighting them, especially 
> by fighting them inefficiently.   Just like weak vaccines and weak 
> antibiotics enhance the evolutionary machine for "germs", weak responses 
> just train a new generation.
>
> It's really worth thinking about whether we are transforming low-grade 
> irritants into highly virulent and damaging strains by our ineffective 
> responses, which just serve as training sets for their evolutionary algorithms.
>
> Our strategy needs to be much more focused on the evolutionary system, not 
> on low-level patches after-the-fact.
From: Rich Kulawiec <rsk@gsp.org>

On Fri, Sep 05, 2003 at 01:45:11AM -0700, Ed Gerck wrote:
> Thus, adding more
> and more ad hoc blacklists (IP and DNS based) is simply making more
> and more likely that an important message will NOT be received --
> irrespective of your efforts.

1. DNSBLs are not "ad hoc".  Most of them are quite thorough.  (Those which
are not aren't widely used, for precisely that reason.)

2. Regardless, if you feel this way about them, then YOU should not
be using them.  However, those of us who understand them and find
them useful and accurate will no doubt continue to do so.

> 1. "Content-sensitive filters ignore the putative sender (since both 
spammers and
> virus/worms frequently forge it) and focus on the content." This is just 
half the
> story. The fact is that content-sensitive filters such as mailwasher do 
use the
> content to blacklist a sender's address.

Mailwasher is an amateurish hack which includes an auto-abuse feature of its
own -- e.g. the bounce-to-sender feature, which enables anyone naive enough
to think that senders aren't forged to abuse another (almost certainly
innocent) person.

This is why the consensus among knowledgeable anti-spammers is that nobody
should be using Mailwasher.  (A few minute's research with Google will
easily bear this out.)  If you want to see a professional-grade
content-sensitive filter, then go look at SpamAssassin.

> And just talk to most users -- who simply
> set their mail client filters to blacklist entire domains (*@aol.com) 
and even
> entire TLDs (*.cn, *.ru, etc.), irrespective of content.

1. I'm sorry that some users have decided to misconfigure their mail clients
in broken ways because they don't understand the spam problem and proper
strategies/tactics for dealing with it.   But this is not my problem.

2. It is a long-established principle of spam fighting that the best place
to block spam is that place which is nearest the source.  Ideally, this
means convincing the spammer-hosting ISP to immediately and permanently
remove their spamming parasites without prior notice.  The next-best place
is that the receiving MTA.  Blocking in the end-user's client is largely
ineffective and is very much a close-the-barn-door-after-the-horse solution.

> 2. "DNSBLs [RHSBLs] ... completely IGNORE the putative sender and focus
> solely on the originating IP address." The simple fact that we do 
receive many
> 100's of spam/viruses/worms daily, and #3 below, should speak for the
> DNSBLs' lack of usefulness. It is not working.

This is an amazing leap of total illogic.  You have failed to mention what
DNSBLs *your* mail service provider is using, and in precisely what way
they are used.  You have also failed to recognize that spams/viruses/worms
may be coming FROM YOUR OWN ISP, in which case they can hardly be expected
to block mail traffic from themselves.  And you have blithely generalized
from YOUR experience to the experience of others -- I guarantee you that
they are not the same.

I suggest that you need a basic grounding in the theory and practice of
DNSBLs before proceeding further.  It's obvious to me that you don't
understand how they work, and are therefore completely unprepared to
evaluate their advantages/disadvantages.

> 3. "...block SMTP connections from dynamic dial-up IP ranges and from
> residential Cable/DSL connections".  In other words, let's break the 
peer-to-
> peer Internet, let's make some IPs more routable than others, let's create
> "second-class" Internet users and make them 100% dependent on ISPs
> for SMTP.

1. You are apparently unaware that the contractual agreement between nearly all
ISPs who provide dial-up/cable/DSL and their customers REQUIRES that those
customers route their outbound SMTP traffic through the ISP's own mail servers,
and thus there should never be any outbound port 25 from those network blocks.

2. You are also apparently unaware that some of those ISPs block outbound
port 25 at the router level, in order to enforce #1.  This has been a common
practices for years.

3. You are also apparently unaware that at the moment, the dial-up/cable/DSL
networks of a large number of major providers (e.g. Comcast, Verizon, 
Roadrunner,
Charter, etc.) are heavily infested with end-user systems that have been 
hijacked
via SoBig variants and are being used as a massive distributed 
spamplifier.  The
failure of these users to adequately secure their own systems, combined with
the failure of their irresponsible ISPs to take action when duly notified of
same, leaves the rest of the Internet with two choices:

        a) Do nothing about it, and be subjected to a torrent of spam.
        b) Block them as they're detected or pre-emptively en masse.

Most sensible people have done (b).  People who chose (a) are no doubt
drowning in spam, but that's certainly their choice.

> "Black hole" lists look a lot like a "vigilante" solution [....]

1. I suggest that you are in no position to analyze them, or their impact,
because you obviously do not understand how they work.  You need to read
-- for a minimal introduction -- the FAQs I referenced in my previous
message.   You should also read the FAQs those reference, and so on,
until you have at least a rudimentary understanding of the current state
of spam/anti-spam strategies.

2. There is nothing in the least bit" "vigilante" about deciding how
one's own systems and networks are to be used: it is merely the exercise
of basic property rights.  This is biased, inflammatory language which
indicates, once again, a lack of even rudimentary knowledge on this issue.

---Rsk
From: Nancy McGough <nm-reverse-spam-filter@ii.deflexion.com>



> Date: Thu, 04 Sep 2003 14:29:35 -0700
> From: Ed Gerck <egerck@nma.com>
> Subject: we are all blacklisted
>
> The lesson here is that current systems for automatically
> handling large volumes of spam and virus, as well as
> "black-hole" lists, are backfiring.

I think the main lesson that I've learned is that most people,
including unfortunately people who produce filtering software, do
not understand that the From:, To:, and Cc: headers do not
matter. What matters is the SMTP envelope. A couple things we can
do right now to help the situation are:

* Never reject (bounce back to sender) a message after it leaves
  the SMTP world.

* Insist that mail-hosting providers inject headers that state
  the original envelope sender and original envelope recipient.
  This will help users to filter their messages appropriately.

* Teach users that it is too late to "bounce back" a message by
  the time they deal with it with their filters. Bouncing
  (rejecting) can only be done correctly during the SMTP dialog.

From: Rich Kulawiec <rsk@gsp.org>

On Thu, Sep 04, 2003 at 08:59:57PM -0400, Dave Farber wrote:
> >Anyone who owns a well-known email address is probably being
> >blacklisted right now by automatic spam- and virus-handling agents
> >worldwide, frantically trying to deal with SoBig and other culprits that
> >send emails from faked addresses.

This is extremely unlikely.  Content-sensitive filters ignore the putative
sender (since both spammers and virus/worms frequently forge it) and focus
on the content.  E.g. AV programs look for signatures characteristic of
known malware; anti-spam programs such as SpamAssassin look for evidence
of spam by correlating against known spam samples.

Neither "blacklists" any user addresses because there's nothing to gain
by doing so.  (Though a few user addresses may be useful in accruing
evidence: e.g. mail from big@boss.com is probably an early SoBig variant,
mail from CacheFlowServer@<anything> is probably a hijacked CacheFlow
server being used to send spam, etc.  But there are only a handful of
isolated examples such as this, and they're insignificant when compared
to the millions of forged addressess -- some of which exist, some of
which don't.)

Non-content-sensitive filters ignore the originating address because
it's usually of no value whatsoever -- see below.

> >Anyone whose address "looks good" is a preferential target. The lesson
> >here is that current systems for automatically handling large volumes of
> >spam and virus, as well as "black-hole" lists, are backfiring.

I see absolutely no evidence that any such systems are "backfiring".
Among other things, this isn't how DNSBLs [1] (aka "black-hole lists")
work: they completely IGNORE the putative sender and focus solely on
the originating IP address.  This in turn is one of the reasons that
DNSBLs are highly effective -- they rely on information that's much
harder to forge [2] than the sender.

---Rsk

[1] A DNSBL is a DNS-based Block List.  All modern mail transport
agents (MTAs) such as sendmail, postfix, etc. can use these.  The
way they work is that when an incoming connection to the MTA shows up,
the MTA queries one or more DNSBLs, using the originating IP address
of the connection.  Various codes are returned (depending on the DNSBL)
which indicate "okay", "spam source", "open relay", "open proxy", etc.
The MTA then decides what to do with this information: block the message,
drop the connection, tag the message, whatever.  DNSBLs acquire their
information by various means, including automated testing of connecting
MTAs, manual testing, spamtraps, and other means.  There are now several
hundred of them in operation, nearly all run by volunteers.  See
http://www.spamfaq.net/ or http://www.openrbl.org/ for more info.

[2] But not impossible: some spammers have recently started forging
documents claiming ownership of IP blocks abandoned during the dot-com
flameout.  They then convince ISPs to route traffic for them, and use
these "zombie networks" to spam (of course).  The countermeasure for
this is to simply list the entire zombie network in the appropriate
DNSBLs and deny all traffic from it.  See http://zombie.dnsbl.sorbs.net/
for one such DNSBL.

-------------------------------------
To manage your subscription, go to
 http://v2.listbox.com/member/?listname=ip

Archives at: http://www.interesting-people.org/archives/interesting-people/