[Olpc-sysadmin] Fwd: [support-gang] 4PM SUN AUG 24: Support Mtg w/ Guest Speaker CHUCK KANE, OLPC President

[Henry Edward Hardy]

thanks!

--HH.

On Mon, Aug 25, 2008 at 10:39 AM, Michail Bletsas <mbletsas at laptop.org>wrote:

>
> Henry,
>
> I have configured rt as an MX record pointing to spam which then delivers
> mail for rt.laptop.org to solar.
> Spam volumes should go down.
>
> M.
>
>
>
>
>  *"Henry Edward Hardy" <hhardy01 at gmail.com>*
>
> 08/24/2008 04:18 PM
>  Please respond to
> henry at laptop.org
>
>   To
> "Michail Bletsas" <mbletsas at laptop.org>  cc
>   Subject
> Fwd: [support-gang] 4PM SUN AUG 24: Support Mtg w/ Guest Speaker CHUCK
> KANE, OLPC President
>
>
>
>
> Michail, can we please set up the Barracuda such that mail to *
> rt.laptop.org* <http://rt.laptop.org/> gets filtered as well as that to *
> laptop.org* <http://laptop.org/>?
>
> thanks,
>
> --HH.
>
> ---------- Forwarded message ----------
> From: *Henry Edward Hardy* <*hhardy01 at gmail.com* <hhardy01 at gmail.com>>
> Date: Sun, Aug 24, 2008 at 4:11 PM
> Subject: Re: [support-gang] 4PM SUN AUG 24: Support Mtg w/ Guest Speaker
> CHUCK KANE, OLPC President
> To: Chris Leonard <*cjlhomeaddress at gmail.com* <cjlhomeaddress at gmail.com>>
> Cc: Kimberley Quirk <*kim at laptop.org* <kim at laptop.org>>, Adam Holt <*
> holt at laptop.org* <holt at laptop.org>>, Chris Ball <*cjb at laptop.org*<cjb at laptop.org>
> >
>
>
> CJL, I spoke to Michail about having the rt mail go through the barracuda
> box, I will follow up with him.
>
> Michail, can we please set up the Barracuda such that mail to *
> rt.laptop.org* <http://rt.laptop.org/> gets filtered as well as that to *
> laptop.org* <http://laptop.org/>?
>
> thanks!
>
> --HH.
>
>
> On Fri, Aug 22, 2008 at 12:27 PM, Chris Leonard <*cjlhomeaddress at gmail.com
> * <cjlhomeaddress at gmail.com>> wrote:
> Henry,
>
>
> Here is the analysis I did (and shared with Adam), oddly enough not too
> long before I saw tha the virus-attachement was a reality and not jsut a
> liklihood.  I really feel pretty strongly tha there are strong business
> cases to be made for doing something about the spam issue both on cost and
> ethical grounds.  I don't want to come across as a crusader, but I think the
> earlier dismissals have been based on faulty anecdotal impressions and not
> real business metric analysis, so I wrote up this stub, it needs polishing,
> but the numbers are pretty solid.  4 of 10 as spam is just too high.  I see
> *maybe* 1 in 10000, if that.
>
> cjl
>
> ---------- Forwarded message ----------
> From: *Chris Leonard* <*cjlhomeaddress at gmail.com*<cjlhomeaddress at gmail.com>
> >
> Date: Thu, Aug 21, 2008 at 8:19 PM
> Subject: Re: [support-gang] 4PM SUN AUG 24: Support Mtg w/ Guest Speaker
> CHUCK KANE, OLPC President
> To: Adam Holt <*holt at laptop.org* <holt at laptop.org>>
>
>
> On Thu, Aug 21, 2008 at 7:22 PM, Adam Holt <*holt at laptop.org*<holt at laptop.org>>
> wrote:
> AGENDA:
>
>
> 1) Chuck Kane will answer *as much as he can* about this autumn's new
> and different G1G1 program(s).  Important decisions are still taking
> shape as we speak -- plz ask your questions in advance here:
>  *http://wiki.laptop.org/go/Support_meetings*<http://wiki.laptop.org/go/Support_meetings>
>  *
> http://laptop.org/teamwiki/index.php/Team:Support_meetings/20080824_2008_Aug_24
> *<http://laptop.org/teamwiki/index.php/Team:Support_meetings/20080824_2008_Aug_24>
>
>
> Adam,
>
> Would it be fair to ask "What is OLPC going to do about the spam problem (4
> out of every 10 RT tix) that it hasn't yet acknowledged?"  Maybe after
> posting the analysis below to SG or on TeamWiki?
>
> I'd be happy to provide the exact queries, but the numbers check, analysis
> is one of my lines of business.
>
> ==Introduction==
> Until recently, there have been systematic biases resulting in the severe
> underestimation of the scope of the spam problem.  "Rejected" was only
> weeks/months ago defined as the proper resolution for spam.  Many messages
> were previously "deleted" and in some cases simply "resolved" or even just
> left sitting in queues like content.
>
> I performed a reasonably exhaustive hunt for improperly classified spam and
> tried to clean the classification metadata, I also cleared out the spam in
> the content queue as well as a large amount of foreign language spam (first
> checking with Google Translations) that had previously been left untouched.
>  In the process, I've reviewed/handled about one third of all "rejected"
> tickets.
>
> I think I have a pretty clear picture of the spam problem and why it was
> previously thought by some to be "no big deal" on hte basis of anecdotal
> evidence.  A genuine and thorough analysis of the available business metrics
> shows that it is, in fact, a substantial and growing problem that deserves
> to be seriously addressed by OLPC management in a meaningful fashion. Here
> is a little brief analysis of the scope of the spam problem on RT (much
> easier to do and more meaningful, now that the data quality has been
> improved.
>
> ==Analysis==
> For the purposes of this analysis, "rejected" = spam.
> 2892 rejected
> 7215 resolved
> 181 stalled
> 1098 open
> 1563 new
> 12949 Total
>
> 18234 was last ticket # issued at time of queries, therefore
> 5285 deleted
>
> ===Best case/worst case estimates:===
>
> 22.33% best case scenario spam/ham ratio   2892 / 12949
> (ignores 5285 deleted messages)
>
> 44.85% worst case scenario spam/ham ratio   8177 / 18234
> (assumes all deleted messages are spam)
>
> In other words, no matter how you slice it, just about every third message
> in the RT queues is spam.
>
> ===Time series analysis of worsening problem:===
> Let's take a look at whether this problem is getting better or worse over
> time.  Tabulating the number of rejected messages and grouping them into
> "how many months ago" the ticket was created gives the time series below.
>  Note: zero months ago means in the last 4 weeks.
>
> Months ago Spam count
> 0  695
> 1  628
> 2  449
> 3  348
> 4  416
> 5  155
> 6  100
> 7   11
> 8   18
> 9   14
> 10    4
> 11   46
> 12    8
> Grand Total    2892
>
> Although the early data is quite unreliable (because previously spam was
> not routinely being handled by using the "rejected" flag) and many of the
> 5,285 deleted message are almost certainly spam.  Even the "best case" data
> presents an clear picture of a significant and rapidly worsening spam
> problem.  For the last three or four months (the ones with better data), the
> volume of spam passed through to RT has been increasing at a rate from 10%
> to 30% month-over-month and the ration of spam to ham is 2:3 (see below).
>
> ===What is the current rate of spam===
>
> Let's repeat the simple analysis that I started with, but limit it to
> recent tickets (since about July 1st)
>
> 3086 tickets > #15400  (July 1st, 2008)
>
> 1258 rejected
> 487 new
> 386 open
> 0 stalled
> 921 resolved
> 34 deleted (by elimination)
>
> 1258/3086 = 40.8 %
>
> Greater than 40% of all messages are rejected (as spam) since July 1st,
> this also suggests that the "worst case scenario" described above likely to
> be far closer to the truth.  This fact was previously masked by the
> extensive use of "deleted" as a spam resoultion before recent attempts to be
> more systematic about employing the "rejected" resoultion.
>
> ===Summary of spam problem===
>
> What I conclude from all of this is that:
> a) The spam problem is bad, very bad.  No less than one-third of all RT
> messages are spam (something like ~22 spam messages/day and ~44 ham).  In
> recent months, the ratio is closer to two out of five incoming messssages
> being spam.  This is several fold higher in both volume and percentage than
> the offhand guesses that have been used to dismiss spam as an non-issue.
>  This is clearly not an acceptable spam filter leakage rate by any
> reasonable measure.
>
> b) The spam problem is getting progressively worse, and at rapid pace.  It
> can only expected to get much worse when G1G1 2008 kicks off.
>
> ==Questions to be raised==
>
> ===What is the cost of doing nothing about the spam problem (financial)?===
> Assumptions:
>
> I will use an estimate of about 25 spam messages per day going into the RT
> system. This doesn't include all other *laptop.org* <http://laptop.org/>addresses, although they would benefit from better spam filtering too.
>
> I will use an estimate of about one minute per message to assess and
> delete, it is a little faster for english language spam, and slower for the
> increasing amount of non-lang-en spam.
>
> I will use an estimate of ($21.00 / hour) as the hourly value of volunteer
> time.  This number is entirely within the acceptable costing estimates of
> volunteer time recommended by the IRS (*), and it rounds nicely to 35 cents
> per minute.
>
> The annual opportunity cost to OLPC of volunteers handling the current spam
> load is approximately $3,150/year in lost productivity.
> 25 msgs/day X 1 minute X 0.35 dollars = 8.75 dollars / day
> $8.75 X 360 days in year = $3,150/year
>
> *  The estimated dollar value of volunteer time nationwide is $19.51 per
> hour for 2007, in Massachusetts it is $24.29 per hour. The value of
> volunteer time is based on the average hourly earnings of all production and
> nonsupervisory workers on private nonfarm payrolls (as determined by the
> Bureau of Labor Statistics). Independent Sector takes this figure and
> increases it by 12 percent to estimate for fringe benefits.
> *http://www.independentsector.org/programs/research/volunteer_time.html*<http://www.independentsector.org/programs/research/volunteer_time.html>
>
> This number is the widely used standard for cost valuation of volunteer
> time and specifically is accepted for IRS and FASB accounting purposes.
>
> ===What is the cost of doing nothing about the spam problem
> (non-financial)?===
>
> There are substantial security risks represented by malware (embedded in
> attachments of all sorts) as well as link-based social-engineering infection
> routes.  More detail can be provided if needed, although it has been well
> documented in the trades.  This is not vendor-driven FUD, in the computing
> environment for which I am responsible, I have been fortunate to stop a
> spear-phishing attack based on a sophisticated attachment contained weblink
> combination that launched a piece of custom malware that had yet to be
> profiled by TrendMicro.  Suffice it to say, I no longer rely on TrendMicro
> as my mail-gateway filtering agent.  OLPC bears some ethical responsibility
> to the many volunteers handling the RT queue (and attachements) without
> sophisticated isolation environments, and also incurs risks on their own
> infrastructure that shoudl be more adequately addressed.
>
> ==Recommendations==
>
> Look into a better spam filtering solution, or spend a lot of time tweaking
> the existing solution to function at a minimally acceptable level.
>
> I have been quite happy with IronPort's c10 device (now owned by Cisco)
> combined with their management services for the past several years.  It is
> very ine3xpensive comapred to teh lost productivity it prevents (user and
> admin) and the peace-of-mind it provides. I would be happy to share more
> informatiomn about my positive experience with the high success rate and
> extraordinarily low (read none) false positive rate combined with never even
> having to look at or adjsut the device beyond the first week of the install.
>  You can get a 30 day free trial from them, it is well worth considering
> among other options, because existing Barraqcuda configuration is just not
> doing the job.  I am not a Cisco partner or shareholder, just a very
> satisfied customer.
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.laptop.org/pipermail/olpc-sysadmin/attachments/20080825/91eac33b/attachment-0001.htm