Feb 09

How Markus Made the World Better Today

Markus Stange did a pretty awesome thing for those of us who work with the Mozilla trees and tinderboxen:

First of all, bookmark that action.

As for how it works, a couple minutes of playing around should explain it.  Basically it’s a running changelog with associated builds and status, plus an at-a-glance view of the tree in the bottom right, and details of a selected run in the bottom left.

If it were covered in shaved black truffle and a velvety 25 year old balsamic, it would be only marginally more delicious.

If you see mstange on IRC, give him your love.  If you have suggestions, give him your patches.

[Update: Why yes, it does require querySelectorAll support.  I trust you have a browser that does that, right?]

Jan 09

SSL Information Wants to be Free

Recent events have really thrown light onto something I’ve been feeling for a while now: we need better public information about the state of the secure internet.  We need to be able to answer questions like:

  • What proportion of CA-signed certs are using MD5 signatures?
  • What key lengths are being used, with which algorithms?
  • Who is issuing which kinds of certificates?

So I decided to go get some of that information, so that I could give it to all of you wonderful people.

Continue reading →

Dec 08

Firefox Malware?

A crappy thing happened last week – someone wrote some malware that infects Firefox. We obviously don’t like that very much at all, but I wanted to at least make it clear what is and isn’t happening, since there’s some confusion out there.

What is going on?

Basically for as long as there has been software, there have been nasty people out there who get you to download and install software which turns out to have hidden cargo.  Security folks use names like “virus,” “trojan,” “worm,” and “malware” to describe different types, but the point is that if a person can be tricked into running nasty programs, they can do nasty things.

In this case, rather than wiping your hard drive or turning all your icons upside down, this particular jerk has decided to mess with your Firefox. Once you run the program, it hooks into your Firefox and watches for you to visit certain sites, at which point it will steal your username and password.

How Can I Tell If I Have It?

You can open up your Firefox addons manager (Tools->Add-ons) and go to the “Plugins” section.  If you have a plugin called “Basic Example Plugin for Mozilla” you should disable it.

Original credit to TrustDefender Labs’ blog post on the subject

Does This Mean that Firefox is Insecure?

No, and here’s why:

  • This particular malware targets our program, but once you have malicious software running on your system, it can just as easily attack other programs, or harm your computer in other ways.
  • This isn’t contracted by just browsing around the web with Firefox 3. In fact, the Malware Protection features in Firefox 3 are designed specifically to prevent sites from being able to attack your computer.

The people getting infected here are either downloading enticing files that have the malware hiding inside (which is why Firefox 3 hands off all downloads to your computer’s virus scanner once downloaded) or, as some sites are reporting, people who have already been infected in the past having their computers forced to download this file as well.

Typical Firefox 3 users who avoid downloading software they don’t trust are unlikely to ever see this, and even the sites reporting it describe its incidence as “rare”.

What’s this I hear about GreaseMonkey?

There are some mentions of greasemonkey in a couple of the early reports based on some analysis of the code used by this malware, but I want to be clear that the (legitimate, and awesome) Greasemonkey Addon is not involved in this malware in any way. It is not involved in the installation or execution of the attack.

As always, the best defense is vigilance.  Use a browser with a solid security record and modern anti-malware defenses built in, and be very careful about downloading and running programs you find online.  If a bad guy is able to get you to run a program on your machine they will be able to do bad things, so we’ll keep trying to stop them and you keep trying to as well.

More details are also available on the official Mozilla security blog.

Nov 08

Performance Dashboard (v2)

Way back when, (almost exactly a year ago, actually) I built a dashboard for getting at-a-glance views of our performance metrics, to make it easier to spot regressions and assess the state of the tree.

And so, of course, days later we decommissioned those boxes and that whole way of reporting performance, and the dashboard fell into disrepair.

A couple days ago I rebuilt it.

It’s in its infancy right now.  It only pulls data for the 1.9.1/Firefox3.1 branch, and it only pulls a couple tests thus far, but those are easy to add.  It has no fun widgets or user-preference memory or any of that, but patches are accepted.

The code is in a public hg repo here in case you want to beat me to any particular feature.  To run your own copy, just clone the repo, run the scrapedata.py script to get some up to date stats, and then open index.html in a suitably awesome browser.

The graphs are built with google’s really excellent charting API.  It’s reasonably flexible, and great for quick stuff BUT I’m not looking to replace our existing graph server.  That thing has all kinds of charting goodness that I absolutely don’t aim to reinvent.

This is a quick, dumb dashboard; not an immersive data navigation environment.  It isn’t complicated, it’s just something I thought would be useful.  How would you make it better?

[UPDATE: It’s not just coincidence that Rob has been thinking about these issues too, but it is kind of funny to me that we posted within hours of each other.  Clearly it was an idea whose time had come. ]

Nov 08

New in Firefox 3.1: Linkified View Source

Look what Curtis just did:


Curtis Bartley is the newest member of the Firefox front end team and, to get his feet wet, he made the world a better place by fixing a very old bug. And its 7 duplicate bugs.

Specifically, he set it up so that resources which are referenced in source are now clickable links.  Want to know what that external javascript does?  Click the link, and it will be loaded in the source viewer.  Likewise CSS.  Maybe you clicked “View Source” only to discover you were looking at a frame set, and actually wanted the source for a frame – that works too.

And yes, back and forward keyboard shortcuts work. And yes, both relative and absolute links work. And yes, you can have this in a tab instead of a separate window, either by sticking view-source: on to the front of your URLs (see?), or by finding one of the addons that does it for you.

Way to go Curtis, keep ’em coming!

Nov 08

SSL Error Pages in Firefox 3.1

If you’re using Firefox 3.1 nightlies or the upcoming Firefox 3.1 beta 2, you might notice some changes in the way we handle SSL errors. I landed them last week, and since it’s a topic that readers of this blog have historically wanted to talk about, I thought I would highlight some of the changes here. Continue reading →

Oct 08

SSL Infoquickie (with Bonus Firefox Pro-Tip!)

There is less public information out there about SSL certificate usage than one might like to see. Netcraft has a for-pay report with some interesting figures, and occasionally makes some of that data public, and I’ve blogged about other sources in the past, but in general, it’s pretty sparse. I keep meaning to do something coordinated about that, I have some ideas, but they keep getting back-burnered.

So it came to pass that when someone idly remarked that it would be nice to know what percentage of certs on the top sites were valid, I pounced upon it as a way to quickly release some pent-up info-gathering angst.

It’s profoundly unscientific, but so was the question. Are the Alexa top 500 sites even a good reflection of the most popular SSL sites? Not really. I think it will bias the data towards higher counts of untrusted certs (since the admins aren’t expecting them to be used) and towards lower overall cert counts (since many of those sites won’t answer SSL hails, whereas presumably a list of the top 500 SSL sites all would). Is blindly connecting to their main page on port 443 the best way to harvest their certs? Probably not, lots of them use secure.frobber.tld constructions, so that will also bias the data lower. Let’s just agree that it’s a sort of fun number to have as an order-of-magnitude style signpost.

Of the 500 top sites on Alexa, October 15, 2008:

  • 217 responded to an SSL query on port 443
  • 199 of those replies used valid certs chaining to trusted roots
  • The other 18 were a mix of self-signed, bad chains (likely from trusted roots, though I didn’t investigate), and expired certs.

If you prefer pretty pictures:

SSL Certificate Stats

Any conclusions you want to draw from this data will be only as good as the aforementioned biases within it, but don’t say I never do anything for you in a feeble attempt to vent my own info-lust urges.

Bonus Firefox Pro-Tip: If you are on Firefox 3.1 Nightlies or the upcoming Firefox 3.1 Beta 2, you now have the ability to turn off link-visited colouring.  David Baron recently landed a fix for bug 147777 that adds a new about:config preference to control the behaviour, layout.css.visited_links_enabled.

“Great!” I hear you all saying, “We’ve been hoping for a way to turn off an occasionally useful feature!”

And who hasn’t, really? But the thing of it is that colouring links can give away information to tricky sites about where you’ve been. It’s up to you whether you think that privacy/functionality trade-off is worth making, and the bug is still open while more universal solutions are contemplated, but in the meantime, you have the choice.

Sep 08

Why Did Talos Eat My Face?

Shawn has a great post up about some detective work we’ve been doing this week.  He gives me a bunch of credit, but I am convinced that I did the easy parts – he’s the one making our SQLite happier, so send him the laurels.  Nevertheless, Dave asked if I could write down the basic process used to narrow down the regression, which I agreed to do, because I am a sucker.

Back Story

So Shawn has been trying to update our version of SQLite for some time now.  Updating it fixes half a dozen crasher bugs, and should net us some modest perf wins in some places too.  But three times he’s tried to land it, and three times he’s had to back it out because of a sizeable performance regression on linux.  Specifically, Ts was going from about 1550ms to about 2500ms.  I wouldn’t call that a 65% regression or anything, but I would be pretty comfortable calling it a 61.2% regression so, you know, holy jumping tree squirrels.

The thing is, there was no meaningful hit on Windows or Mac, and even on Linux, Shawn didn’t have much of an idea where to start.  I (as sheriff for the day) didn’t want that sitting in our tree, but understood Shawn not wanting to back it out without any notion of whether the regression was even real, let alone what was causing it.

Enter Standalone Talos

As is so often the case, Alice came to the rescue.  A while ago she put together the standalone talos package, and I keep being happy that she did.  Running talos on your local machine is a bit of a noisy way to gather performance data (depending on the test) but it’s a great way to make your Firefox do exactly the same thing as the real talos does, for the purposes of profiling.  Which is what we did.  And it worked.

How We Dissected The Problem in Six Easy Steps, Only Two of Which Involved Cussin’

  1. Follow the standalone talos set up instructions to get your own local talos setup and pointed to a build without the offending patch.
  2. Edit the sample.config file, drop down to the bottom of the file, and delete every test you don’t care about, since a full talos run is a lot of work.  Tp, in particular, is not a fast test.
  3. Run talos with your profiler of choice.  In our case, the venerable strace. Our command looked like this:
    strace -T -o without-patch-strace.log python run_tests.py sample.config
  4. Swear, because your log is way too short, because you weren’t tracing the child processes, and hence were presumably watching some python crap. Try again with:
    strace -Tf -o without-patch-strace-AGAIN.log python run_tests.py sample.config
  5. Swear, because now you have a gigantic log with everything interleaved stupidly. Read the damned man page this time and come up with the -ff argument to generate a separate trace file for each pid:
    strace -Tff -o without-patch-strace-OMG.log python run_tests.py sample.config
  6. Repeat step 5 after pointing Talos at a build with the patch (and presumably changing the output filename).

This will leave you with a large number of trace files, but it’s pretty easy to recognize the ones you care about.  In Shawn’s case, he just grep‘d out the ones that referenced places.sqlite, and behold, he had before and after traces of precisely the same startup sequence that talos was seeing.

From there it was a short trip to see a massive increase in fsync calls, something we did not expect, but which certainly explains the lag, given the problems we’ve had historically with fsync on Linux.

Total execution time, before and after

Total execution time, before and after

Needless to say, Shawn is on it.