02
May 13

Stargazing

1 year ago, I started a spreadsheet. For 9 months we’d been rewriting Firefox for Android from the ground up, and on May 16, 2012 we released the first beta version to the Android market. I started a spreadsheet to track our star rating. I was nervous.

9 months of rewrite. In web time— in mobile web time— that’s years. Rewrites are almost always the wrong call. You get to throw away bad code, but you throw away good code, too. Hard won, battle-hardened fixes are expensive to throw away, and the code is never cleanly labelled “baby” and “bathwater.” The decision to rewrite Firefox was one of the loudest weeks I’ve had. Even once we’d decided, my hand hovered over the send key for a long time.

3.5 stars. We didn’t pull the old version from the market during the rewrite. We wanted to keep those users safe with regular security updates; that meant keeping the product online. It meant taking black eyes every day from users who tried us, who loved us for our desktop product, and who were disappointed. And because the Android rating system rounds, our already painful 3.7-star rating rounded down to 3.5. We were dead last.

7:1. If you want to maintain a 4.5 star rating, it takes seven 5-star reviews to counteract each 1-star review. Star ratings are deeply flawed for all kinds of reasons: selection bias, survivorship bias, false dilemmas, unidimensionality, reporting bias, et cetera. I know this. And still I watched them. When you try to reach people through an app store, your star rating is the first assessment of your product they’ll see. And when you try to make something great, reviews are real pieces of feedback from real human beings and they are painful and they are frustrating and they are golden.

1181 reviews in the first week. 588 of them were 5-stars. 105 of them were 1-star. I argued with my screen over 1-star comments about bugs we had already fixed. I swooned at 5-star reviews that said they were reversing earlier 1-star reviews.

6 weeks after beta, we pushed to release. By this point beta had climbed to 3.8, which rounds to 4 stars. Fists were pumped. Release sat at 3.6.

Time passed.

200,000 reviews and 6 releases later, this week, today, Firefox and Beta both show 4.5 (rounded) stars in the market and the team is still going strong. I’m immensely proud of the work they’ve done. It’s made me reflective and maybe a bit wordy. I want to have some profound and pithy lesson to separate then and now. Something that I can package up and hand to you. We certainly learned lessons, profound lessons, but in repeating them they sound trite:

Listen. Care about what people think. Be hungry for feedback. Don’t work forward from the tech you have to the product you can build; work backward from the product your users deserve to the tech you’ll need to get there. Ask for help, and accept it even when it hurts to admit that you need it. Don’t throw things away lightly, but be able to throw them away in those rare cases where it’s necessary. Surround yourself with the most excellent people you can find. That one helps a lot.

The haters will say I over-focus on star ratings. There are certainly lots of other good things to measure, and lots of bad things to say about stars. But the stars abide. They are inescapable. They sit front and center in your primary distribution channel. Those five little stars. And I believe they do say something about what we’ve built.

Building good things is hard and my hat is off to anyone who earnestly tries. If the good thing you build reaches people through an app store, I want you to know that I know how you feel. And I’m rooting for you.

[I posted a version of this post on Medium, because it's fun to try new things.]


08
Mar 13

Horseplay

I want to make a point about changing the world, but first we need to talk about the horse head.

For Christmas last year I bought my brother a horse head and, while I was at it, picked one up for the office. It felt like the kind of thing that Mozilla Toronto would enjoy.


Good times.

Between meetings we needed a place to store it; the sad, flat way it sort of collapsed when left on a desk was unsatisfying. So Madhava and I went out in search of a head on which to mount it. Steps from our office is a mannequin supply store, which helped.

I put the horse head on its styrofoam mount and, for giggles, set it up in the window. Facing outwards. Staring at the office in the next building.

For 2 weeks it sat there and I was pretty happy with it. And then something happened. Something wonderful and magical happened. I wasn’t there for it, but lmandel and overholt were.

The other office responded.

They taped a note to the glass.

“What’s with the horse?”

We responded.

“Who you callin’ a horse?”

They responded.

“Why you, of course!”

And so it went. Back and forth.

Stick with me, I’m getting to the world changing.

So last week we bought them their own horse head. And yesterday mconley delivered it to their office. They invited him in. They brought the box over to the window where we could see. They gathered around their window and we gathered around ours. And when they opened it and realised what it was they actually jumped up and down, and applauded, and mouthed “thank you” through the window at us.

In Bowling Alone, Robert Putnam’s utterly definitive work on society and community, he writes that the best predictor of a school’s success is the activity of its PTA, and that the activity level of a PTA can be changed dramatically by one or two committed parents. Derek Sivers’ TED talk is all about the powerful change that happens when one person being silly becomes two people being silly, and cognitive scientists have been talking about the power of allies for 50 years.

The horse head(s) didn’t change the world; I’m not that pompous. But changing the world is hard work and it’s worth getting some practice in. So go start something. And, this is crucial, if you see someone else starting something: play along. Maybe the thing you’re playing with draws a great big crowd and changes the world. Maybe it never amounts to more than an office of strangers 50ft away silently jumping up and down and saying thanks.

It’s not such a terrible downside.

 

UPDATE: The world keeps getting more awesome. Denise from the other office just sent me email saying that the 4th floor office in our building saw their “best day ever” sign, and posted a reply. They don’t even know about the horses. They just wanted to share in the happy.

and tomorrow will be even better (photo credit: denise)


02
Aug 12

What is it like?

This question popped up on Quora recently and I offered a response (though, to be honest, I’m more curious about other people’s responses). Dave Dash, formerly of Mozilla web dev answered as well, and Jared Wein answered in blog form.

I’ve included my answer below even though, re-reading it a few days later, there’s so much more I want to add (I can’t believe I didn’t mention working with our worldwide community of employees and volunteers, or the impact of video conferencing, or the miracle of california tacos, or qdb, or mozillamemes…)

What’s your experience?
Continue reading →


11
Jul 12

At Our Most Excellent

Jono recently wrote a blog post about Firefox updates, and Atul wrote a follow up. They are two of the brightest usability thinkers I know. When they talk about users, I listen. I listen, even though some of the things they say sound confused to me, and some are plain wrong. And I listen because if people as bright and in tune with Mozilla as them think these things, I bet others do, too.

When I read (and re-read) the posts, I see 3 main points:

  1. The constant interruption of updates is toxic to the usability of any piece of software, especially one as important as your web browser.
  2. Our reasons for frequent updates were arbitrary, and based on the wrong priorities.
  3. We take our users for granted.

To be honest, if it weren’t for the third point, I wouldn’t be writing this. Anytime you do something that impacts lots of people, especially lots of impassioned and vocal people, you’re gonna get criticism. Listening to that is essential, but fanning the flames can consume all your energy and even still some people won’t be convinced. The third point, though, made by two people who know and love Mozilla even if they haven’t been close to the release process, isn’t something I want to leave sitting. I understand how it can fit a narrative, but it’s just not true.

Since I’m writing anyhow, though, let’s take them in order.

Interruptions Suck

Yes. They do. One criticism that I think we should openly accept is that the move to regular releases was irritating. The first releases on the new schedule had noisy prompts (both ours and the operating systems’). They broke extensions. Motives aside, our early execution was lacking and we heard about it. Plenty.

Today our updates are quiet. Addons have been compatible by default since Firefox 10 back in January. But that was a mountain of work that would have been much nicer to have in hand up front. As Jono says, hindsight is 20/20, but we should have done better with foresight there.

Motivations

It was hard for me to read the misapprehension of motives in these posts. Hard because I think Mozilla’s earned more credit than that, and hard because it means I haven’t done a good job articulating them.

Let me be clear here because I’m one of the guys who actually sits in these conversations: when we get together to talk about a change like this, concepts like “gotta chase the other guys” are nowhere in the conversation. When we get together and draw on whiteboards, and pound on the table, and push each other to be better, it is for one unifying purpose: to do right by our users and the web.

I wrote about this a while back, but it bears repeating. We can’t afford to wait a year between releases ever again; we can’t afford to wait 6 months. Think how much the web changes in a year, how different your experience is. Firefox 4 was 14 months in the making. A Firefox that updates once every 14 months is not moving at the speed of the web; we can’t go back there. Every Firefox release contains security, compatibility, technology and usability improvements; they should not sit on the shelf.

There’s nothing inviolate about a 6 week cycle, but it’s not arbitrary either. It is motivated directly from our earnest belief that it is the best way for us to serve our users, and the web at large.

And so the hardest thing for me to read was the suggestion that…

We Take Our Users For Granted

Nonsense. I don’t know how else to say it. In a very literal way, it just doesn’t make sense for a non-profit organization devoted to user choice and empowerment on the web to take users for granted. The impact of these changes on our users was a topic of daily conversation (and indeed, clearly, remains one).

To watch a Mozilla conversation unfold, in newsgroups or in blogs, in bugzilla or in a pub, is an inspiring thing because of how passionately everyone, on every side of an issue, is speaking in terms of the people of the web and how we can do right by them. We are at our most excellent then.

There’s beauty in the fact that this is another of those conversations. It is not lost on me, nor on Jono and Atul, I’d wager. They are Mozillians. And I believe they care deeply about Firefox users. I hope they realize how much the rest of us do, too.


08
May 12

A Compendium of Awesome

Team Firefox 2012Two weeks ago, the Firefox team got together for a work week in Toronto. It was amazing. Walking through a room with that many excellent people doing excellent things was inspiringhumblingunbelievable and the hits kept on rolling.

The combined mobile and graphics teams cut the beta blocker list for Fennec Native in half. The desktop team banged together a working prototype of sign in to the browser. The firefox tech leads worked with product and project management to nail down the kilimanjaro bug list for desktop. Madhava gave a great talk about the future of Firefox UX. I would have scored it as a strong success based on those outcomes alone.

And then this happened:

Continue reading →


28
Mar 12

42 Days

In a recent thread on dev.apps.firefox, someone suggested we shift our 6 week release schedule to avoid Microsoft Patch Tuesdays, or other unfortunate timing coincidences. I’m not announcing any changes to our release schedule, but I did make a point there that I want to repeat a little more broadly (and with emphasis added):

Let’s go back to first principles for a minute. Releasing every 6 weeks is a cadence we set for ourselves to satisfy several constraints. Constraints like:

  • Delivering security and stability fixes on a regular basis.
  • Getting new features out to our users promptly, and being able to iterate on the feedback we receive.
  • Containing the amount of code change and change-interaction that happens per release.
  • Giving ourselves time to react to problems discovered before release (on Nightly, Aurora, or Beta)

Releasing daily wouldn’t work very well; it runs afoul of the last constraint. Releasing yearly would hurt us on the first 3. But the constraints are just about as well satisfied by 40 days or 44 days as they are by 42 days.

We derive great benefit from our current schedule. It satisfies these constraints much better than the old, monolithic release model did. But that is not to say that we should treat 42-day cycles as inviolate. We will adjust. We will add or drop days, or add or drop weeks when we need to. We’ll be respectful of the fact that other people build plans around our plans, and try not to alter schedules without notice, but at the end of the day we’ll do right by our users first and foremost.

Right now we’re not looking at moving the release to another day of the week (Tuesdays do have some nice properties), or adding a skip week somewhere to take us off-cycle for the next patch Tuesday, but those discussions are absolutely on the table when they make sense.

No religion here. 6 weeks is a nice spot in the constraint space, not a law. The first time we miss it, people will talk about us “slipping,” but the tail doesn’t wag the dog here. Firefox still ships when it’s ready.


25
Jan 12

Bringing Android Native Firefox to Beta

I like trains. Last year, we put Firefox on a train-based release model: every six weeks, another train leaves the station. When a feature catches the train it moves through iterative testing on our Nightly, Aurora and Beta channels and, if that testing confirms its stability and general excellence, it goes out to hundreds of millions of Firefox users. If testing reveals an issue, we pull the feature out for another round of review, and let it catch a later train. The trains have run on time ever since, and the results have been incredible. Firefox improvements reach our users regularly, faster than ever before.

However, when we decided to rebuild Firefox for Android using Native UI, we recognized that the first release couldn’t ride the trains. The iterative release model that serves us so well with Firefox works best when most changes are incremental and independent. Building a new high-performance front end for Firefox on Android, by contrast, involves many interconnected pieces being rebuilt in tandem.

Right now, the engineering team is focused on building an amazing browser for Android phones, and we’ll have a beta to show you in the coming weeks. It might coincide with one of our regular 6 week trains, but it’s quite possible it won’t. If it doesn’t, don’t worry. It’s cool. Firefox for Android will get back on the trains once the native UI rebuild is finished, but for a change this major we have extra work we want to do before we send it out the door. We’ll only ship it once we’re happy with its quality and performance. If you can’t wait that long, check us out on tablets or try our early release Aurora builds. I think you’ll be pleased.

[This post originally appeared on the Future of Firefox blog]


26
Aug 11

Rapidity

[This is a re-post of a post that originally appeared on the Future of Firefox blog]

Last week we released a new version of Firefox. We shipped on time, 6 weeks after the last update, making it our first true rapid release milestone. There was cake. Now that we know that we’re capable of this velocity, I’d like to revisit the reasons why it’s important, and the lessons we’ve already learned.

Mission drives Mozilla. People sometimes forget that we’re a non-profit, that our only job is to make the Web a better place. Rapid release advances our mission in important ways. We get features and improvements to users faster. We get new APIs and standards out to web developers faster. We are delivering on the promise of the web at web speed.

Small, frequent releases improve quality, too. Engineers in the Mozilla community regularly say things now like “I don’t like not understanding this piece, let’s back it out and I’ll catch the next train.” We move deliberately. We don’t rush. And, even though it sounds like a contradiction, when we take our time we go faster.

There’s a great deal for us to be proud of, but we also need to be humble. This change was hard for us to make, and it’s been hard for some of our supporters, too. We have been glib or dismissive in the way we’ve communicated about parts of it. We live rapid release daily, and that makes it easier for us to see past the problems. We are also tenacious about the necessity of our new schedule, and tenacity can be mistaken for obstinacy.

We, everyone in the Mozilla community, all of us, need to communicate with clarity and sensitivity. We need to help the people who support our mission to understand why these changes are essential. We need to keep listening, and adjusting as we learn. We need to, and we will.

The push to ship faster isn’t some kind of software machismo. We push ourselves to ship faster because the web is under threat. Amazing and innovative people are doing amazing and innovative things and right now they have a choice: build for the web, or build for the walled gardens. The web can win that fight.

The open web is the most amazing, universal communication and distribution platform ever built. To win, the web needs to be agile and responsive. To help it, we need to be agile and responsive, too. That’s why rapid release matters.