Open Speaker Series: Camille Fournier on Organizational Culture

Editor’s note: This is a recap from the Open Speaker Series, a regular series of talks held in-house at the Times featuring industry leaders in technology, design, product, organizational culture and leadership.

The Open Speaker Series and Women in Tech recently co-sponsored a conversation with Camille Fournier, founding CTO of Rent the Runway and the author of The Manager’s Path and Ask the CTO. Camille is currently Managing Director of Platform Engineering at Two Sigma.

The discussion centered on three topics — career, management advice and promoting diversity in tech.

Here are five highlights:

  • On learning and leadership: “I think the most important thing new people in tech can do is get comfortable looking dumb. The most important thing that experienced people in tech can do is get comfortable letting people ask dumb questions and not shaming them for asking dumb questions.”
  • On developing a diverse organization: “I found that when I was more flexible in where I looked, I found really amazing talent that had more non-traditional backgrounds, and they were more creative and actually worked better with the product and the team that we needed to build.”
  • On dealing with bureaucracy: “I do encourage always digging in on the bottlenecks and inefficiencies in process, and asking the question and raising the issue and seeing what happens.”
  • On improving technical interviews: “I don’t think that someone has cracked the code of how to give the best, most accurate interview […] I think that questioning, “what are we even looking for?”, in an interview is a good thing to do. I definitely think questioning, “how do we determine who is qualified to interview with us, for which roles?”, is another good thing to do.”
  • On self-improvement: “I am way smarter because I know a hundred people smarter than me that are willing, that I have helped out myself and who can then teach me things in return. Don’t expect to know it or do it all yourself. You’re never going to be able to successfully do it all yourself, but relying on those around you — and being there for those around you — delegating your brain out a little bit. You’d be surprised what people will do if you just ask them nicely.”

Open Speaker Series: Camille Fournier on Organizational Culture was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

React, Relay and GraphQL: Under the Hood of the Times Website Redesign

The New York Times website is changing, and the technology we use to run it is changing too.

As the new site rolls out over the next several months, a look under the hood will reveal a number of modern technologies designed to make the site faster and easier to use — for readers, most importantly, but also for our developers.

At the center of this has been our adoption of React, Relay and GraphQL.

Take a look under the hood …

The problem we’re solving

More than a year ago, when we first started talking about the technology that would power our new website, simplifying our stack was one of our biggest priorities.

Our current desktop and mobile websites are written in entirely different languages: The desktop is predominately PHP; mobile runs on Node. Other products, such as our foreign-language sites, run on their own unique codebases (Español, 中文 (Chinese). Some do not even rely on our standard CMS, Scoop. All these sites read data from different origins and endpoints in different ways. It is hard to find a common denominator between them all.

If I want to make a new app tomorrow, chances are I need to:

  • Obtain credentials for multiple web services
  • Write an HTTP client (for the umpteenth time) to talk to said services
  • Create my view layer, probably from scratch, because there is no real central repository for NYT components

We thought it would be nice if there was one place to add and retrieve data and one way to authenticate against it. It would also be helpful if there was a common language and repository for creating and reusing components. If a new developer joins our team, I want to point them at a single page of documentation that explains how to get up and running — and preferably start building apps the same day.

This is not at all a dream scenario. We are moving towards this reality. That future is Relay and GraphQL.

GraphQL and Relay

Relay is an open source project by Facebook that exposes a framework they have been using internally for years. It is the glue that binds components written in React to data fetched from a GraphQL server. Relay is written in JavaScript, and we are using it as the basis for our new website’s codebase to power our desktop and mobile versions as one on Node.

GraphQL is “a query language for APIs”, which has a default implementation in Node. Facebook developed it to provide a data source that can evolve without breaking existing code and to favor speed on low-powered and low-quality mobile devices. The schema can evolve, but should never break. Products are described in graphs and queries, instead of the REST notion of endpoints.

It works like this: GraphQL queries contain nodes, and only the nodes that are requested are returned in a given response. GraphQL nodes do not have to represent a flat data structure — each node can be resolved in a custom manner. Here is a simple example of a GraphQL query:

  me {
    friends {

It doesn’t matter how the query is resolved. The hard initial work is designing it in a way that will survive redesigns, backend migrations and framework changes.

A query might be resolved by multiple data sources: REST APIs, database, a flat JSON file. A product might begin by returning data from a simple CSV file, and later be grow to return data from a cluster of databases or remote storage like BigTable.

GraphQL is simply a clearinghouse for queries. It also comes with a tool called GraphiQL that allows you to view and debug your queries visually. And Facebook has open-sourced a library, called DataLoader, that makes it easy to query multiple backends asynchronously without having to write custom Promise logic that ends up in callback hell.

Relay acts as a partner to GraphQL and React. A top-level query usually happens on a route — a URL pattern that loads a component when it is matched.

// queries/Page.js

import { graphql } from 'react-relay';

const PageQuery = graphql`
  query Page_Query($slug: String!) {
    viewer {

// routes/index.js
import Route from 'found/lib/Route';
import Page from 'routes/Page';
import PageQuery from 'queries/Page';

GraphQL “fragments” are co-located with your React components. A component describes what slices of data it needs on certain types. Relay queries “spread” the fragments of other components. In this particular case, the “slug” is extracted from the URL path and passed to our GraphQL query. The Page component will be populated with a “viewer” prop that contains the data specified below:

// routes/Page/index.js
import { graphql, createFragmentContainer } from 'react-relay';
import styles from './Page.scss';
const Page = ({ viewer: { page } }) => {
  if (!page) {
    return <Error />;

  const { title, content, featuredMedia } = page;

  return (
    <article className={styles.content}>
        <h1 className={styles.title}>{title}</h1>
      {featuredMedia && <Media media={featuredMedia} />}
      <section dangerouslySetInnerHTML={{ __html: content }} />

export default createFragmentContainer(
    fragment Page_viewer on Viewer {
      page(slug: $slug) {
        featuredMedia {
          ... on Image {

As React components become more nested, queries can become increasingly complex. In Relay Classic, all of the query-parsing logic happened at runtime. As of Relay Modern, queries are now parsed at build time and are static at runtime. This is great for performance.

One Caveat

Migrating from Classic to Modern can be a big lift. The project has provided a compatibility guide to allow your code to incrementally adopt new features, but the fragmented nature of the Node ecosystem can make this complex. Your codebase might be on the latest version, but some of your dependencies might be pinned to an earlier version.

We handle a lot of the complexity around upgrades and dependencies using our open source project, kyt. React Modern is such a massive improvement that it requires a “cutover” of old to new code.

However, the benefits are exciting. By default, GraphQL queries are sent to the server by Relay as an HTTP POST body containing the text of a query and the variables needed to fulfill it. Queries compiled by Relay Modern at build time can be persisted to a datastore, and IDs can be sent to the GraphQL server instead. We look forward to taking advantage of this optimization.

It has been exciting moving our codebase to React, and leaning on the great features kyt provides out of the box, such as CSS Modules. We are finally creating the central component repository we’ve longed for.

As we transition from away from using REST APIs, we no longer have to query the canonical representation of an article, when all we really need in some display modes is five to seven fields.

When we want to update our design across all products, we will no longer have to make changes across several codebases. This is the reality we are moving towards. We think Relay and GraphQL are the perfect tools to take us there.

Scott Taylor is a senior software engineer on the Web Frameworks team.

React, Relay and GraphQL: Under the Hood of the Times Website Redesign was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

All In a Week’s Work — How We Tackled the Five Day Sprint Process

All In a Week’s Work — How We Tackled the Five Day Sprint Process

Kelci Shipley/The New York Times

It may have been the second week in October, but it felt like summer camp. For five consecutive days, eight members of the New York Times Games team hid away in a conference room armed with pads of sticky notes, a blank whiteboard wall and a bit of nervous energy: we were actually going to give this thing a shot.

“This thing” being an entire work week dedicated to product development and design thinking, inspired by the book “Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days” by Jake Knapp, John Zeratsky and Braden Kowitz at Google Ventures.

Our team is known for the New York Times Crossword, but we’re always thinking about ways to improve our existing product and new playing experiences we could create. Compressing these ambitions into the five day sprint process meant that each day had a specific focus, which allowed the week to go from talking about big problems on Monday to testing a solution with a working prototype on real users on Friday.

But we didn’t just show up Monday to see what happened — we did a lot of prep work. We assembled our sprint team and assigned roles. I took on the role of Facilitator, which meant reading the book cover to cover and knowing how to guide the team through the process while keeping everyone on track.

By the time the first morning came, we were ready.


The first day also happened to be the boldest. The morning started by revisiting our long-term goal to create an experience that gives our users a sense of accomplishment every day. We turned our assumptions into thoughtful questions and asked ourselves things like, “What does it mean for a user to feel accomplished?” and “What motivates them?”.

Kelci Shipley/The New York Times

The afternoon was spent talking to our Marketing, Design, Editorial and Technology teammates who are experts on our product. Those conversations helped us define the challenges we wanted to focus on for the week: to create a sense of fulfillment and meet users in their individual moments.

Our product’s complex nature means that we have an array of users with different levels of knowledge and skill. But whether our users can solve a Monday puzzle in 15 minutes, or whether they consider doing half of a Sunday puzzle a victory, we want them to feel good in each instance of their experience.

At the end of the day, we synthesized this information and mapped it to a user journey, which helped us focus on the solution we wanted to design for.


With day one behind us, we started sketching solutions to the problems we defined in our discovery. We started by discussing elements of comparable experiences that we could look to for inspiration, such as developing reading skills or feeling accomplished in a fitness app. The solution sketching segments were highly structured and timed, meant to emphasize critical thinking over artistry. Each person left with one solution sketch to be reviewed the next day.


One of the keys for the week was to keep moving through it, even when we found ourselves in moments of doubt. The uncertainty of our success was my greatest challenge because I had to balance the execution of a particular day’s segment with the hope that I was guiding everyone on the correct path. This made Wednesday a critical day in the sprint, and a convenient one to get breakfast catered.

Energized and caffeinated, we reviewed each person’s sketches using methods of evaluation such as silent voting, written comments and verbal discussion. The methodical approach felt a bit awkward at times, but it minimized irrelevant tangents and accelerated decision-making. And it led us to one of the most exciting moments of the week: illuminating the thing we would create.

At this point, we threw some of “Sprint’s” rules out the window. The book suggests creating a prototype with elements from each sketch, but we decided to focus on one idea we really liked and evolve it.

Kelci Shipley/The New York Times


On Wednesday, we put together a storyboard of how our prototype would function, so Thursday was devoted to bringing that prototype to life. To make our prototype, we used a sequence of pieces of paper to mimic the look and feel of interacting with a digital screen. The afternoon was spent designing game elements, crafting editorial and shuffling through a lot of Spotify songs. I did my best to keep the momentum going with help from of Michael Jackson, Whitney Houston and other ’80s pop icons.

And then like that, it was on to day five.


Getting to Friday is an accomplishment in and of itself. By design, the week was packed with moments of anticipation, doubt and exhilaration. Though we felt all of these emotions leading up to Friday, when we gathered to watch real people interact with our prototype a sense of calm settled over the room.

Armed once again with pads of sticky notes and sharpies we noted interesting behaviors, insights and emotions as the five user testing sessions were conducted by The Times’ Audience Insights team. Through the user testing, we were able to confirm that what we created was something people were delighted to play. But even if our prototype was something people didn’t enjoy, it wouldn’t have meant the sprint was unsuccessful.

The point of going through a five day sprint is to learn and gather insights, so the pressure wasn’t on whether or not the prototype performed well (although it was a nice bonus). In addition to seeing what worked well in the prototype, we learned what things people craved and how we could provide our users with an even better experience.

The goal of the sprint was to learn and we achieved it.

So what happens now? After that week, we left with many questions we still wanted to answer. We’ve been actively iterating on the prototype and figuring out more variables we want to test with users.

The sprint asked for a big commitment, however our return on the five day investment was immense. We gained new tools and frameworks for executing brainstorms and we continue to incorporate pieces from this process into our discovery strategy.

Kelci Shipley is a Program Manager for The New York Times Games team. She works closely with the Crossword product and the people that build it.

All In a Week’s Work — How We Tackled the Five Day Sprint Process was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

The New York Times, with a little help from automation, is aiming to open up most articles to comments

The New York Times’ strategy for taming reader comments has for many years been laborious hand curation. Its community desk of moderators examines around 11,000 individual comments each day, across the 10 percent of total published articles that are open to commenting.

For the past few months, the Times has been testing a new tool from Jigsaw — Google parent Alphabet’s tech incubator — that can automate a chunk of the arduous moderation process. On Tuesday, the Times will begin to expand the number of articles open for commenting, opening about a quarter of stories on Tuesday and shooting for 80 percent by the end of this year. (Another partner, Instrument, built the CMS for moderation.)

“The bottom line on this is that the strategy on our end of moderating just about every comment by hand, and then using that process to show readers what kinds of content we’re looking for, has run its course,” Bassey Etim, Times community editor, told me. “From our end, we’ve seen that it’s working to scale comments — to the point where you can have a good large comments section that you’re also moderating very quickly, things that are widely regarded as impossible. But we’ve got a lot left to go.”

These efforts to improve its commenting functions were highlighted in the Times announcement earlier this month about the creation of a reader center, led by Times editor Hanna Ingber, to deal specifically with reader concerns and insights. (In the same announcement, it let go Liz Spayd and eliminated its public editor position.)

Nudging readers towards comments that the Times “is looking for” is no easy task. Its own guidelines, laid out in an internal document and outlining various rules around comments and how to take action on them, have evolved over time. (I took the Times’ moderation quiz — getting only one “correct” — and at my pace, it would’ve taken more than 24 hours to finish tagging 11,000 comments.)

Jigsaw’s tool, called Perspective, has been fed a corpus of Times comments that have been tagged by human editors already. Human editors then trained the algorithm over the testing phase, flagging mistakes in moderation it made. In the new system, a moderator can evaluate comments based on the likelihood of rejection and checks that the algorithm has properly labeled comments that fall into a grayer zone (comments with 17 to 20 percent likelihood of rejection, for instance). Then the community desk team can set a rule to allow all comments that fall between 0 to 20 percent, for instance, to go through.

“We’re looking at an extract of all the mistakes it’s made, evaluate what the impact of each of those moderating mistakes might be on the community and on the perceptions of our product. Then based on that, we can choose different forms of moderation for each individual section at the Times,” Etim said. Some sections could remain entirely human-moderated; some sections that tend to have a low rate of rejection for comments could be automated.

Etim’s team will be working closely with Ingber’s Reader Center, “helping out in terms of staffing projects, with advice, and all kinds of things,” though the relationship and roles are not currently codified.

“It used to be when something bubbled up in the comments, maybe we’d hear repeated comments or concerns about coverage. You’d send that off to a desk editor, and they would say, ‘That’s a good point; let’s deal with this.’ But the reporter is out reporting something else, then time expires, and it passes,” Etim said. “Now it’s at the point where when things bubble up, [Ingber] can help us take care of it in the highest levels in the newsroom.”

I asked Etim why the Times hadn’t adopted any of the Coral Project’s new tools around comment moderation, given that Coral was announced years ago as a large collaborative effort between The Washington Post, the Times, and Mozilla. It’s mostly a matter of immediate priorities, according to Etim, and he can see the Times coming back to the Coral Project’s tools down the line.

“The Coral Project is just working on a different problem set at the moment — and the Coral Project was never meant to be creating the New York Times commenting system,” he said. “They are focusing on helping most publishers on the web. Our business priority was, how do we do moderation at scale? And for moderation at our kind of scale, we needed the automation.

“The Coral stuff became a bit secondary, but we’re going to circle back and look at what it has in the open source world, and looking to them as a model for how to deal with things like user reputation,” he added.

Photo of chattering teeth toy by Wendy used under a Creative Commons license.

Why Having a Diverse Team Will Make Your Products Better


In March, The New York Times published an article about how hard navigating the subway can be in a wheelchair, and it touched me a lot more than I thought it would when I started reading it. Months later, it’s still on my mind. The piece was penned by Sasha Blair-Goldensohn, an engineer from Google who had an accident that left him partially paralyzed. As he started navigating the city on wheels he discovered that “inflexible bureaucracies with a ‘good enough’ approach to infrastructure and services can disenfranchise citizens with disabilities, many of whom cannot bridge these gaps on their own.”

Most subway stations in New York are not wheelchair accessible, and the ones that are often have broken elevators that can leave commuters stranded above or below ground. I’d like to imagine what the considerations for how to build an accessible subway system would be if there were more people with disabilities on the teams that make these decisions. It seems like more thoughtful accessibility would make everyone’s experience with this public product better.

It’s so exciting for me to see the ways in which the conversation about diversity and its impact on product development is accelerating. Diversity comes in many forms, and can include characteristics that are innate or acquired, such as country of birth, being multilingual, degree of abledness, race and socioeconomic background, among others. Having less homogenous teams makes us more innovative, can make us smarter and increases profits.

Product Successes

The variety of perspectives that come from diverse teams help make products stronger, and ultimately serve users better.

Companies with more women are more likely to introduce radical new innovations into the market (hello, Rent the Runway and Stitch Fix!) and companies with a culturally diverse leadership team are more likely to develop new products.

Slack celebrated their diversity very publicly last year when they sent four black female engineers to accept their award from TechCrunch for fastest rising startup.

Sallie Krawcheck started Ellevest in response to an investing industry that was primarily “by men, for men” and kept women from achieving their financial goals. Halla Tómasdóttir steered financial services firm Audur Capital through the financial storm in Iceland by applying traditionally “feminine” values.

And after almost 60 years, Barbies finally come in different shapes, sizes, skin tones and eye color in an effort to attract a wider demographic and increase sales.

Product Failures

Without diverse perspectives and experiences in designing, building and testing, products can and will fail female and minority users.

Some of the first air bags to be installed cars failed to protect women because they were built to men’s specifications, tested with male crash test dummies, and didn’t take the female anatomy into account. The first voice recognition programs didn’t recognize female voices or many accents because they were built and tested by men and native English speakers. You may remember Google Photos’ image recognition software labeled two black people as “gorillas.” When Apple first launched their Health app, there was one glaring exception to their promise to “monitor all of your metrics that you’re most interested in”: it didn’t track menstruation. And some phones are even too big for women’s hands.

Remember Microsoft’s paperclip office assistant? The company spent $100k on market testing and ignored female participants’ feedback that the characters were too male (90% of women didn’t like the characters). The reaction was largely rejected because the men leading the project couldn’t see the issue themselves; they shipped the product with 10 male characters and 2 female characters.

One of the hardest things to remember when building products is that you are not your user.

Tips for Building Better Products

Staffing teams with people who think differently from one another can remind us of our blind spots and hopefully lead us to better solutions for all users. Here are some things you can do to make better products:

  • Great products start with great teams, so ensure you have a diverse team to design, build and test your products.
  • Create spaces that guarantee everyone’s voice is heard by remembering that not everyone likes to speak up in meetings. Set agendas ahead of time so everyone can contribute; create space in group meetings for individual brainstorming; and provide other channels for feedback.
  • Cultivate an environment that includes psychological safety: this allows team members to take risks and speak up when they have novel or particularly unorthodox ideas.
  • Talk to users! If organizing in-house user testing is too difficult, ensure users can submit feedback via email and then make sure time is dedicated to reading some of their responses. Regularly check the public feedback you get in Google Play or the App Store. To make keeping up with these messages simpler, consider setting up a bot that sends these messages to a Slack channel.
  • Reading user feedback makes it harder for you to ignore user needs that you don’t personally feel. It can be easy to make assumptions when building a product, but listening to users whose experiences are different from your own can highlight issues you may not have encountered.

With a less homogenous mix of voices at the table, imagine what we can build! If you’re hiring for digital teams and having trouble figuring out ways to diversify your staff, there are numerous organizations you can partner with, including Coalition for Queens, Code 2040 and Power to Fly.

Modupe Akinnawonu is a product manager at The New York Times. She focuses on their Android news app, among other projects.

Why Having a Diverse Team Will Make Your Products Better was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introducing the New Open Blog


Illustration by Erik Söderberg

I’m very pleased to announce today that we are relaunching The New York Times Open blog. In the process, we are making three important changes:

First, while the blog began life with an engineering focus, we are expanding coverage to include everyone who builds our digital products at the Times. You’ll see posts on design, product development, management, editorial, and yes, definitely engineering. Most posts will come from our team, but you may also see occasional guest posts, from people we are collaborating with in some way.

Second, we are greatly increasing our output. Previously we posted “every sometimes”, but from now on we are committed to posting weekly — at least. So be sure to follow us here on Medium, as well as on Twitter.

Last, as you can see, we are on Medium! While it may seem strange for us, a publisher, to post on Medium rather than our own platform, we are here for a simple reason: the community. Medium is where so many of our people are, so much of the product, design and development community, so we wanted to be here too.

So what is Times Open all about? Why are we doing this at all? We are very proud of our crew, and want to give them a platform to share the good work they are doing every day. We also want to share with the community, and help others who may be able to learn from our work. Writing is good for the mind — it helps each of us organize our thoughts, and become better communicators and thinkers.

Today we are launching with three new posts:

And you can also take a look at the archives, which we have ported over from the old blog. Here are a few of my favorites:

So take a look, and watch this space. And, huge thanks to Chase Davis, Sarah Bures and Allen Tan for getting this next version of Open off the ground!

Introducing the New Open Blog was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Designing a Faster, Simpler Workflow to Build and Share Analytical Insights


Data is critical to decision-making at The New York Times. Every day, teams of analysts pore over fine-grained details of user behavior to understand how our readers are interacting with The Times online.

Digging into that data hasn’t always been simple. Our data and insights team has created a new set of tools that allows analysts to query, share and communicate findings from their data faster and easier than ever before.

One is a home-grown query scheduling tool that we call BQQS — short for BigQuery Query Scheduler. The other is the adoption of Chartio, which our analysts use to visualize and share their results.

The result has been more analysts from more teams being able to more easily derive insights from our user data. At least 30 analysts across three teams now have almost 600 queries running on a regular cadence on BQQS, anywhere between once a month to every five minutes. These queries support more than 200 custom dashboards in Chartio. Both represent substantial improvements over our previous model.

What problems were we trying to solve?

This effort began when we migrated our data warehousing system from Hadoop to Google’s BigQuery. Before we built new tools, we worked with analysts to come up with several core questions we wanted to answer:

  • What patterns and processes did the analysts use to do their work?
  • Which of those processes could we automate, in order to make the process more hands-off?
  • How could we make it easier for our growing list of data-hungry stakeholders to access data directly, without having to go through an analyst?
  • How could we ensure ease of moving between business intelligence products to avoid attachment to eventual legacy software?

Until the migration to BigQuery, analysts primarily queried data using Hive. Although this allowed them to work in a familiar SQL-like language, it also required them to confront uncomfortable distractions like resource usage and Java errors.

We also realized that much of their work was very ad-hoc. Regular monitoring of experiments and analyses was often discarded to make way for new analyses. It was also hard for them to share queries and results. Most queries were stored as .sql files on Google Drive. Attempts to solve this using Github never took off because it didn’t fit with analysts’ habits.

The act of automating queries was also unfamiliar to the analysts. Although the switch to BigQuery made queries much faster, analysts still manually initiated queries each morning. We wanted to see if there way ways to help them automate their work.

Query Scheduling with BQQS

Before we considered building a scheduling system in-house, we considered two existing tools: RunDeck and AirFlow. Although both of these systems were good for engineers, neither really provided the ideal UI for analysts who, at the end of the day, just wanted to run the same query every night.

Out of this came BQQS: our BigQuery Query Scheduler. BQQS is built on top of a Python Flask stack. The application stores queries, along with their metadata, in a Postgres database. It then uses Redis to enqueue queries appropriately. It started with the ability to run data pulls moving forward, but we eventually added backfilling capabilities to make it easier to build larger, historical datasets.

A testing dashboard in BQQS

This solution addressed many of our pain points:

  • Analysts could now “set it and forget it,” barring errors that came up, effectively removing the middleman.
  • The system stored actual analytics work without version control being a barrier. The app stores all query changes so it’s easy to find how and when something changed.
  • Queries would no longer be written directly into other business intelligence tools or accidentally deleted on individual analysts’ computers.

Dashboards with Chartio

Under our old analytics system, “living” dashboards were uncommon. Many required the analyst to update data by hand, were prone to breaking, or required tools like Excel and Tableau to read. They took time to build, and many required workarounds to access the variety of data sources we use.

BigQuery changed a lot of that by allowing us to centralize data into one place. And while we explored several business intelligence tools, Chartio provided the most straightforward way to connect with BigQuery. It also provided a clean, interactive way to build and take down charts and dashboards as necessary.

One example of a dashboard generated by Chartio

Chartio also supported team structures, which meant security could be handled effectively. To some degree, we could make sure that users had access to the right data in BigQuery and dashboards in Chartio.

Developing new processes

Along with new tools, we also developed a new set of processes and guidelines for how analysts should use them.

For instance, we established a process to condense each day’s collection of user events — which could be between 10 and 40 gigabytes in size — into smaller sets of aggregations that analysts can use to build dashboards and reports.

Building aggregations represents a significant progression in our analytical data environment, which previously relied too heavily on querying raw data. It allows us to speed queries up and keep costs down.

In addition, being able to see our analysts’ queries in one place has allowed our developers to spot opportunities to reduce redundancies and create new features to make their lives easier.

Moving forward

There’s much more work to do. Looking ahead, we’d like to explore:

  • How to make it easier to group work together. Many queries end up being the same with slightly different variables and thus a slightly different result. Are there ways to centralize aggregations further so that there are more common data sets and ensure data quality?
  • Where it makes sense to design custom dashboard solutions, for specific use cases and audiences. Although Chartio has worked well as a solution for us with a smaller set of end-users, we’ve identified constraints with dashboards that could have 100+ users. This would be an excellent opportunity to identify new data tools and products that require the hands of an engineer.

Shane Murray is the VP of the Data Insights Group. Within that group, Josh Arak is the Director of Optimization and Ed Podojil is Senior Manager of Data Products.

Designing a Faster, Simpler Workflow to Build and Share Analytical Insights was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Headline Balancing Act


The New York Times can be read on your phone, tablet, laptop, and on many other networked screens, and it’s impossible to know in advance how every headline appears on every display. Sometimes, the headline wraps just fine. But there are many times when they don’t, introducing unsightly widows. Even when there aren’t strict widows, instances where one line is dramatically shorter than others can still hurt legibility and reading flow.

These blemishes are easily fixed in print. On a fixed canvas, we can fit copy to fill a space, and designers can work with editors to get the text to behave just right. On the web, where space is dynamic, we can’t adjust layouts by hand. But that doesn’t mean we have to just accept bad typography, we just have to use a different approach: translate and codify good design guidelines (which can be intuitive and circumstantial) into a concrete, reusable set of instructions.

We have made several attempts to tackle this problem. For a while, we were relying on Adobe’s balance-text jQuery plugin on special feature articles. While the result looked great, performance was not ideal: sharp-eyed readers would see the headline update after the page’s fonts loaded. And since the headline is one of the first things someone will look at, this was not great.

The previous jQuery plugin (from Adobe) in action.

So during our Maker Week last summer, I suggested coming up with a more robust headline balancer that could be used anywhere — not just special features. After seeing some examples of bad wrapping, a few engineers agreed to search for a better solution. The winning idea came from one of our interns, Harrison Liddiard. He came up with a lightweight implementation (without a jQuery dependency, even!) that gave us what we were looking for.

Michael Beswetherick proceeded to make this script ready for production. Combing through hundreds of headlines of varying lengths, we measured the effectiveness and efficiency of our balancer, adjusting based on what we saw. You can see the before/after for just a few of the headlines:

And now … it can be yours

We’re more than a little excited to release our work on Github. Now, we realize that a piece of code that only works on headlines might not be very useful, so we’ve abstracted our solution and named it text-balancer: (and also, an npm module)

(Now, remember: moderation is best in all things. You should apply this selectively, not to everything you can get your hands on. We suggest headlines, blockquotes, and other places where you’re using large display type. We do not recommend using this on body type, buttons, or navigational links.)

Wondering how text-balancer actually works? Well, it’s a binary search algorithm applied against a text element’s max-width. We then adjust its max-width until the element squeezes right up to the point that it spills onto another line.

Here it is in action: (slowed down so you can see how it works)

We calculate the max-width to be the average of a bottom range that starts at 0 and changes depending on whether the updated max-width makes the text fall onto another line.

One of the more subtle aspects of text-balancer is that the text element will always remain the same number of lines that it was before. It will also re-adjust when the browser size changes; all you need to do is set it up once and you can rest assured that the text will always be balanced.

One Last Thing

When we were first testing it, we kept noticing that our text look…well, didn’t actually look balanced. Finally, we figured out that we were using text-balancer before the headline font had finished loading. So: you should wait to run text-balancer until after your fonts have loaded.

We looked for a way to detect when our fonts had loaded and came across Bram Stein’s Calling the observer’s load method returns a promise that will tell us the right time to balance our text.

const chelt = new FontFaceObserver('cheltenham');
chelt.load().then(() => {
console.log('fonts have loaded yay');

What Else?

In the future, we’d like to be able to place line breaks with awareness to style guide conventions: not splitting within names and phrases, not splitting after a lowercase word, and so on. (If someone wants to add this and send in a pull request, we won’t say no.)

In the meantime, give text-balancer a try and let us know what you like or don’t like!

A balanced headline on an article.

Headline Balancing Act was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Building a Cross Platform 360-degree Video Experience at The New York Times


Over the past few months, 360-degree videos have gained a lot of traction on the modern web as a new immersive storytelling medium. The New York Times has continuously aimed to bring readers as close to stories as possible. Last year we released the NYT VR app with premium content on iOS and Android. We believe VR storytelling allows for a deeper understanding of a place, a person, or an event.

This month, we added support for 360-degree videos into our core news products across web, mobile web, Android, and iOS platforms to deliver an additional immersive experience. Times journalists around the world are bringing you one new 360 video every day: a feature we call The Daily 360.

The current state of 360 videos on the Web

We’ve been using VHS, our New York Times Video Player, for playback of our content on both Web and Mobile Web platforms for the last few years. Building support for 360 videos on those platforms was a huge challenge. Even though the support for WebGL is relatively mature nowadays, there are still some issues and edge cases depending on platform and browser implementation.

To circumvent some of those issues, we had to implement a few different techniques. The first was the use of a “canvas-in-between”: We draw the video frames into a canvas and then use the canvas to create a texture. However, some versions of Microsoft Internet Explorer and Microsoft Edge are not able to draw content to the canvas if the content is delivered from different domains (as happens with a content delivery network, or CDN), even if you have the proper cross-origin resource sharing (CORS) headers set. We investigated this issue and found out that we could leverage the use of HTTP Live Streaming through the use of an external library called hls.js to avoid this problem.

Safari also has the same limitation regarding CORS. It seems to have been an issue in the underlying media framework for years and for this scenario, the hls.js workaround doesn’t solve the problem. We tackled this issue with the combination of two techniques:

  • The creation of an iframe with the video player embedded in it.
  • The use of progressive download renditions such as MP4 or WebM on the embedded player.

By doing this, we avoid the CORS video texture bug since the content and the iframe are in the same domain as the CDN and we were able to show the player in the parent domain, and the content inside the iframe.

Many of our users see our videos from within social media apps on their phones. On iOS, almost all of these social network applications load off-site content on their own in-app browsers instead of using the native browser, which raises a longstanding technical issue: the lack of support for inline playback video, even on iOS 10. This happens because inline playback support is still disabled by default on web views.

The technical problems listed above aside, the basic theory on how we should view 360 content is pretty straightforward. There are basically four steps to implement a simple 360 content view solution:

  1. Have an equirectangular panoramic image or video to be used as a source.
  2. Create a sphere and apply the equirectangular video or image as its texture.
  3. Create a camera and place it on the center of the sphere.
  4. Bind all the user interactions and device motion to control the camera.

These four steps could be implemented just using the WebGL API but there are 3D libraries like three.js that provide an easier way to use renderers for canvas, svg, CSS3D and WebGL. The example below shows how one could implement the four steps described above to render 360 videos or images:

CodePen Embed – sphere 360

When we first started to work on supporting 360 video playback on VHS, we researched a few projects and decided to use a JavaScript library called Kaleidoscope. Kaleidoscope supports equirectangular videos and images in all versions of modern browsers. The library is lightweight at 60kb gzipped, simple to use and easy to embed into the player when compared with other solutions.

The 360 video mobile native app experience on iOS and Android

Solving 360 video playback on iOS and Android was interesting and challenging since there wasn’t a video library that satisfied our requirements on both platforms. As a result, we decided to go with a different approach for each platform.

For the iOS core app, we created a small Objective-C framework that uses the same approach as Kaleidoscope. Initially we considered to start the development using Metal and OpenGL, but those are lower-level frameworks which require significant development work to create scenes and manipulate 3D objects.

Luckily, there’s another option: SceneKit is a higher-level framework that allows manipulation and rendering of 3D assets in native iOS apps. Investigation revealed that SceneKit provided adequate playback performance, so we chose to use it to render the sphere and camera required for 360-degree video playback.

We also needed to extract video frame buffers into a 2D texture to be applied as a material for the sphere, and to do that we decided to use SpriteKit. SpriteKit is a powerful 2D graphics framework commonly used in 2D iOS games. Our playback framework uses a standard iOS AVPlayer instance for video playback and uses SpriteKit to render its video onto the sphere.

Finally, we bind user interactions and device movements to control the camera’s motion using standard iOS gesture recognizers and device motion APIs.

By using these tools we were able to create a 360 video framework that is very similar to Kaleidoscope. We call it NYT360-Video, and we are happy to announce that we are open sourcing the framework.

On the Android platform we did a deep evaluation of some open source libraries that support 360 video and images, and after an initial prototyping, the Android team decided to use the Google VR SDK. The NYTimes Android app works on various devices and Android OS versions, and Google VR SDK has the features and capabilities that we needed and a straightforward API that allowed a relatively easy integration.

The Google VR SDK has evolved quite a lot from the day we started to work on the integration, and the Google VR team has invested a lot of time improving the project. Along the way, we worked together with Google on feature requests and bug fixes, and that collaboration gave us the certainty that we made the right decision to adopt it. The integration worked as we expected and now we have an immersive 360 video experience on Android.

The future of 360 video encoding and playback at The New York Times

We are investigating new ways of encoding and playing 360 videos to increase performance and improve the user experience. We are excited to explore other interesting features such as spatial audio, stereo images and video.

On the video transcoding side, we are exploring the use of cube map projections, which avoid the use of equirectangular layouts for a more space efficient approach. In theory, we can reduce the bitrate applied to the video by approximately 20 percent while keeping the same quality.

Below you can see a very basic example on how we could support playback of 360 videos encoded with cube map:

CodePen Embed – cubemap 360

The use of cube map projections is a more complex approach than using equirectangular projections since it would not only require changing our video player but also the way we transcode our videos. Earlier this year Facebook released a project called Transform, an FFmpeg filter that converts a 360 video in equirectangular projection into a cube map projection. We are investigating ways to integrate this into our video pipeline. We are also open sourcing the video encoding presets that we use to transcode all of our 360 video outputs.

We hope to receive your crucial feedback and generate contributions from the open source community at large. Feel free to ask questions via GitHub Issues in each project.

Check them out:

Building a Cross Platform 360-degree Video Experience at The New York Times was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Using Microservices to Encode and Publish Videos at The New York Times


For the past 10 years, the video publishing lifecycle at The New York Times has relied on vendors and in-house hardware solutions. With our growing investment in video journalism over the past couple of years, we’ve found ourselves producing more video content every month, along with supporting new initiatives such as 360-degree video and Virtual Reality. This growth has created the need to migrate to a video publishing platform that could adapt to, and keep up with, the fast pace that our newsroom demands and the continued evolution of our production process. Along with this, we needed a system that could continuously scale in both capacity and features while not compromising on either quality or reliability.

A solution

At the beginning of this year, we created a group inside our video engineering team to implement a new solution for the ingesting, encoding, publishing and the syndication of our growing library of video content. The main goal of the team was to implement a job processing pipeline that was vendor agnostic and cloud-based, along with being highly efficient, elastic, and, of course, reliable. Another goal was to make the system as easy to use as possible, removing any hurdles that might get in the way of our video producers publishing their work and distributing it to our platforms and third-party partners. To do that, we decided to leverage the power of a microservices architecture combined with the benefits of the Go programming language. We named this team Media Factory.

The setup

The first version of our Media Factory encoding pipeline is being used in production by a select group of beta users at The New York Times, and we are actively working with other teams to fully integrate it within our media publishing system. The minimum viable product consists of these three different parts:

Acquisition: After clipping and editing the videos, our video producers, editors, and partners export a final, high-resolution asset usually in ProRes 442 format. Our producers then upload the asset to an AWS S3 bucket to get it ready for the transcoding process. We implemented two different upload approaches:

  1. An internal API that supports multipart uploads, called video-acquisition-api, used from server-side clients, like small scripts or jobs.
  2. A JavaScript wrapper that uses EvaporateJS to upload files directly from the browser, which is integrated with our internal Content Management System (CMS), Scoop.

Transcoding: After the acquisition step is complete, we use another microservice called video-transcoding-api to create multiple outputs based on the source file. Currently, we create a single HLS output with six resolutions and bitrates to support adaptive streaming, four different H.264/MP4 outputs, and one VP8/WebM for the benefit of the 1 percent of our users on the Mozilla Firefox browser running on Microsoft Windows XP.

The transcoding service is by far the most crucial part of our workflow. In order to integrate with cloud-based transcoding providers, we decided to design a tiny wrapper containing provider-specific logic. This design gives us great flexibility. We can schedule and trigger jobs based on a set of parameters such as speed, reliability, current availability, or even the price of the encoding operation for a specific provider. For instance, we can transcode news clips (which are time-sensitive) on the fastest, most expensive encoding service, while simultaneously transcoding live action videos, documentaries, and animations (which are not time-sensitive) using lower-cost providers.

Distribution: The transcoding step transfers the final renditions into another AWS S3 bucket. Since we use a content delivery network (CDN) to deliver the video to our end users, we need a final step to transfer the files from S3 to the CDN (leveraging Aspera’s FASP protocol to do so). Once the files are on the CDN, our video journalists are able to publish their content on The New York Times.

Giving back to the community

Today, we are open sourcing the video-transcoding-api and the video encoding presets that we use to generate all of our outputs. We are also open sourcing the encoding-wrapper, which contains a set of Go clients for the services we support and that are used by the video-transcoding-api.

We believe the format we’ve created will be of particular interest to the open source community. By leveraging the abstractions found in the video-transcoding-api, any developer can write the code necessary to send jobs to any transcoding provider we support without having to rewrite the base preset or the job specification. Sending a job to a different provider is as simple as changing a parameter.

We currently support three popular transcoding providers and plan to add support for more. See a sample preset below, in JSON format:

  "providers": ["encodingcom", "elementalconductor", "elastictranscoder"],
  "preset": {
    "name": "1080p_hls",
    "description": "1080p HLS",
    "container": "mp4",
    "profile": "Main",
    "profileLevel": "3.1",
    "rateControl": "VBR",
    "video": {
      "height": "1080",
      "width": "",
      "codec": "h264",
      "bitrate": "3700000",
      "gopSize": "90",
      "gopMode": "fixed",
      "interlaceMode": "progressive"
    "audio": {
      "codec": "aac",
      "bitrate": "64000"
  "outputOptions": {
    "extension": "m3u8",
    "label": "1080p"

Our philosophy for presets: “Write once, run anywhere”

Our future plans

In order to fulfill our vision of having a fully open sourced video encoding and distribution pipeline, we thought it best to also tackle the issue of actually encoding the video. We’re officially taking on the development and maintenance of the open source project Snickers to serve this purpose. We’ll not only gain the freedom of deploying our own encoding service anywhere, but we’ll also be able to experiment and implement new features that may not be available with existing service providers or and respond to specific requests from our newsroom. A few examples on the horizon are the automatic generation of thumbnails and accurate audio transcripts.

We’ve also turned our sights to fragmented MP4 (fMP4), and we’ll be investing some time into fully moving to an HLS-first approach for our on-demand videos. In case you missed it, last June at WWDC 2016, Apple introduced fMP4 to the HLS protocol, making it so now almost all devices and browsers support fMP4 playback natively. This means we can now eliminate the overhead of having to transmux the MPEG-TS segments into fMP4 on the fly when playing videos using our video player (we use hls.js to do this) and instead just concatenate and play fMP4 fragments on our local buffer.

Lastly, content-driven encoding is a trendy topic within the online video community, especially after the release of VMAF. We are planning to adopt this approach by splitting the content-driven encoding project into two phases:

  1. Classify our content into four different categories, each with its own preset. For example, animation videos, like the ones we have for our Modern Love show, require fewer bits than our high-motion videos, like some of our Times Documentaries, to achieve the same content fidelity.
  2. Create and integrate an additional microservice within the Media Factory pipeline for the purpose of checking the quality of our outputs using VMAF and triggering new re-encoding jobs with optimized presets.

Come along with us!

Our Media Factory team (Maxwell Dayvson da Silva, Said Ketchman, Thompson Marzagão, Flavio Ribeiro, Francisco Souza and Veronica Yurovsky) believes that these projects will help address the encoding challenges faced by many of you in the online video industry. We hope to receive your crucial feedback and generate contributions from the open source community at large.

Check them out:

And feel free to ask questions via GitHub Issues in each of the projects!

Using Microservices to Encode and Publish Videos at The New York Times was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.



We are thrilled to announce that we have begun to enable HTTPS on, an effort that helps protect the privacy of our readers and ensures the authenticity of our content. This is a significant milestone in the 21-year history of our website, and though it’s taken us some time, we are very excited to share this wit our readers

What’s included? consists of millions of pages, so we’ve prioritized HTTPS for areas of our site that receive the most visits. You should already be seeing a padlock next to our URL in your browsers on the following:

What Does This Mean for You?

  • Improved privacy: HTTPS encrypts the data sent between your computer and our servers, making it more difficult for a third party to monitor what you are doing. While HTTPS will not hide the fact that you are visiting, it will significantly diminish the ability of a third party, such as your internet provider, to see which articles you are reading.
  • Authentic news: Another benefit of HTTPS is that it validates that your computer is communicating with the website you intended to reach, and that any data you receive has not been modified in-transit. When you see the padlock in your address bar, the browser has validated that you are getting authentic content.
  • Enhanced experience: Some newer web technologies are only made available to HTTPS pages. As we implement HTTPS, we are able to take advantage of these features to make our pages load faster, create innovative interactive projects and provide more personalized content.

HTTPS for the News

The benefits of HTTPS that we wrote about in 2014 remain relevant today. Other media companies have migrated to HTTPS: The Washington Post, Wired, BuzzFeed, The Guardian, and most recently, Quartz. (For more information, the Freedom of the Press Foundation launched a service to track HTTPS implementations on many major media sites.)

It’s been a complex undertaking for us and we’ve discovered a lot in the process. We’ll be sharing a deeper dive into the technical aspects and the challenges we encountered on our journey to HTTPS.

What’s Next?

This is just the beginning, and we intend to bring the rest of our site under the HTTPS umbrella. There is still a significant amount of work remaining, but we are committed to seeing it through. Securing our site is good for our users and the right thing to do. Our core purpose as a company is “to enhance society by creating, collecting and distributing high-quality news and information.” We believe the implementation of HTTPS furthers this purpose.

HTTPS on was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑