Tracking (and analyzing) what we post on The Atlantic’s homepage

This post was originally published on this site


In the newspaper business, every reporter holds their breath during the “A1 meeting,” when top editors haggle over what should appear on the next day’s front page. The first page of the “A” section is a newspaper’s most important piece of real estate (besides the crossword). And given that picking what goes “on front” is probably the biggest editorial decision a publication makes on any given day, the rulings that come out of that meeting can either make a reporter’s afternoon (hooray, my story is going above the fold!), or ruin it (c’mon, Page B5 again?).

Our new homepage design has eight main curated slots, varying in size and purpose. But that flexibility can be paralyzing. Multiple times a day, our editors ask themselves the same questions:

  • What the heck should we put on the homepage?
  • Wait, didn’t we put that story there earlier?
  • GET THIS STORY UP HURRY HURRY HURRY

Unlike a newspaper’s front page, a homepage has no printed record. That means coming up with the answers to these questions often relied on our homepage editors having good memories, or laboriously tracking articles. What’s more, we couldn’t say much about long-term trends — which types of stories tended to get more homepage love, or how often we switched things up.

A tentative solution

While our homepage is impermanent, that doesn’t mean it can’t be recorded. For example, we could hire an intern to write down the position of every story, every hour, all week long. That would be a pretty miserable internship!

Fortunately, it’s easy enough to code a web script to do essentially the same thing. Enter HomepageCreeper, a name I made up right now:

Every 10 minutes, HomepageCreeper puts itself in the reader’s seat and scrapes all the headlines on The Atlantic’s homepage, logging their position and URL. It drops these stories into a database, marking the times they appeared in a particular slot.

If the headline changes the next time HomepageCreeper comes around, that means a new story has taken over, and the previous story can be closed out.

Pretty simple! But having this scraper helps us in two ways.

First, our homepage staff has perspective about which stories we’ve highlighted before. That makes placement decisions easier, and provides a record to show editors.

Secondly, we’re collecting data that we could later parlay into a deeper analysis of traffic patterns on our site. The homepage still remains an important source of promotion for any given story; at some point, I’d love to pursue an analysis of how homepage placement impacts an article’s total reach.

And on my end, it’s been a neat reminder of how hard our reporters and editors work. Here’s all the stories that cycled through our lead slot on Monday, July 10, as the news regarding Donald Trump Jr.’s communications with a Kremlin-affiliated lawyer heated up:

That would be quite the A1 meeting.


Tracking (and analyzing) what we post on The Atlantic’s homepage was originally published in Building The Atlantic on Medium, where people are continuing the conversation by highlighting and responding to this story.

Comments are closed.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑