The reasons, approach, and next steps
At the beginning of 2016, The Atlantic decided to embrace an industry best practice and transition from HTTP to HTTPS, or “secure.” The decision to devote time and resource to the effort required support and participation across our organization — from our writers who rely on embedding content from a variety of sources, to our advertising operations team that trafficks ad tags, to our developers. And while the decision to move to secure was relatively easy, the process to get there has been a more arduous journey.
What is HTTPS and Why Go Secure?
In 2014, The New York Times called for all sites to transition to HTTPS by the end of 2015. Among the 9 justifications for the move were “Authenticity of News Delivery”, “Privacy” and “Security”.
In a nutshell, HTTPS means that data will be encrypted between our servers and your browser thus reducing opportunities for spying or nefarious interference while you’re reading our site. A user knows a site is secure when they see the “lock” symbol by the URL in a standard browser.
In concept, anything we can do to verify the authenticity and integrity of our site is a no brainer. However, at the end of 2015, Poynter reported that the progress of publishers to embrace HTTPS was slow. Most organizations, including the New York Times, had failed to make the transition.
Our Journey Began
We began our process to transition to HTTPS in April by porting over images and individual elements, so that when it came time for articles to transition to HTTPS, there would be no mixed content on our pages. One of our developers scanned our database to find any of these elements that might break the site post-transition.
Next, we devoted a beta server to testing and addressed third party scripts on the pages, reaching out to third party vendors for HTTPS versions of scripts.
One concern for the transition was that our journalists would inadvertently include insecure images, interactives or other elements in our content management system (CMS). Even though these elements would show up blank in preview, we wanted to ensure errors wouldn’t persist in our backend. To address this, we created a warning that would notify editors on save that their story contained insecure elements.
A Challenge: Insecure Ads
Next, we worked with our ad ops and planning team to change our ad specifications and require HTTPS compatibility from our direct advertisers. We then made a list of our programmatic vendors and began requesting updated tags or code for HTTPS compatibility.
Once we had our tags updated and our QA process in place, we began monitoring the beta site. The most common issue we ran into when testing the secure site were assets that had initially passed QA and later showed up with insecure elements. Often, this was the result of an advertiser changing the creative and inadvertently adding an insecure element, like a tracking tag, after the ad was live.
It was clear that we needed a better monitoring process, and in June we began trials with systems that could do just that. We determined that a key requirement for our new validation software would be to monitor ads on a continuous loop.
Going Live with HTTPS (gradually)
We’ve benefited throughout this process from the learnings of those who’ve gone before us. In particular, these product posts from The Washington Post and Wired were extremely helpful in identifying challenges and pitfalls. Both publishers described gradual transitions to ensure they were safeguarding against search traffic loss and unexpected errors. We knew, we’d want to follow a similar process so we could monitor errors and impact closely.
Once we were comfortable with what we saw in the beta test environment, our first step was to turn off the redirect on visitors that typed in “HTTPS” before “theatlantic.com.” With less than 1% of daily traffic hitting the secure site, we were able to safely monitor for errors. Over a period of 3–4 weeks, we identified one error in a third party script and a handful of third party ad creatives that were not secure.
This week, we will be taking a bigger step as we transition our homepage to secure. This will take us from <1% secure to roughly 20% of the traffic being secure. We plan to monitor that change for 1–2 weeks. If all goes well, we’ll move onto making newly published content secure and finally, to redirecting the archive. In total, we are aiming to complete the transition to 100% secure by the end of 2016.
With a little luck, we’ll be ringing in 2017 with a fully secure site. And that’s something to toast to! 🍻
Many thanks to Delaney Chambers for her contributions to this project and this post.
Our Journey to Secure: Transitioning to HTTPS at The Atlantic was originally published in Building The Atlantic on Medium, where people are continuing the conversation by highlighting and responding to this story.