What resources do we need to break down barriers to open science? #MozFest 2017 session recap

I was recently interviewed for Read, Write, Participate about my Mozilla Open Leaders project to create an open science toolkit for astronomers: Resources for Open Science in Astronomy (ROSA). As part of my application for the program, I submitted a session proposal for Mozilla Festival 2017 relating to my project — I wanted to get input on what researchers would actually find useful in an open science toolkit. Continue reading “What resources do we need to break down barriers to open science? #MozFest 2017 session recap”

Tech giants are using open source frameworks to dominate the AI community


Tech giants such as Google and Baidu spent from $20 billion to $30 billion on AI last year, according to the recent McKinsey Global Institute Study. Out of this wealth, 90 percent fueled R&D and deployment, and 10 percent went toward AI acquisitions.

Research plays a crucial role in the AI movement, and tech giants have to do everything in their power to seem viable to the AI community. AI is mostly based on research advances and state-of-the-art technology, which is advancing very quickly. Therefore, there is no business need to make closed infrastructure solutions, because within a few months everything will be totally different.

In such a situation, the only winning strategy for tech giants is to offer open source solutions to attract members of the AI community and eventually become part of the AI community themselves. This is a relatively new model in the tech industry. Continue reading “Tech giants are using open source frameworks to dominate the AI community”

Democratizing Brainwaves over the Internet

Brainwaves to Browsers | An Open Project Spotlight

AJ Keller, CEO and founder of Push the World, is using hardware, software and firmware to push neuroscience forward. His company’s goal is to create a thought recognition headset that makes human-computer interaction effortless. To get closer to that goal, AJ has been working on a project called Brainwaves to Browsers in our current round of Mozilla Open Leaders. Continue reading “Democratizing Brainwaves over the Internet”

PLOS Channels Provide Opportunity for Discovery, Exploration and Contextual Insights

Last month, the new PLOS Cholera Channel joined existing Veterans Disability & Rehabilitation Research, Tuberculosis and Open Source Toolkit Channels in providing distinct and cohesive scholarly homes for research communities. These innovative forums increase the visibility of curated, quality research and reliable news and commentary, bridging a gap in relevance that contributes to public misunderstanding of research.

The Channels Program launched with Veterans Disability & Rehabilitation Research (VDRR), and as Veterans Day in the US approaches it’s an opportunity to take a moment to relay the channels origin story, highlight the latest content and re-introduce the editors behind this program. Continue reading “PLOS Channels Provide Opportunity for Discovery, Exploration and Contextual Insights”

Register: Better In App Billing Testing on Android

Illustration by Kevin Zweerink

Google Play Store’s In-app Billing API makes purchases and subscriptions easy. In just a few calls, your app can purchase items and subscriptions, verify previous purchases, etc. However, at The New York Times we ran into limitations while following Google’s recommended practices for testing our app’s use of the Billing API. Continue reading “Register: Better In App Billing Testing on Android”

Birkbeck Centre for Technology and Publishing releases scholarly communications platform, “Janeway”

Birkbeck’s Centre for Technology and Publishing is pleased to announce the open-source release of its software for academic publishing. “Janeway” is a press and journal system designed for open-access publishing that is free to download, use, and modify. Continue reading “Birkbeck Centre for Technology and Publishing releases scholarly communications platform, “Janeway””

Building a Cross Platform 360-degree Video Experience at The New York Times

By THIAGO PONTES and MAXWELL DA SILVA

Over the past few months, 360-degree videos have gained a lot of traction on the modern web as a new immersive storytelling medium. The New York Times has continuously aimed to bring readers as close to stories as possible. Last year we released the NYT VR app with premium content on iOS and Android. We believe VR storytelling allows for a deeper understanding of a place, a person, or an event.

This month, we added support for 360-degree videos into our core news products across web, mobile web, Android, and iOS platforms to deliver an additional immersive experience. Times journalists around the world are bringing you one new 360 video every day: a feature we call The Daily 360.

The current state of 360 videos on the Web

We’ve been using VHS, our New York Times Video Player, for playback of our content on both Web and Mobile Web platforms for the last few years. Building support for 360 videos on those platforms was a huge challenge. Even though the support for WebGL is relatively mature nowadays, there are still some issues and edge cases depending on platform and browser implementation.

To circumvent some of those issues, we had to implement a few different techniques. The first was the use of a “canvas-in-between”: We draw the video frames into a canvas and then use the canvas to create a texture. However, some versions of Microsoft Internet Explorer and Microsoft Edge are not able to draw content to the canvas if the content is delivered from different domains (as happens with a content delivery network, or CDN), even if you have the proper cross-origin resource sharing (CORS) headers set. We investigated this issue and found out that we could leverage the use of HTTP Live Streaming through the use of an external library called hls.js to avoid this problem.

Safari also has the same limitation regarding CORS. It seems to have been an issue in the underlying media framework for years and for this scenario, the hls.js workaround doesn’t solve the problem. We tackled this issue with the combination of two techniques:

  • The creation of an iframe with the video player embedded in it.
  • The use of progressive download renditions such as MP4 or WebM on the embedded player.

By doing this, we avoid the CORS video texture bug since the content and the iframe are in the same domain as the CDN and we were able to show the player in the parent domain, and the content inside the iframe.

Many of our users see our videos from within social media apps on their phones. On iOS, almost all of these social network applications load off-site content on their own in-app browsers instead of using the native browser, which raises a longstanding technical issue: the lack of support for inline playback video, even on iOS 10. This happens because inline playback support is still disabled by default on web views.

The technical problems listed above aside, the basic theory on how we should view 360 content is pretty straightforward. There are basically four steps to implement a simple 360 content view solution:

  1. Have an equirectangular panoramic image or video to be used as a source.
  2. Create a sphere and apply the equirectangular video or image as its texture.
  3. Create a camera and place it on the center of the sphere.
  4. Bind all the user interactions and device motion to control the camera.

These four steps could be implemented just using the WebGL API but there are 3D libraries like three.js that provide an easier way to use renderers for canvas, svg, CSS3D and WebGL. The example below shows how one could implement the four steps described above to render 360 videos or images:

CodePen Embed – sphere 360

When we first started to work on supporting 360 video playback on VHS, we researched a few projects and decided to use a JavaScript library called Kaleidoscope. Kaleidoscope supports equirectangular videos and images in all versions of modern browsers. The library is lightweight at 60kb gzipped, simple to use and easy to embed into the player when compared with other solutions.

The 360 video mobile native app experience on iOS and Android

Solving 360 video playback on iOS and Android was interesting and challenging since there wasn’t a video library that satisfied our requirements on both platforms. As a result, we decided to go with a different approach for each platform.

For the iOS core app, we created a small Objective-C framework that uses the same approach as Kaleidoscope. Initially we considered to start the development using Metal and OpenGL, but those are lower-level frameworks which require significant development work to create scenes and manipulate 3D objects.

Luckily, there’s another option: SceneKit is a higher-level framework that allows manipulation and rendering of 3D assets in native iOS apps. Investigation revealed that SceneKit provided adequate playback performance, so we chose to use it to render the sphere and camera required for 360-degree video playback.

We also needed to extract video frame buffers into a 2D texture to be applied as a material for the sphere, and to do that we decided to use SpriteKit. SpriteKit is a powerful 2D graphics framework commonly used in 2D iOS games. Our playback framework uses a standard iOS AVPlayer instance for video playback and uses SpriteKit to render its video onto the sphere.

Finally, we bind user interactions and device movements to control the camera’s motion using standard iOS gesture recognizers and device motion APIs.

By using these tools we were able to create a 360 video framework that is very similar to Kaleidoscope. We call it NYT360-Video, and we are happy to announce that we are open sourcing the framework.

On the Android platform we did a deep evaluation of some open source libraries that support 360 video and images, and after an initial prototyping, the Android team decided to use the Google VR SDK. The NYTimes Android app works on various devices and Android OS versions, and Google VR SDK has the features and capabilities that we needed and a straightforward API that allowed a relatively easy integration.

The Google VR SDK has evolved quite a lot from the day we started to work on the integration, and the Google VR team has invested a lot of time improving the project. Along the way, we worked together with Google on feature requests and bug fixes, and that collaboration gave us the certainty that we made the right decision to adopt it. The integration worked as we expected and now we have an immersive 360 video experience on Android.

The future of 360 video encoding and playback at The New York Times

We are investigating new ways of encoding and playing 360 videos to increase performance and improve the user experience. We are excited to explore other interesting features such as spatial audio, stereo images and video.

On the video transcoding side, we are exploring the use of cube map projections, which avoid the use of equirectangular layouts for a more space efficient approach. In theory, we can reduce the bitrate applied to the video by approximately 20 percent while keeping the same quality.

Below you can see a very basic example on how we could support playback of 360 videos encoded with cube map:

CodePen Embed – cubemap 360

The use of cube map projections is a more complex approach than using equirectangular projections since it would not only require changing our video player but also the way we transcode our videos. Earlier this year Facebook released a project called Transform, an FFmpeg filter that converts a 360 video in equirectangular projection into a cube map projection. We are investigating ways to integrate this into our video pipeline. We are also open sourcing the video encoding presets that we use to transcode all of our 360 video outputs.

We hope to receive your crucial feedback and generate contributions from the open source community at large. Feel free to ask questions via GitHub Issues in each project.

Check them out:
github.com/NYTimes/ios-360-videos
github.com/NYTimes/video-presets


Building a Cross Platform 360-degree Video Experience at The New York Times was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Using Microservices to Encode and Publish Videos at The New York Times

By FLAVIO RIBEIRO, FRANCISCO SOUZA, MAXWELL DA SILVA and THOMPSON MARZAGÃO

For the past 10 years, the video publishing lifecycle at The New York Times has relied on vendors and in-house hardware solutions. With our growing investment in video journalism over the past couple of years, we’ve found ourselves producing more video content every month, along with supporting new initiatives such as 360-degree video and Virtual Reality. This growth has created the need to migrate to a video publishing platform that could adapt to, and keep up with, the fast pace that our newsroom demands and the continued evolution of our production process. Along with this, we needed a system that could continuously scale in both capacity and features while not compromising on either quality or reliability.

A solution

At the beginning of this year, we created a group inside our video engineering team to implement a new solution for the ingesting, encoding, publishing and the syndication of our growing library of video content. The main goal of the team was to implement a job processing pipeline that was vendor agnostic and cloud-based, along with being highly efficient, elastic, and, of course, reliable. Another goal was to make the system as easy to use as possible, removing any hurdles that might get in the way of our video producers publishing their work and distributing it to our platforms and third-party partners. To do that, we decided to leverage the power of a microservices architecture combined with the benefits of the Go programming language. We named this team Media Factory.

The setup

The first version of our Media Factory encoding pipeline is being used in production by a select group of beta users at The New York Times, and we are actively working with other teams to fully integrate it within our media publishing system. The minimum viable product consists of these three different parts:

Acquisition: After clipping and editing the videos, our video producers, editors, and partners export a final, high-resolution asset usually in ProRes 442 format. Our producers then upload the asset to an AWS S3 bucket to get it ready for the transcoding process. We implemented two different upload approaches:

  1. An internal API that supports multipart uploads, called video-acquisition-api, used from server-side clients, like small scripts or jobs.
  2. A JavaScript wrapper that uses EvaporateJS to upload files directly from the browser, which is integrated with our internal Content Management System (CMS), Scoop.

Transcoding: After the acquisition step is complete, we use another microservice called video-transcoding-api to create multiple outputs based on the source file. Currently, we create a single HLS output with six resolutions and bitrates to support adaptive streaming, four different H.264/MP4 outputs, and one VP8/WebM for the benefit of the 1 percent of our users on the Mozilla Firefox browser running on Microsoft Windows XP.

The transcoding service is by far the most crucial part of our workflow. In order to integrate with cloud-based transcoding providers, we decided to design a tiny wrapper containing provider-specific logic. This design gives us great flexibility. We can schedule and trigger jobs based on a set of parameters such as speed, reliability, current availability, or even the price of the encoding operation for a specific provider. For instance, we can transcode news clips (which are time-sensitive) on the fastest, most expensive encoding service, while simultaneously transcoding live action videos, documentaries, and animations (which are not time-sensitive) using lower-cost providers.

Distribution: The transcoding step transfers the final renditions into another AWS S3 bucket. Since we use a content delivery network (CDN) to deliver the video to our end users, we need a final step to transfer the files from S3 to the CDN (leveraging Aspera’s FASP protocol to do so). Once the files are on the CDN, our video journalists are able to publish their content on The New York Times.

Giving back to the community

Today, we are open sourcing the video-transcoding-api and the video encoding presets that we use to generate all of our outputs. We are also open sourcing the encoding-wrapper, which contains a set of Go clients for the services we support and that are used by the video-transcoding-api.

We believe the format we’ve created will be of particular interest to the open source community. By leveraging the abstractions found in the video-transcoding-api, any developer can write the code necessary to send jobs to any transcoding provider we support without having to rewrite the base preset or the job specification. Sending a job to a different provider is as simple as changing a parameter.

We currently support three popular transcoding providers and plan to add support for more. See a sample preset below, in JSON format:

{
  "providers": ["encodingcom", "elementalconductor", "elastictranscoder"],
  "preset": {
    "name": "1080p_hls",
    "description": "1080p HLS",
    "container": "mp4",
    "profile": "Main",
    "profileLevel": "3.1",
    "rateControl": "VBR",
    "video": {
      "height": "1080",
      "width": "",
      "codec": "h264",
      "bitrate": "3700000",
      "gopSize": "90",
      "gopMode": "fixed",
      "interlaceMode": "progressive"
    },
    "audio": {
      "codec": "aac",
      "bitrate": "64000"
    }
  },
  "outputOptions": {
    "extension": "m3u8",
    "label": "1080p"
  }
}

Our philosophy for presets: “Write once, run anywhere”

Our future plans

In order to fulfill our vision of having a fully open sourced video encoding and distribution pipeline, we thought it best to also tackle the issue of actually encoding the video. We’re officially taking on the development and maintenance of the open source project Snickers to serve this purpose. We’ll not only gain the freedom of deploying our own encoding service anywhere, but we’ll also be able to experiment and implement new features that may not be available with existing service providers or and respond to specific requests from our newsroom. A few examples on the horizon are the automatic generation of thumbnails and accurate audio transcripts.

We’ve also turned our sights to fragmented MP4 (fMP4), and we’ll be investing some time into fully moving to an HLS-first approach for our on-demand videos. In case you missed it, last June at WWDC 2016, Apple introduced fMP4 to the HLS protocol, making it so now almost all devices and browsers support fMP4 playback natively. This means we can now eliminate the overhead of having to transmux the MPEG-TS segments into fMP4 on the fly when playing videos using our video player (we use hls.js to do this) and instead just concatenate and play fMP4 fragments on our local buffer.

Lastly, content-driven encoding is a trendy topic within the online video community, especially after the release of VMAF. We are planning to adopt this approach by splitting the content-driven encoding project into two phases:

  1. Classify our content into four different categories, each with its own preset. For example, animation videos, like the ones we have for our Modern Love show, require fewer bits than our high-motion videos, like some of our Times Documentaries, to achieve the same content fidelity.
  2. Create and integrate an additional microservice within the Media Factory pipeline for the purpose of checking the quality of our outputs using VMAF and triggering new re-encoding jobs with optimized presets.

Come along with us!

Our Media Factory team (Maxwell Dayvson da Silva, Said Ketchman, Thompson Marzagão, Flavio Ribeiro, Francisco Souza and Veronica Yurovsky) believes that these projects will help address the encoding challenges faced by many of you in the online video industry. We hope to receive your crucial feedback and generate contributions from the open source community at large.

Check them out:

https://github.com/NYTimes/video-transcoding-api
https://github.com/NYTimes/video-presets
https://github.com/NYTimes/encoding-wrapper
https://github.com/snickers/snickers

And feel free to ask questions via GitHub Issues in each of the projects!


Using Microservices to Encode and Publish Videos at The New York Times was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Continuous Deployment to Google Cloud Platform with Drone

By TONY LI and JP ROBINSON

Over the course of the last year, the software development teams at The New York Times have been evaluating Google Cloud Platform for use in some of our future projects. To do this, we’ve surveyed a wide variety of software management techniques and tools, and we’ve explored how we might standardize building, testing and deploying our systems on GCP.

Our newly formed Delivery and Site Reliability Engineering team came up with two methods of deployment with Google Container Engine and Google App Engine for computing environments and the open source implementation of Drone as a continuous integration tool. As a result of this work, we are open sourcing two plugins for Drone: drone-gke and drone-gae respectively.

Container Engine is Google’s managed Kubernetes container orchestration platform. Kubernetes is an open source project that provides a declarative approach to managing containerized applications, enabling automatic scaling and healing properties. It encourages a common standard of how our applications are designed, deployed, and maintained across many independent teams. And because Kubernetes pools compute resources, developers can run many isolated applications in the same cluster, maximizing its resource usage density.

App Engine is a mature serverless platform Google has offered since 2008. It is capable of quickly scaling up and down as traffic changes, which is ideal for many scenarios at The New York Times when you consider possible sudden spikes from breaking news alerts or the publication of the new daily crossword every weekday at 10 p.m.

Drone is an open source continuous integration and delivery platform based on container technology, encapsulating the environment and functionalities for each step of a build pipeline inside an ephemeral container. Its flexible yet standardized nature enables our teams to unify on a plugin-extensible, ready-to-use CI/CD pipeline that supports any custom build environment with isolation, all with a declarative configuration similar to commercial CI/CD services. The result is the ability for developers to confidently ship their features and bug fixes into production within minutes, versus daily or weekly scheduled deployments. As a containerized Go application, it is easy to run and manage, and we hope to contribute to the core open source project.

Google provides an excellent set of command line utilities that allow developers to easily interact with Google Container Engine and Google App Engine, but we needed a way to encapsulate those tools inside of Drone to simplify the workflow for developers. Luckily, plugins for Drone are simple to create as they can be written in Go and are easily encapsulated and shared in the form of a Docker container. With that in mind, the task of creating a couple reusable plugins was not that daunting.

drone-gke is our new plugin to wrap the gcloud and kubectl commands and allow users to orchestrate deployments to Google Container Engine and apply changes to existing clusters. The Kubernetes yaml configuration file can be templated before being applied to the Kubernetes master, allowing integration with Drone’s ability to encrypt secrets and injecting build-specific variables.

Here is an example Drone configuration to launch a Go application in a container via a Kubernetes Deployment resource into Google Container Engine:

# test and build our binary
build:
  image: golang:1.7
environment:
    - GOPATH=/drone
commands:
    - go get -t
    - go test -v -cover
    - CGO_ENABLED=0 go build -v -a
when:
    event:
      - push
      - pull_request
# build and push our container to GCR
publish:
  gcr:
    storage_driver: overlay
    repo: my-gke-project/my-app
    tag: "$$COMMIT"
    token: >
      $$GOOGLE_CREDENTIALS
when:
      event: push
      branch: master
# create and apply the Kubernetes configuration to GKE
deploy:
  gke:
    image: nytimes/drone-gke
zone: us-central1-a
    cluster: my-k8s-cluster
    namespace: $$BRANCH
    token: >
      $$GOOGLE_CREDENTIALS
vars:
      image: gcr.io/my-gke-project/my-app:$$COMMIT
      app: my-app
      env: dev
    secrets:
      api_token: $$API_TOKEN
when:
      event: push
      branch: master
And the corresponding Kubernetes configuration:
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: {{.app}}-{{.env}}
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: {{.app}}
        env: {{.env}}
    spec:
      containers:
        - name: app
          image: {{.image}}
          ports:
            - containerPort: 8000
          env:
            - name: APP_NAME
              value: {{.app}}
            - name: API_TOKEN
              valueFrom:
                secretKeyRef:
                  name: secrets
                  key: api-token
---
kind: Service
apiVersion: v1
metadata:
  name: {{.app}}-{{.env}}
spec:
  type: LoadBalancer
  selector:
    app: {{.app}}
    env: {{.env}}
  ports:
    - port: 80
      targetPort: 8000
      protocol: TCP
And the corresponding Kubernetes secrets configuration:
kind: Secret
apiVersion: v1
metadata:
  name: secrets
type: Opaque
data:
  api-token: {{.api_token}}
drone-gae is our new plugin to wrap the gcloud and appcfg commands and allow users to make deployments to Google App Engine standard environment with Go, PHP or Python or to the flexible environments with any language.
Here’s a very basic example of all the configuration required to launch a new version of a Go service to Google App Engine’s standard environment with a second step to migrate traffic to that version:
deploy:
  # deploy new version to GAE
  gae:
    image: nytimes/drone-gae
environment:
      - GOPATH=/drone
action: update
    project: my-gae-project
    version: "$$COMMIT"
    token: >
      $$GOOGLE_CREDENTIALS
when:
      event: push
      branch: master
# set new version to 'default', which migrates 100% traffic
  gae:
    image: nytimes/drone-gae
action: set_default_version
    project: my-gae-project
    version: "$$COMMIT"
    token: >
      $$GOOGLE_CREDENTIALS
when:
      event: push
      branch: master
Deploying new versions to the flexible environment requires a little more work, but it’s straightforward when using the plugin. We first use a build step to test and compile the code, then a publish step to build and publish a Docker container to Google Container Registry (via the drone-gcr plugin) and finally, we kick off the deployment via our new plugin.
# test and build our binary
build:
  image: your-dev/golang:1.7
environment:
    - GOPATH=/drone
commands:
    - go test -v -race ./...
    - go build -o api .
when:
    event:
      - push
      - pull_request
# build and push our container to GCR
publish:
  gcr:
    storage_driver: overlay
    repo: my-gae-project/api
    tag: "$$COMMIT"
    token: >
       $$GOOGLE_CREDENTIALS
when:
      branch: [develop, master]
      event: push
deploy:
  # deploy a new version using the docker image we just published and stop any previous versions when complete.
  gae:
    image: nytimes/drone-gae
action: deploy
    project: my-gae-project
    flex_image: gcr.io/my-gae-project/api:$$COMMIT
    version: "$${COMMIT:0:10}"
    addl_flags:
     - --stop-previous-version
    token: >
      $$GOOGLE_CREDENTIALS
when:
      event: push
      branch: develop
We hope open sourcing these tools helps other engineers who want to leverage Drone as a continuous delivery solution. We are also looking to the community to take a look and help us harden our systems. Please raise an issue if you find any problems and follow the contributing guidelines if you make a pull request. For further reading and more documentation, you can visit the code repositories on Github:
github.com/NYTimes/drone-gae
github.com/NYTimes/drone-gke


Continuous Deployment to Google Cloud Platform with Drone was originally published in Times Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

At a Glance – Open Innovation

An open approach leading to opportunity

Open innovation is an approach to business strategy in which companies look for new technology and ideas outside of the firm itself. The term was coined by Henry Chesbrough, the author of a book published in 2003 entitled, ‘Open Innovation: The New Imperative for Creating and Profiting from Technology’. In the book, Chesbrough explains that open innovation has replaced closed innovation, which relies entirely on internal research and ideas. Open innovation, on the other hand, encourages co-operation with suppliers, competitors and other parties to improve customer value.

Collaboration has emerged as a key trend in business strategy. Companies are willing to share their intellectual property (IP) if it leads to an improved service or product. In the automotive industry, for example, automakers have partnered up with software companies to enhance their autonomous technology. Businesses across all industries have also teamed up with academic institutions to use their expertise. Open-source software is another demonstration of the changing attitude to innovation, allowing external developers to work on and presumably improve systems.

Open innovation has been enabled by the growing accessibility of information, and the availability of venture capital funding. Knowledge is no longer confined to labs, and companies can gain investment for their projects without needing the extensive resources of a major business. Instead of jealously guarding IP, it is traded as an important tool. Businesses can benefit from open innovation in a number of ways. Firstly, it saves time, effort and money. It also widens the scope for potential development. Companies don’t necessarily have to come up with the initial idea to benefit from it, and can profit from the work of other parties. This approach is especially useful when negotiating the rise of disruptive technology, as multiple parties need to follow the same standards and regulations to allow for responsible adoption.

The post At a Glance – Open Innovation appeared first on Disruption.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑