Amazon is partnering with Netflix and Expedia as it fires back at the startups that accuse it of unfairly profiting from their open source code (AMZN, ESTC)

Amazon Adrian Cockcroft

Amazon Web Services announced on Monday that it will partner with Expedia Group and Netflix to launch Open Distro for Elasticsearch, an open source software project.
Open Distro for Elasticsearch is based on the open source search software originally created by software company Elastic.
AWS says that it is doing this to invest in open source, even as it combats a perception in the open source community that Amazon contributes little code to open source projects compared to its peer companies.
AWS says it plans to contribute its new features to Open Distro for Elasticsearch and the main Elasticsearch project, and some of these features are part of Elastic's paid Elasticsearch product for enterprises.

Amazon is trying to fight the perception that it doesn't give back to open source software — but it might also be posing a new threat to Elastic, an open source software company.

Since 2015, Amazon Web Services has been selling Elasticsearch, an open source software project originally created by $6 billion company Elastic, as a service to software developers. Elasticsearch is popular on its own merits, and used by apps like Uber and Tinder to store, search, and analyze large amounts of data.

On Monday, AWS, in partnership with Expedia and Netflix, announced Open Distro for Elasticsearch, a version of the software that Amazon will support with new features and updates, but that is also available as free open source — a move that AWS says is also intended to underscore its commitment to open source software.

In open source parlance, a distribution, or "distro," is a customized version of open source software, hence the name.

Notably, the first release of Open Distro for Elasticsearch will have some features that Elastic, the company, had only made available to paying customers of its own premium version of Elasticsearch. Amazon says that it warned Elastic of its plans, but that it decided to stay the course. Elastic was not immediately available for comment.

Amazon under fire

The move comes as Amazon is scrutinized for its relationship with open source software, as smaller companies like Redis Labs, Confluent, and MongoDB have all taken dramatic steps in changing their software licenses to stop AWS and other big cloud providers from taking and selling its open source software as a service. Those new licenses have attracted criticism, in turn, for what some percieve as undermining the foundations of open source.

In the blog entry announcing Open Distro for Elasticsearch, AWS VP of Cloud Architecture Strategy Adrian Cockcroft warns that those efforts are "muddying the waters" in open source, making it unclear what's considered open source and what might be considered proprietary intellectual property.

Indeed, that's why AWS decided to launch Open Distro for Elasticsearch, as a way to circumvent what he says is an "intermingling" of proprietary and open source code in the original Elasticsearch project — something that could get Amazon, or its customers, into trouble, in his estimation.

"This is hard to track and govern, could lead to breach of license, and could lead to immediate termination of rights," Cockcroft wrote. "Individual code commits also increasingly contain both open source and proprietary code, making it very difficult for developers who want to only work on open source to contribute and participate."

Cockcroft said that AWS discussed its concerns about Elasticsearch with Elastic.

"We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch," Cockcroft wrote. "They have made it clear that they intend to continue on their current path."

Amazon's contribution to open source

On the subject of Amazon's commitment to open source, Cockcroft wrote that AWS contributes to projects like Apache Lucene, Hadoop (which started at Yahoo) and Kubernetes (which started at Google), and that the company invests in open source communities by training developers and sponsoring events.

However, there's a perception among some in Silicon Valley that the company doesn't support open source. According to a data analysis by Google developer advocate Felipe Hoffa, AWS lags behind Microsoft and Google in contributing to open source projects, although it increased its contributions significantly in 2018.

AWS announced a major open source project of its own back in November; and then another one, focused on artificial intelligence, in January. These were met with some surprise, as Amazon doesn't have a reputation for contributing these kinds of major projects to the community. Cockroft says that the pace will only continue.

"Over the years, customer usage and dependencies on open source technologies have been steadily increasing; this is why we’ve long been committed to open source, and our pace of contributions to open source projects – both our own and others’ – continues to accelerate," Cockcroft wrote.

Forking the code

With Open Distro for Elasticsearch, it seems like AWS is forking the code — essentially copying the original Elasticsearch code and remixing it to make a distinct piece of software. However, Cockcroft says that Amazon doesn't intend to totally upstage the original Elasticsearch, and AWS will still contribute back to that project.

"Our intention is not to fork Elasticsearch, and we will be making contributions back to the Apache 2.0-licensed Elasticsearch upstream project as we develop add-on enhancements to the base open source software," Cockcroft wrote.

Cockcroft said that above all else, it's important for open source projects to not pull the rug out from under users by changing their terms dramatically, or by privileging one company over another.

"If we look closely at many successful open source projects, they have all benefited from access to unfettered open source software. In fact, arguably those projects would not exist today without an ability to quickly assemble and innovate on top of pre-existing open source software," Cockcroft wrote.

This isn't the first time AWS has pulled a move like this. For example, when Oracle announced it would stop providing free public updates for Java unless users buy a subscription, AWS started offering Corretto, its own free distribution of Java, for which AWS has committed to providing security updates.

"When important open source projects that AWS and our customers depend on begin restricting access, changing licensing terms, or intermingling open source and proprietary software, we will invest to sustain the open source project and community," Cockcroft wrote.

Join the conversation about this story »

NOW WATCH: Everything we know about Samsung’s foldable phone