|
|
|
|
“Every minute your developers aren’t working on business logic is wasted time.”
___
Lyft’s Matt Klein, QCon New York.
|
|
|
Apache Kafka itself is not changing, but rather the typical use cases are. Confluent’s third annual Apache Kafka Report surveyed over 600 users of the technology, which is a 71 percent increase compared to last year’s sample. What appears to be a broadening user community has a better idea about Kafka’s business value. A more nuanced definition of the technology and increased adoption of microservices results in a different outlook on Kafka’s association with the term “stream processing.”
Data pipelines and messaging are now the top two uses of Kafka, followed by microservices/event processing and stream processing. The addition of several answer choices partially explains significant drops in the use of Kafka for stream processing (66 percent in 2017 to 48 percent in 2018) and data integration (60 percent to 46 percent). The chart above shows that people distinguish between the use of Kafka capabilities for event processing and streaming ETL, resulting in the broad stream processing category getting fewer responses. Another explanation for the changes is that is that Kafka is no longer viewed just as a backend for stream processing in competition with Apache Spark and Storm, and connecting with Hadoop. Thus, Spark may be used for real-time analysis while Kafka is the source for real-time data.
|
|
|
At ChefConf 2018 in Chicago in late May, we sat down with Chef CTO and co-founder Adam Jacob, and Senior VP of products and engineering Corey Scobie, to discuss the cultural currents in the enterprise. Specifically, Jacob focused on the proliferation of DevOps and DevOps tools within the business community. One of the things that is potentially a roadblock for uptake of DevOps in the enterprise has been the internal culture of many organizations. With so many new ideas and tools out there, it can feel like your company doesn’t get it at all, and even perhaps fears DevOps, he explained.
|
|
Developers as the Weak Link in the Security Chain
Behind closed doors, IT professionals sometimes like to mock their users as being the source of many security breaches, through the indiscriminate clicking on suspicious emails or badly managed passwords. But it turns out their coworkers in development down the hall cause just as many issues.
This was the takeaway from the QCon New York 2018 keynote this week by Snyk security’s Guy Podjarny. He talked about all the ways bad development practices have led to huge security holes.
For instance, Uber has suffered two major breaches (2014, and then in 2017) stemming from its use of GitHub, where developers had mistakenly left passwords in code, which were then found and exploited by attackers. After 2017, Uber moved its code to a private repository. In other cases, attackers duped unsuspecting developers into using malicious code, by slipping it into unverified copies of the common dev tools, such as Delphi, and later into Apple’s XCode. As a result, programs written with this corrupted software led to serious compromises in user applications generated by these IDEs. Elsewhere, developers have been too trusting — assuming that the code on GitHub is the same that they download in npm, or from the Docker registry.
Podjarny is a gifted speaker and he dropped many more fascinating details about how developers have been duped into propagating malware. But he also offered some positive tips for mitigation. For one, he examined how the tech giants are handling this issue. One common thing they all do is limit the permissions that each employer has to only what they need. For instance, Google’s BeyondCorp works as a central authentication proxy that checks all employee actions. Microsoft’s Privileged Access Workstations maintains users in a trusted virtual machine. And Netflix uses secure socket shells in a setup called Bless.
Podjarny’s talk was only one of a number of interesting ones at QCon New York this year. Check back on the site within the next few days for more stories from QCon.
|
|
|
SoundHound has expanded beyond its search service to provide a generalized voice interface that the company claims is superior at handling complex queries and follow-up questions. It’s working with a range of big-name enterprises, including 11 automakers, to create custom voice experiences — and wants to help developers add voice components to their own applications.
|
|
|
The newest version of the open source Kubernetes container orchestration engine — version 1.11 — includes a number of significant changes, including new capabilities for custom resource definitions (CRDs). CRDs allow users to extend Kubernetes with custom APIs to their own services, such as for virtual machines. In version 1.11, CRDs received support for end-point monitoring, and the ability to version CRD resources.
|
|
|
With a few adjustments to the technology, Docker containers could bring heretofore unseen efficiencies to supercomputers and high-performance computing (HPC), so predicts Docker Technical Account Manager Christian Kniep, in a series of recent talks at ContainerDays Hamburg, and elsewhere.
|
|
|
Apache Spark is an in-memory data analytics engine. It is wildly popular with data scientists because of its speed, scalability and ease-of-use. Many Pivotal customers want to use Spark as part of their modern architecture, so Pivotal shares some of its experiences working with the tool, in this contributed post.
|
|
|
|
Facebook software engineers Joe O’Neill (left) and Haozhe Gao discussed at QCon how they keep their microservices snappy at the social media giant.
|
|
|
Snyk's Guy Podjarny, fresh from his security keynote at QCon New York.
|
|
|
|
Lyft’s Matt Klein talked at QCon about the possibility of the Envoy service mesh supporting real-time streaming services such as Kafka in the near future.
|
|
|
JULY 16-18, 2018 // SAN FRANCISCO
WSO2Con
Organizations around the world are recognizing the need for digital transformation to compete and thrive. A clear digital strategy is important to drive digital maturity within the enterprise. Join the WSO2 Summit to learn and engage in an interactive discussion on how you can create your digital transformation strategy and put it into practice. 50% off with code WCUTNS50. Register Now!
|
|
|
Kubernetes helps accelerate software delivery in much the same way containers improve the delivery process. While the benefits of containers in the DevOps, continuous integration, and continuous delivery pipelines will be familiar, many developers and DevOps teams are still figuring out how to best implement Kubernetes. In this ebook, we’ll explore use cases and best practices for how Kubernetes helps facilitate continuous integration and continuous delivery.
|
|
|
|
We are grateful for the support of our ebook foundation sponsor:
|
|
|
|
|
|
|
|
|
