Language Selection

English French German Italian Portuguese Spanish

March 2019

Apache: Apark, ASF (Apache Software Foundation) and Apache Kafka vs. Apache Pulsar

Filed under
  • Apache Spark Turns 10: The Secret Sauce Behind One Of The World’s Most Popular Open Source Projects

    It was the changing nature of big data technology and architectural models, that wrote the story for Hadoop. The infrastructure architecture moved towards edge computing, IoT and cloud computing and especially containers where the market is seeing an increase in Kuberenetes workload. With analytical and machine learning workloads increasing, there was an increased need for a unified analytics platform. And that’s exactly how Spark outperformed Hadoop in metrics such as In memory processing vs disk, real-time streaming and batch streaming besides providing a layer for integrating machine learning as well.

    As Apache Spark turned 10 years old, let’s see the strong driver that led to Spark adoption and what keeps it going. Dubbed as the official “in-memory replacement for MapReduce”, the disk-based computational engine is at the heart of early Hadoop clusters. Why Spark took off was because it reflects the changing processing paradigm to a more memory intensive pipeline, so if your cluster has a decent memory and an API simpler than MapReduce, processing in Spark will be faster. The reason why Spark is faster is because most of the operations (including reads) decrease in processing time roughly linearly with the number of machines since it’s all distributed.

  • The Apache Software Foundation Celebrates 20 Years of Community-led Development “The Apache Way”
  • The Apache® Software Foundation Celebrates 20 Years of Community-led Development "The Apache Way"

    World's largest Open Source foundation provides $20B+ worth of software for the public good at 100% no cost...

  • 20 milestones at the Apache Software Foundation

    Not at all a question of parts unknown, more a case of parts where some are better known than others.

    The Apache Software Foundation (ASF)’s Jim Jagielski and Sally Khudairi have detailed 20 major milestones that exist under the auspicious auspices of the ASF today.

    Without detailing every project (and the ASF holds stewardship over 350 community-led projects and initiatives) and repeating the entire story linked above… we’ll tour a handful in celebration of the fact that the ASF passed its 20th Anniversary on 26 March 2019.

    It would be tough not to mention Apache HTTP Server. This is most popular open source HTTP server on the planet — it provides a secure and extensible server that provides HTTP services observing the latest HTTP standards.

  • Pub/sub messaging: Apache Kafka vs. Apache Pulsar

    These days, massively scalable pub/sub messaging is virtually synonymous with Apache Kafka. Apache Kafka continues to be the rock-solid, open-source, go-to choice for distributed streaming applications, whether you’re adding something like Apache Storm or Apache Spark for processing or using the processing tools provided by Apache Kafka itself. But Kafka isn’t the only game in town.

    Developed by Yahoo and now an Apache Software Foundation project, Apache Pulsar is going for the crown of messaging that Apache Kafka has worn for many years. Apache Pulsar offers the potential of faster throughput and lower latency than Apache Kafka in many situations, along with a compatible API that allows developers to switch from Kafka to Pulsar with relative ease.

Complete Guide for using ffmpeg in Linux

Filed under

Curious about using ffmpeg in Linux? Regular user wanting to learn more? Check out our comprehensive guide covering the most versatile media editing utility.
Read more

The low-cost hardware revolution

Filed under

The inventiveness of the maker generation is finding its way into industry with cost-effective hardware, open source platforms and fresh ideas.

The wave of low-cost hardware success is unstoppable. Anyone thinking of low-cost hardware has developer boards front of mind: Raspberry Pi, Arduino or Beagle Bone. Since its launch in 2012, the Raspberry Pi has an amazing success story. With over 17 million devices sold worldwide, Raspberry Pi is the most popular single-board computer of all time. This mini-PC is the initiator of the low-cost trend.

But the Raspberry Pi has now found itself in a competitive market place as more developer boards try to emulate its success. From new one-board controls to accessories and extensions, the market is constantly seeing innovation. Shields, hats, power supplies and sensors – now a whole hardware ecosystem is available with the minicomputers.

Read more

OpenMandriva Appears To Be Experimenting With Profile Guided Optimizations

Filed under

OpenMandriva has been toying with some performance optimizations in recent times like preferring the LLVM Clang compiler over GCC, spinning an AMD Zen "znver1" optimized version of the OS/packages, and apparently now exploring possible Profile Guided Optimizations.

Profile Guided Optimizations (PGO) basically involve feeding the feedback of profiling data back into the compiler so it can better optimize the generated code based upon actual usage behavior of the software under test. PGO can pay off big time depending upon the code-base and how well the profile data models real-world workflows of the said software in question.

Read more

IBM's Red Hat buy 'shows future importance of open source'

Filed under
Red Hat

Ahead of the company's annual conference, SUSECON, which will be held in Nashville, Tennessee, from 1 to 5 April, Andy Jiang, SUSE vice-president and general manager Asia Pacific and Japan, told iTWire in response to queries that SUSE, now an independent business, was in a good position to work closely with its partners and customers to ensure that they could benefit from the "freedom and flexibility" of enterprise open source solutions.

Asked whether SUSE had now decided to adopt a more aggressive attitude towards the competition — as exemplified in a blog post made recently by Ryan Hagen, consulting manager, Global SUSE Services, about loud infrastructure and business mobility vendor VMware — Jiang did not give a direct answer, but said...

Read more

How to build a mobile particulate matter sensor with a Raspberry Pi

Filed under

About a year ago, I wrote about measuring air quality using a Raspberry Pi and a cheap sensor. We've been using this project in our school and privately for a few years now. However, it has one disadvantage: It is not portable because it depends on a WLAN network or a wired network connection to work. You can't even access the sensor's measurements if the Raspberry Pi and the smartphone or computer are not on the same network.

To overcome this limitation, we added a small screen to the Raspberry Pi so we can read the values directly from the device. Here's how we set up and configured a screen for our mobile fine particulate matter sensor.

Read more

today's leftovers

Filed under
  • Destination Linux EP114 - Ryan Fills Your Brains

    On DL114 - Ryan Interviewed, Solus 4, Mate 1.22, Nvidia buys Mellanox, Jetson Nano, Firefox 66, Openshot, OpenXR, Google Stadia, Linux Gaming News plus our Tips, Tricks and Software Spotlight picks!.

  • Arm's Komeda DRM Driver Picking Up Support For The Mali D71

    With the Linux 5.1 kernel there is Arm's new "Komeda" direct rendering manager driver while patched in as new material for Linux 5.2 is support for the Mali D71 display processor with this new driver.

  • De Blob Guide

    I.N.K.T Corporation has taken over Chroma City and removed all color. Nobody goes outside any longer and its up to one blob to save the day. Enter de Blob! Smash I.N.K.T bots to acquire their color and pain-the-streets color again! Use de Blobs abilities to revive Chroma City by mixing colors, completing objectives, freepaint modes and even 4-player split screen modes.

  • Apache Software Foundation's 20th anniversary, 3D-print system for optical cardiography, and more news

    An international research team has developed a multiparametric visual mapping technique that can simultaneously monitor multiple factors affecting heart health while creating 3D models. The method was developed to better understand cardiac arrhythmias. This open source, expandable system is openly available and can potentially save other researchers up to $20,000.

  • Open source is free – yeah, right! [Ed: It's 2019 and this buffoon Martin Banks still doesn't know the difference between freedom and priceless]
  • First Election of the .NET Foundation [Ed: Sellout, backstabber and "traitor" (according to Richard Stallman) helps Microsoft entryism and openwashing still]

    I am stepping down very happy knowing that I achieved my main goal, to turn the .NET Foundation into a more diverse and member-driven foundation.

Servers: IBM, GM, UNIX and Ampere Computing

Filed under
  • Surrounded by inspiration, Appalachian alumnus keeps applications running at IBM

    As a site reliability engineer at IBM, Appalachian State University alumnus Chris Waldon’s favorite aspect of his job is working with great people who inspire him. “I’m surrounded by brilliant software engineers, and I get to learn from them daily,” he said.

    It is not surprising that Waldon ’16 ’18 still has a thirst for learning. He graduated summa cum laude from Appalachian with a Bachelor of Science in computer science in 2016, was an Honors College student as well as a Chancellor’s Scholar, and earned his Master of Science in computer science in 2018.


    Outside of class, Waldon started Linux@App, a club for students interested in using the Linux operating system — an alternative to macOS or Windows. “You can do a lot with Linux that isn’t possible on other systems,” Waldon explained. “Since most of the internet and all of the systems I use at IBM run on Linux, learning and teaching about it at Appalachian helped me develop the practical skill I now use daily.”

  • How GM's Cruise Autonomous Vehicle Effort Is Improving Kubernetes

    Having the right access control in place for authorized systems and individuals is a critical part of any modern computing platform. When Cruise Automation didn't quite get the all the capabilities it needed from within the open-source Kubernetes project, it went out and built its own open-source project to fill the gap.

  • What you need may be “pipeline +Unix commands” only

    The IT field never lacks “new” technologies: cloud computing, big data, high concurrency, etc. However, the thinkings behind these “fancy” words may date back to the era when Unix arose. Unix command line tools are invaluable treasure. In many cases, picking the right components and using pipeline to glue them can satisfy your requirement perfectly. So spending some time in reviewing Unixcommand line manual instead of chasing state-of-the-art techniques exhaustedly, you may gain more.

  • Ampere Computing + Packet Roll Out eMAG To The Public Cloud - 32 Cores For $1 Per Hour

    Ampere Computing and Packet announced on Thursday that eMAG servers will now be available through this public cloud/server provider. The initial configuration allows for 32 Arm cores at 3.3GHz and 128GB of RAM and 480GB of SSD storage for just $1 USD per hour on-demand access. I have run some initial benchmarks from this new compute instance for those interested. 

More in Tux Machines

Simon Steinbeiß of Xfce, Dalton Durst of UBports, KDE Apps 19.08, Huawei – Destination Linux 135

Simon Steinbeiß of Xfce, Dalton Durst of UBports, KDE Applications, CutiePi Open Source Tablet, Huawei To Create Open Source Foundation, Rust Removes Linux Support, Stranded Deep Survival Game Fix Read more

KDE Usability & Productivity: Week 85

I’m not dead yet! KDE’s new goal proposals have been announced, and the voting has started. But in the meantime, the Usability & Productivity initiative continues, and we’re onto week 85! Read more

Leftovers: Kate, Krita, UCLA Library and RcppExamples

  • Kate - Document Preview Plugin - Maintainer Wanted!

    At the moment the Document Preview plugin that e.g. allows to preview Markdown or other documents layout-ed via embedding a matching KPart is no longer maintained. If you want to step up and keep that plugin alive and kicking, now is your chance!

  • The Sprint

    Hi -)) haven’t posted for some time, because I was busy travelling and coding for the first half of the month. From Aug 5 to Aug 9, I went to the Krita Sprint in Deventer, Netherlands. According to Boud, I was the first person to arrive. My flight took a transit via Hong Kong where some flights were affected due to natural and social factors, but fortunately mine was not one of them. Upon arrival in Amsterdam I got a ticket for the Intercity to Deventer. Railway constructions made me take a transfer via Utrecht Centraal, but that was not a problem at all: the station has escalators going both up to the hall, and down to the platforms (in China you can only go to the hall by stairs or elevator (which is often crowded after you get off)). When I got out of Deventer Station, Boud immediately recognized me (how?!). It was early in the morning, and the street’s quietness was broken by the sound of me dragging my suitcase. Boud led me through Deventer’s crooked streets and alleys to his house. For the next two days people gradually arrived. I met my main mentor Dmitry (magician!) and his tiger, Sagoskatt, which I (and many others) have mistaken for a giraffe. He was even the voice actor for Sago. He had got quite a lot of insights into the code base (according to Boud, “80%”) and solved a number of bugs in Krita (but he said he introduced a lot of bugs, ha!). Also I met David Revoy (my favourite painter!), the author of Pepper and Carrot. And Tiar, our developer who started to work full-time on Krita this year; she had always been volunteering to support other Krita users and always on the IRC and Reddit. And two of other three GSoC students for the year: Blackbeard (just as his face) and Hellozee. Sh_zam could not come and lost communications due to political issues, which was really unfortunate (eh at least now he can be connected). It is feels so good to be able to see so many people in the community – they are so nice! And it is such an experience to hack in a basement church.

  • How UCLA Library preserves rare objects with open source

    The University of California, Los Angeles, (UCLA) Library houses a collection of millions of rare and unique objects, including materials dating from 3000 BCE, that could be damaged, destroyed, or otherwise threatened if they were displayed. To make these special collections widely available while keeping them secure, the UCLA Library has been modernizing its digital repository, which was established 15 years ago on now-outdated software. [...] Watch Jen's Lightning Talk to learn more about the UCLA Library's rare collections digitization project.

  • RcppExamples 0.1.9

    The RcppExamples package provides a handful of short examples detailing by concrete working examples how to set up basic R data structures in C++. It also provides a simple example for packaging with Rcpp.

Games: Smith and Winston, 7 Billion Humans Sale