Language Selection

English French German Italian Portuguese Spanish

Development

Python: Security and NumPy 1.20 Release

Filed under
Development
  • Python Package Index nukes 3,653 malicious libraries uploaded soon after security shortcoming highlighted

    The Python Package Index, also known as PyPI, has removed 3,653 malicious packages uploaded days after a security weakness in the use of private and public registries was highlighted.

    Python developers use PyPI to add software libraries written by other developers in their own projects. Other programming languages implement similar package management systems, all of which demand some level of trust. Developers are often advised to review any code they import from an external library though that advice isn't always followed.

    Package management systems like npm, PyPI, and RubyGems have all had to remove subverted packages in recent years. Malware authors have found that if they can get their code included in popular libraries or applications, they get free distribution and trust they haven't earned.

    Last month, security researcher Alex Birsan demonstrated how easy it is to take advantage of these systems through a form of typosquatting that exploited the interplay between public and private package registries.

  • A pair of Python vulnerabilities [LWN.net]

    Two separate vulnerabilities led to the fast-tracked release of Python 3.9.2 and 3.8.8 on February 19, though source-only releases of 3.7.10 and 3.6.13 came a few days earlier. The vulnerabilities may be problematic for some Python users and workloads; one could potentially lead to remote code execution. The other is, arguably, not exactly a flaw in the Python standard library—it simply also follows an older standard—but it can lead to web cache poisoning attacks.

    [...]

    [Update: As pointed out in an email from Moritz Muehlenhoff, Python 2.7 actually is affected by this bug. He notes that python2 on Debian 10 ("Buster") is affected and has been updated. Also, Fedora has a fix in progress for its python2.7 package.]

  • NumPy 1.20 has been released

    NumPy is a Python library that adds an array data type to the language, along with providing operators appropriate to working on arrays and matrices. By wrapping fast Fortran and C numerical routines, NumPy allows Python programmers to write performant code in what is normally a relatively slow language. NumPy 1.20.0 was announced on January 30, in what its developers describe as the largest release in the history of the project. That makes for a good opportunity to show a little bit about what NumPy is, how to use it, and to describe what's new in the release.

    [...]

    NumPy adds a new data type to Python: the multidimensional ndarray. This a container, like a Python list, but with some crucial differences. A NumPy array is usually homogeneous; while the elements of a list can be of various types, an ndarray will, typically, only contain a single, simple type, such as integers, strings, or floats. However, these arrays can instead contain arbitrary Python objects (i.e. descendants of object). This means that the elements will, for simple data types, all occupy the same amount of space in memory. The elements of an ndarray are laid out contiguously in memory, whereas there is no such guarantee for a list. In this way, they are similar to Fortran arrays. These properties of NumPy arrays are essential for efficiency because the location of each element can be directly calculated.

    Beyond just adding efficient arrays, NumPy also overloads arithmetic operators to act element-wise on the arrays. This allows the Python programmer to express computations concisely, operating on arrays as units, in many cases avoiding the need to use loops. This does not turn Python into a full-blown array language such as APL, but adds to it a syntax similar to that incorporated into Fortran 90 for array operations.

Moving commits between independent git histories

Filed under
Development
Gentoo

PyPy is an alternative Python implementation. While it does replace a large part of the interpreter, a large part of the standard library is shared with CPython. As a result, PyPy is frequently affected by the same vulnerabilities as CPython, and we have to backport security fixes to it.

Backporting security fixes inside CPython is relatively easy. All main Python branches are in a single repository, so it’s just a matter of cherry-picking the commits. Normally, you can easily move patches between two related git repositories using git-style patches but this isn’t going to work for two repositories with unrelated histories.

Does this mean manually patching PyPy and rewriting commit messages by hand? Luckily, there’s a relatively simple git am trick that can help you avoid that.

Read more

Programming Leftovers

Filed under
Development

  • Rust Lang team March update

    Did you know that you can see the lang team's active projects on our project board? We're still experimenting and evolving the setup, but the goal is that it should give you a quick overview of what kinds of things the lang team is focused on, and what stage they are in their development. Our minutes contain a writeup for each active project, but let me call out a few highlights here...

  • DIY primary/foreign key relationships, again

    In a blog post in 2020 I described a problem I was finding in linked tables. One table had a primary key field and the other had a foreign key field that should have referred back to the first table. That wasn't always the case, because the tables didn't always come from a database with referential integrity. The tables were sometimes built in spreadsheets and the primary and foreign keys were entered by hand.

    The defective tables usually have formatting differences or orphaned foreign keys. The formatting issue is that the primary key is something like "Abc_def_236-ghi" and the foreign key is "Abc-def-236-ghi"; close, but no cigar. Orphaned foreign keys are correctly formatted entries with no match at all in the primary key set.

  • Flutter 2.0 reaches stable and adds support for foldable and dual-screen devices

    For a while now, Flutter for Desktop has been in an alpha stage, which meant changing APIs, bugs, and performance issues. With Flutter 2.0, Google has moved its status to somewhere between beta and stable. What does that mean? Well, it’s available in Flutter 2.0 Stable, but Google doesn’t think it’s fully complete yet. It should be fine for production use, but there may be a bug here and there.

  • How I Built a Web Scraper with Beautiful Soup and Used it to Land My First Job

    Landing any job, let alone a first job, can be a difficult process. Employers often tell you that you don't have enough experience for them to hire you. But that means you also won't get an opportunity to gain that experience (like a job).

    Landing a job in tech can feel even more challenging. On the one hand you have to answer interview questions well, like any other job. On the other you have to prove that your technical skills can do the job you're interviewing for.

    These hurdles can be difficult to overcome. In this article I'll share how I built a web scraper to help me land my first job in tech. I'll explain what exactly I built and the key lessons I learned. Most importantly, I'll share how I leveraged those lessons to ace my interviews and land a job offer.

  • We Sent 304,654 Coding Tests to Developers from 156 Countries – Here’s What We Learned

    At DevSkiller, we are known for our detailed industry reports that assist IT recruitment professionals with their hiring decisions. And this past year has been the most diverse and data-heavy set of information ever compiled by our team.

    Despite the circumstances that 2020 brought us, the show must go on. We have compiled 304,654 coding tests sent to developers in 156 countries to create the 2021 DevSkiller IT skills report.

    Whilst it’s easy to point to the big tech multinationals that will indeed profit from a crisis like we’ve had, many other small businesses will have a hard time adapting to the market’s fluctuating demands.

  • Qt 6.0.2 Released

    We have released Qt 6.0.2 today. As a patch release, the Qt 6.0.2 does not add any new functionality but provides bug fixes and other improvements.

  • The Month in WordPress: February 2021

    That was Josepha Haden Chomphosy on WordPress is Free(dom) episode of the WP Briefing Podcast, speaking about the four freedoms of open-source software. Those four freedoms are core to how WordPress is developed. A lot of the updates we bring you this month will resonate with those freedoms.

  • Toolbox your Debian

    Last week I needed a Debian system to test things, I had heard others were using toolbox with Debian images without much trouble so decided to give it a go instead of creating a VM.

    Toolbox only requires a handful utilities to work with any given docker image. After a quick search I stumbled upon Philippe’s post which in turn linked into this PR about an Ubuntu based toolbox image. Looks like the last major issues where worked out recently in toolbox and there isn’t anything extra needed apart the image.

  •   

  • February GNU Spotlight with Mike Gerwitz: 23 new releases

    23 new GNU releases in the last month (as of February 25, 2021):
    artanis-0.5
    autoconf-archive-2021.02.19
    binutils-2.36.1
    freeipmi-1.6.7
    freeipmi-1.6.8
    glibc-2.33
    gnuhealth-3.8.0
    gwl-0.3.0
    help2man-1.48.1
    inetutils-2.0
    intlfonts-1.4.1
    libgcrypt-1.9.2
    libredwg-0.12.1
    libredwg-0.12.2
    linux-libre-5.11
    mailutils-3.12
    nano-5.6
    nettle-3.7.1
    octave-6.2.0
    parallel-20210222
    tar-1.34
    unifont-13.0.06
    xorriso-1.5.4.pl02

Programming Leftovers

Filed under
Development
  • How to Deploy a Node.js App – From Server Setup to Production

    In this tutorial, we are going to learn everything we need to know before deploying a Node app to a production server.

    We will start by renting a server on Digital Ocean. Then we'll configure this server, connect to it, install Nginx and configure it, pull or create our Node app, and run it as a process.

    As you can see, there is a lot to do and it will be an action-packed tutorial. So let's get started without wasting any time.

  • React Tutorial – Learn React and JavaScript Programming Basics with Example Code

    React is an open-source JavaScript library that helps you build user interfaces. It's a component-based, declarative, "learn once and write anywhere" tool.

  • How to Build a Solid To-Do App with React

    In this tutorial you will learn how to create a basic Solid to-do app. But what is Solid – not to be confused with SOLID? Well, it's a set of conventions and tools used to build decentralized apps.

    So what do I mean by decentralized? Currently, all our data is centralized in a few web platforms: Facebook, Google, and others. This has various consequences for privacy that we're all aware of, but it also endangers the principle of universality of the web: the web must be accessible to everyone.

  • Working with RESTful Web Services in JavaScript - Creating your First Web Service

    Some developers consider themselves to be server-side specialists, while others focus most of their efforts on client-side coding. The departments of IT shops often reflect those tendencies. However, that need not be the case. Thanks to the emergence of technologies like Node.js, you can write both the server and client components using essentially the same language. In this series on RESTful Web services we'll cover how to to do it all using JavaScript. Today's installment will focus on setting up a basic Web service using Node.js and the Express framework.

  • How to Install R Programming Language Tools on Linux Mint 20

    By following today’s tutorial, you will be able to install the “R” programming language on your Linux Mint 20 system instantly. However, as soon as you feel like you do not want to work with this language anymore, you can conveniently uninstall it from your system to free up your resources.

  • Using Delve to debug Go programs on Red Hat Enterprise Linux - Red Hat Developer

    Delve is now available on Red Hat Enterprise Linux (RHEL). Starting in the RHEL 8.2 and devtools-2020.2 releases, the Go language debugger Delve will be installed with the Go toolchain itself via the go-toolset package.

    Being tailored specifically for Go, Delve has intricate knowledge of the Go runtime and provides features and an environment not available in other debuggers. The tool aims for simplicity of use, staying out of your way as you figure out what’s going wrong with your program. Delve also offers powerful features that let you debug your Go programs as quickly as possible.

  • Python's datetime Module – How to Handle Dates in Python

    In this quick guide to Python's datetime module, you'll learn how to parse dates, extract meaningful information from dates, handle timedelta objects and much more.

    So without further ado let's start counting time with Python!

    Most programming languages provide libraries for easy handling of dates. Python offers the powerful datetime module with its many functions and lucid documentation which makes parsing dates easy.

  • Elegant bash conditionals

    The if-statement is a very basic thing, not just in bash, but in all of programming. I see them used quite a lot in shell scripts, even though in many cases they can be replaced with something much more elegant.

    In this rather short article, I'll show how control operators can be used instead. Many probably know about this, but don't realize how to use them nicely. This will help you write cleaner shell scripts in the future.

  • How to write a function in bash

    When you are writing a complex bash script (or any complex program for that matter), subdividing the functional logic of the script in smaller modules and writing/testing each module is often an effective coding strategy. A modularized bash script not only makes the script easier to understand, but also makes individual modules re-usable. In bash, such modular programming is achieved with bash functions.

    Even if you have little coding experience, you are probably familiar with the notion of a function in programming. A function is basically a self-contained block of code that performs a specific task via well-defined input/output interfaces. Let's find out how a bash function is written and how to use a function in bash. This bash tutorial will specifically cover how to create a bash function, how to pass arguments to a bash function, how to call a bash function, how to return a bash function, etc.

  • 5 signs you might be a Rust programmer

    I'm a fairly recent convert to Rust, which I started to learn around the end of April 2020. But, like many converts, I'm an enthusiastic evangelist. I'm also not a very good Rustacean, truth be told, in that my coding style isn't great, and I don't write particularly idiomatic Rust. I suspect this is partly because I never really finished learning Rust before diving in and writing quite a lot of code (some of which is coming back to haunt me) and partly because I'm just not that good a programmer.

    But I love Rust, and so should you. It's friendly—well, more friendly than C or C++; it's ready for low-level systems tasks—more so than Python, it's well-structured—more than Perl; and, best of all, it's completely open source from the design level up—much more than Java, for instance.

    Despite my lack of expertise, I noticed a few things that I suspect are common to many Rust enthusiasts and programmers. If you say "yes" to the following five signs (the first of which was sparked by some exciting recent news), you, too, might be a Rust programmer.

  • What Every Developer Must Know About Encoding and Unicode

    If you are coding an international app that uses multiple languages, you'll need to know about encoding. Or even if you're just curious how words end up on your screen – yep, that's encoding, too.

    I'll explain a brief history of encoding in this article (and I'll discuss how little standardisation there was) and then I'll talk about what we use now. I'll also cover some Computer Science theory you need to understand.

ABI checking

Filed under
Development

There is no day so wasted that you can’t take notes about what didn’t work, so here’s some talk about ABI-compliance-checking. ABI-compliance, or ABI-compatibility, is roughly when a shared library can be changed (to a different version, usually an update and upgrade) and users of that shared library (applications, or other libraries) just work with the new version. This requires some discipline, and there are tools to help out.

[...]

One way to help maintain binary compatibility is to use tools that check the ABI: figure out the shape of the ABI in one version, the shape in another version, and compare those shapes. KDE Frameworks have checks in place, like this one (that link assumes openSUSE and Qt 5.15 are still in use and that there was a recent successful build).

Generally, an ABI-shape getting bigger is not a problem (from a technical perspective, although you can have all kinds of semantic mix-ups). Things that go away – functions, variables, etc. – those are problematic.

Calamares is a Linux system installer – it can be customized by Linux distro’s to act as the installer for their ISO images. It’s a C++ program offering modules for all kinds of system-installation services. It also offers an ABI: the modules use the ABI of the Calamares libraries to talk to the main program. Calamares supports “third-party” modules, e.g. modules specific to one distro or otherwise customized, and for those third-party modules, ABI compatibility suddenly becomes an issue: it would be nice if they didn’t have to be recompiled when a new Calamares library comes out. That can only happen if the Calamares libraries commit to ABI compatibility.

Read more

Programming Leftovers

Filed under
Development
  • The Hijacking of Perl.com

    For a week we lost control of the Perl.com domain. Now that the incident has died down, we can explain some of what happened and how we handled it. This incident only affected the domain ownership of Perl.com and there was no other compromise of community resources. This website was still there, but DNS was handing out different IP numbers.

    First, this wasn’t an issue of not renewing the domain. That would have been a better situation for us because there’s a grace period.

    Second, to be very clear, I’m just an editor for the website that uses the Perl.com domain. This means that I’m not actually the “injured party” in legal terms. Tom Christiansen is the domain registrant, and should legal matters progress, there’s no reason for me, nor anyone else, to know all of the details. However, I’ve talked to many of the people involved in the process.

  • Learn Java with object orientation by building a classic Breakout game

    Several of my courses use Processing, a software engine that uses Java. Java is a great language for learning programming concepts, in part because it's a strongly typed language.

    Despite being free to choose any language or framework for my Breakout project, I chose to continue in Java to apply what I've learned in my coursework. I also wanted to use a framework so that I did not need to do everything from scratch. I considered using Godot, but that would mean I would hardly need to program at all.

    Instead, I chose TotalCross. It is an open source software development kit (SDK) and framework with a simple game engine that generates code for Linux Arm devices (like the Raspberry Pi) and smartphones. Also, because I work for TotalCross, I have access to developers with much more experience than I have and know the platform very well. It seemed to be the safest way and, despite some strife, I don't regret it one bit. It was very cool to develop the whole project and see it running on the phone and the Raspberry Pi.

  • Python Developers Survey 2020 Results | JetBrains: Developer Tools for Professionals and Teams

    This is the fourth iteration of the official annual Python Developers Survey, conducted as a collaborative effort between the Python Software Foundation and JetBrains. In October 2020, more than 28,000 Python developers and enthusiasts from almost 200 countries/regions took the survey to reveal the current state of the language and the ecosystem around it.

  • Python Developer Survey Highlights

    The annual Python Developers Survey, conducted by the Python Software Foundation and JetBrains, explores Python usage among 28,000 Python developers from almost 200 countries and regions.

    According to this year’s results, 85% of survey respondents use Python as their main programming language. Additionally, JavaScript is the most popular language to combine with Python, with 75% of web developers using both. “Together with HTML/CSS, Bash/Shell, and SQL, they create a stack of languages where 2 out of every 5 Python devs are using at least one of them,” the report states.

  • 3 features that debuted in Python 3.0 you should use now

    This is the first in a series of articles about features that first appeared in a version of Python 3.x. Python 3.0 was first released in 2008, and even though it has been out for a while, many of the features it introduced are underused and pretty cool. Here are three you should know about.

    [...]

    Python 3.0 and its later versions have been out for more than 12 years, but some of its features are underutilized. In the next article in this series, I'll look at three more of them.

  • How to Plot Data in Pandas Python – Linux Hint

    Data visualization plays an important role in data analysis. Pandas is a strong data analysis library in python for data science. It provides various options for data visualization with .plot() method. Even if you are a beginner, you can easily plot your data using the Pandas library. You need to import the pandas and matplotlib.pyplot package for data visualization.

    In this article, we will explore various data plotting methods by using the Pandas python. We have executed all examples on the pycharm source code editor by using the matplotlib.pyplot package.

    [...]

    In this article, you have seen how to plot DataFrames in Pandas python. Different kinds of plotting are performed in the above article. To plot more kinds such as box, hexbin, hist, kde, density, area, etc., you can use the same source code just by changing the plot kind.

  • Qt 6.1 Beta Released

    I am pleased to announce that we released the first beta of Qt 6.1 today. As the first feature update in the Qt 6 series, Qt 6.1 adds some important new functionality and brings support for multiple additional libraries. We will continue to provide subsequent beta releases via the online installer throughout the beta phase.

  • Qt 6.1 Beta Released With Porting More Modules To Qt6 - Phoronix

    The first beta is out today of the forthcoming Qt 6.1 toolkit.

    It's not even been three months yet since the official release of Qt 6.0 while Qt 6.1 Beta has already arrived as part of the expedited Qt 6 releases this year in trying to button things up so that more developers are ready to migrate their codebases from Qt 5 to Qt 6 with remaining gaps being filled.

    It was in mid-February that Qt 6.1 Alpha was released as part of the v6.1 release schedule that aims to officially ship Qt 6.1.0 by the end of April.

  • Qt for MCUs – A perfect development platform for the fitness industry

    Historically, the fitness industry has been extremely commoditized, especially in the cardiovascular equipment space. The main cardiovascular products [treadmill, upright stationary bike, recumbent stationary bike, and elliptical machine] have minor differences among the different product brands. For example, a treadmill is made up of a deck, deck cushioning, belt, motor, uprights with handrails, heartrate monitor, tray, “Deadman’s switch” key clip, power switch with cord, and console. These parts are considered “table stakes” meaning every treadmill, regardless of brand, has them.

  • RPushbullet 0.3.4: Small Update, Nicer Docs

    Release 0.3.4 of the RPushbullet package arrived on CRAN today. RPushbullet interfaces the neat Pushbullet service for inter-device messaging, communication, and more. It lets you easily send (programmatic) alerts like the one to the left to your browser, phone, tablet, … – or all at once.

    This release contains a contributed PR to better reflect an error code, and adds a mkdocs-material-based documentation site (just like a few other packages of mine). See below for more details.

  • Dirk Eddelbuettel: RcppSimdJson 0.1.4 on CRAN: Small Bugfix

    A quick note to say that we finalized a bugfix release 0.1.5 of RcppSimdJson yesterday which got onto CRAN earlier today. RcppSimdJson wraps the fantastic and genuinely impressive simdjson library by Daniel Lemire and collaborators. Via very clever algorithmic engineering to obtain largely branch-free code, coupled with modern C++ and newer compiler instructions, it results in parsing gigabytes of JSON parsed per second which is quite mindboggling. The best-case performance is ‘faster than CPU speed’ as use of parallel SIMD instructions and careful branch avoidance can lead to less than one cpu cycle per byte parsed; see the video of the talk by Daniel Lemire at QCon (also voted best talk).

  • Remove First Character from String in JavaScript

    In the previous article, you have learned to remote last character of a string in JavaScript. If you are looking for remove last character from string, visit here. This tutorial describe you to how to remove first character of a string in JavaScript. You can choose any one of the following methods.

  • Fortran newsletter: March 2021

    Welcome to the March 2021 edition of the monthly Fortran newsletter. The newsletter comes out at the beginning of every month and details Fortran news from the previous month.

Programming Leftovers

Filed under
Development
  • DRY enums for Absinth macros

    Absinth is a great GraphQL library for Elixir, but it brings a few challenges as it’s practically implemented using macros. One of these challenges is a DRY way of reusing enumerables in Absinth enums.

    [...]

    The only thing we had to do is to use require to require the module beforehand.

  • Gzip::Zopfli - another compression module

    Following on from the Gzip::Libdeflate I mentioned before, I also made this: Gzip::Zopfli

    It is based on the Zopfli gzip compression library from Google Research.

  • Weird architectures weren't supported to begin with

    You don’t know about any of the above until the bug reports start rolling in: users will report bugs that have already been fixed, bugs that you explicitly document as caused by unsupported configurations, bugs that don’t make any sense whatsoever.

    You struggle to debug your users’ reports, since you don’t have access to the niche hardware, environments, or corporate systems that they’re running on. You slowly burn out as an unending torrent of already fixed bugs that never seem to make it to your users. Your user base is unhappy, and you start to wonder why you’re putting all this effort into project maintenance in the first place. Open source was supposed to be fun!

    What’s the point of this spiel? It’s precisely what happened to pyca/cryptography: nobody asked them whether it was a good idea to try to run their code on HPPA, much less System/3906; some packagers just went ahead and did it, and are frustrated that it no longer works. People just assumed that it would, because there is still a norm that everything flows from C, and that any host with a halfway-functional C compiler should have the entire open source ecosystem at its disposal.

  • Woodruff: Weird architectures weren't supported to begin with

    William Woodruff has posted a rant of sorts on the adoption of Rust by the Python Cryptography project, which was covered here in February.

Programming Leftovers

Filed under
Development
  • The RedMonk Programming Language Rankings: January 2021 [Ed: Microsoft-sponsored Stephen O'Grady/Redmonk uses Microsoft data to rank programming languages as if a proprietary software repository occupied by a monopolies says what the trends are in industry at large
  • IAR Systems adds Functional Safety certification for build tools for Linux

    IAR Systems®, the future-proof supplier of software tools and services for embedded development, announces that its build tools supporting deployment in Linux-based frameworks has been certified by TÜV SÜD for functional safety development.

  • Clazy Framework Employed To Help Port Qt 5 Code To Qt 6 - Phoronix

    The Qt Company is now offering some checks for the Clazy framework to help in porting Qt 5 code to Qt 6 compatibility.

    Clazy is KDE's Qt-focused static code analyzer built atop LLVM's Clang. Clazy has been very useful for years in helping KDE/Qt developers discover bugs in their code and also help in some areas with automatic refactoring.

  • Porting from Qt 5 to Qt 6 using Clazy checks [Ed: Moving to proprietary software releases of Qt, which is no longer suitable for freedom-respecting developers]

    If you are looking for some help to port from Qt 5 to Qt 6, look no further. Within the Clazy framework, we've implemented some checks and fixits dedicated to help porting your Qt-based project.

    Those checks can be run using Clazy as a compiler plugin, using clazy-standalone on a .json file or from within Qt Creator.

    [..]

    First, you need to get Clazy or make sure your version is up to date. Version 1.10 will contain a corrected check for the deprecated API fixes, in the mean time please use the master branch.

  • Using Increment (++) and Decrement (–) Operators in Bash

    Similar to other programming language bash also supports increment and decrement operators. The increment operator ++ increases the value of a variable by one. Similarly, the decrement operator -- decreases the value of a variable by one.

  • Rakudo Weekly News: 2021.09 Best of Raku?

    Daniel Sockwell has started a discussion on what to do with the contents provided by the CCR Project with an idea to publish a “Best Of Raku” book. Modelled after books such as Coders at Work and Introduction to Best Software Writing, it would ask the Raku community to select 15-30 blog posts that do a good job of telling the story of the Raku Programming Language, thereby providing a good overview of what Raku is all about. Further suggestions, and other ideas, are very much welcome! And on a related note, 328 blog posts have been remastered so far!

  • Henri Sivonen: Rust Target Names Aren’t Passed to LLVM

    TL;DR: Rust’s i686-unknown-linux-gnu target requires SSE2 and, therefore, does not mean the same as GCC’s -march=i686. It is the responsibility of Linux distributions to use a target configuration that matches what they intend to support.

    From time to time, claims that Rust is “not portable” flare up. “Not portable” is generally means “LLVM does not support my retrocomputing hobby target.” This is mostly about dead ISAs like DEC Alpha. There is a side track about x86, though: the complaint that Rust’s default 32-bit x86 (glibc) Linux target does not support all x86 CPUs that are still supported by a given Linux distribution.

    Upstream Rust ships with two preconfigured 32-bit x86 glibc Linux targets: The primary one has the kind of floating-point math that other ISAs have and requires SSE2. “Primary” here means that the Rust project considers this “guaranteed to work”. The secondary does not require SSE2 and, therefore, works on even older CPUs but has floating-point math that differs from other ISAs. “Secondary” here means that the Rust project considers this only “guaranteed to build”. Conceptually, this is simple: x86 with SSE2 and x86 without SSE2. Pick the former if you can and the latter if you must.

Programming Leftovers

Filed under
Development
  • Revisiting Html in Java

    Some time ago I wrote a post about creating an embedded dsl for Html in Java. Sadly, it was based on an abuse of lambda name reflection that was later removed from Java.

    I thought I should do a followup because a lot of people still visit the old article. While it’s no longer possible to use lambda parameter names in this way, we can still get fairly close.

  • Use Dash as /bin/sh

    I want startup scripts and everything that has a #!/bin/sh shebang to use the lightest possible shell by default, but I still want my trusty bash in interactive terminal sessions, and for complex scripts.

  • How to Use Group by in Pandas Python – Linux Hint

    Pandas group by function is used for grouping DataFrames objects or columns based on particular conditions or rules. Using the groupby function, the dataset management is easier. However, all related records can be arranged into groups. Using the Pandas library, you can implement the Pandas group by function to group the data according to different kinds of variables. Most developers used three basic techniques for the group by function. First, splitting in which data divide into groups based on some particular conditions. Then, apply certain functions to these groups. In the end, combine the output in the form of data structure.

    In this article, we will walk through the basic uses of a group by function in panda’s python. All commands are executed on the Pycharm editor.

  • gfldex: Undocumented escape hatch

    On my quest to a custom when-statement I did quite a bit of reading. The study of roast and Actions.nqp can lead to great gain in knowledge.

  • Knowing when to look past your code

    At some point, though, your journies will take you to places where things aren’t so clear cut, and you’ll start to gain a sixth sense; a kind of visceral experience that things are not as they have been promised to be.

    A few weeks ago, that sixth sense whispered in my ear: “what if, instead of your cruddy bootloader written in a pre-1.0 systems language for a platform you don’t fully understand, it’s the 20 year-old project with 80,000 commits that’s wrong?” And it was right.

  • Cambalache…
  • C++ Friend Function – Linux Hint

    A function is a block of code that performs a certain task and provides the output. It is mainly used to eliminate repetitive code. In this tutorial, we will look into the friend function in C++ and explain its concept with working examples.

  • mrcal: principled camera calibrations

    In my day job I work with images captured by cameras, using those images to infer something about the geometry of the scene being observed. Naturally, to get good results you need to have a good estimate of the behavior of the lens (the "intrinsics"), and of the relative geometry of the cameras (the "extrinsics"; if there's more than one camera).

    The usual way to do this is to perform a "calibration" procedure to compute the intrinsics and extrinsics, and then to use the resulting "camera model" to process the subsequent images. Wikipedia has an article. And from experience, the most common current toolkit to do this appears to be OpenCV.

    People have been doing this for a while, but for whatever reason the existing tools all suck. They make basic questions like "how much data should I gather for a calibration?" and "how good is this calibration I just computed?" and "how different are these two models?" unanswerable.

Programming Leftovers

Filed under
Development
  • The HTTP Referer header is fading away (at least as a useful thing)

    The HTTP Referer header on requests is famously misspelled (it should be Referrer), and also famously not liked because of privacy and security concerns. The privacy and security concerns are especially strong with external ('cross-origin') Referers, which is also the ones that many people find most useful because they tell you where visitors to your pages are coming from and let you find places where people have linked to you or are mentioning you.

  • Top 10 Natural Language Processing (NLP) Trends To Look Forward

    AI and Machine Learning have gifted us marvelous things. NLP or Natural Language Processing is one of them. It is one of the most prominent applications of AI. We are using this technology in our day-to-day life without even knowing. Translators, speech recognition apps, chatbots are actually NLP-powered products. Tech giants like Google and Microsoft are making new developments in NLP every year. If you are an AI enthusiast, you should go deep inside NLP. Chill! We got you covered. Just go through the article, and know about the top NLP trends that most data scientists are talking about.

  • Russ Allbery: DocKnot 4.01

    DocKnot is my software documentation and release management tool. This release adds support for a global user configuration file separate from the metadata for any given project and adds support for signing generated distribution tarballs with GnuPG. Currently, the only configuration options for the global configuration file are to set the destination location of generated distributions and the PGP key to use when signing them.

  • horizonator: terrain renderer based on SRTM DEMs

    I just resurrected and cleaned up an old tool I had lying around. It's now nice and usable by others. This tool loads terrain data, and renders it from the ground, simulating what a human or a camera would see. This is useful for armchair exploring or for identifying peaks. This was relatively novel when I wrote it >10 years ago, but there are a number of similar tools in existence now. This implementation is still useful in that it's freely licensed and contains APIs, so fancier processing can be performed on its output.

  • Happy birthday, Python, you're 30 years old this week: Easy to learn, and the right tool at the right time

    The 30th anniversary of Python this week finds the programming language at the top of its game, but not without challenges.

    "I do believe that Python just doesn’t have the right priorities these days," said Armin Ronacher, director of engineering at software monitoring biz Sentry and creator of Flask, the popular Python web app framework, in an email interview with The Register.

    Ronacher, a prolific Python contributor, remains a fan of the language. He credits Python's success to being both easy to learn and having an implementation that was easy to hack. And in its early years, Python didn't have a lot of competitors with those same characteristics, he said.

  • Google fires 150 game developers hired for Stadia: Report

    In about two years, Google has announced to shut down the in-house Stadia game development division, as it sees a great adoption of its technology by third-party developers and publishers to create world-class games.

    Google has said that it will not be investing further in bringing exclusive content from its internal development team SG&E, beyond any near-term planned games.

Syndicate content

More in Tux Machines

Canonical Chooses Google’s Flutter UI SDK to Build Future Ubuntu Apps

For those not in the known, Flutter is an open-source UI SDK (software development kit) created by Google to helps those who want to build quick and modern applications for a wide-range of operating systems, including Android, Linux, Mac, iOS, Windows, Google Fuchsia, that work across desktop, mobile, and the Web. A year ago, Canonical teamed up with Google to make the Flutter SDK available on Linux as Snap, the universal software deployment and package management system for Ubuntu `and other GNU/Linux distributions, allowing those interested in building beautiful apps on the Linux desktop. Read more

Python: Security and NumPy 1.20 Release

  • Python Package Index nukes 3,653 malicious libraries uploaded soon after security shortcoming highlighted

    The Python Package Index, also known as PyPI, has removed 3,653 malicious packages uploaded days after a security weakness in the use of private and public registries was highlighted. Python developers use PyPI to add software libraries written by other developers in their own projects. Other programming languages implement similar package management systems, all of which demand some level of trust. Developers are often advised to review any code they import from an external library though that advice isn't always followed. Package management systems like npm, PyPI, and RubyGems have all had to remove subverted packages in recent years. Malware authors have found that if they can get their code included in popular libraries or applications, they get free distribution and trust they haven't earned. Last month, security researcher Alex Birsan demonstrated how easy it is to take advantage of these systems through a form of typosquatting that exploited the interplay between public and private package registries.

  • A pair of Python vulnerabilities [LWN.net]

    Two separate vulnerabilities led to the fast-tracked release of Python 3.9.2 and 3.8.8 on February 19, though source-only releases of 3.7.10 and 3.6.13 came a few days earlier. The vulnerabilities may be problematic for some Python users and workloads; one could potentially lead to remote code execution. The other is, arguably, not exactly a flaw in the Python standard library—it simply also follows an older standard—but it can lead to web cache poisoning attacks. [...] [Update: As pointed out in an email from Moritz Muehlenhoff, Python 2.7 actually is affected by this bug. He notes that python2 on Debian 10 ("Buster") is affected and has been updated. Also, Fedora has a fix in progress for its python2.7 package.]

  • NumPy 1.20 has been released

    NumPy is a Python library that adds an array data type to the language, along with providing operators appropriate to working on arrays and matrices. By wrapping fast Fortran and C numerical routines, NumPy allows Python programmers to write performant code in what is normally a relatively slow language. NumPy 1.20.0 was announced on January 30, in what its developers describe as the largest release in the history of the project. That makes for a good opportunity to show a little bit about what NumPy is, how to use it, and to describe what's new in the release. [...] NumPy adds a new data type to Python: the multidimensional ndarray. This a container, like a Python list, but with some crucial differences. A NumPy array is usually homogeneous; while the elements of a list can be of various types, an ndarray will, typically, only contain a single, simple type, such as integers, strings, or floats. However, these arrays can instead contain arbitrary Python objects (i.e. descendants of object). This means that the elements will, for simple data types, all occupy the same amount of space in memory. The elements of an ndarray are laid out contiguously in memory, whereas there is no such guarantee for a list. In this way, they are similar to Fortran arrays. These properties of NumPy arrays are essential for efficiency because the location of each element can be directly calculated. Beyond just adding efficient arrays, NumPy also overloads arithmetic operators to act element-wise on the arrays. This allows the Python programmer to express computations concisely, operating on arrays as units, in many cases avoiding the need to use loops. This does not turn Python into a full-blown array language such as APL, but adds to it a syntax similar to that incorporated into Fortran 90 for array operations.

4 Best Free and Open Source Graphical MPD Clients

MPD is a powerful server-side application for playing music. In a home environment, you can connect an MPD server to a Hi-Fi system, and control the server using a notebook or smartphone. You can, of course, play audio files on remote clients. MPD can be started system-wide or on a per-user basis. MPD runs in the background playing music from its playlist. Client programs communicate with MPD to manipulate playback, the playlist, and the database. The client–server model provides advantages over all-inclusive music players. Clients can communicate with the server remotely over an intranet or over the Internet. The server can be a headless computer located anywhere on a network. There’s graphical clients, console clients and web-based clients. To provide an insight into the quality of software that is available, we have compiled a list of 4 best graphical MPD clients. Hopefully, there will be something of interest here for anyone who wants to listen to their music collection via MPD. Here’s our recommendations. They are all free and open source goodness. Read more

LWN on Kernel: 5.12 Merge, Lockless Algorithms, and opy_file_range()

  • 5.12 Merge window, part 1 [LWN.net]

    The beginning of the 5.12 merge window was delayed as the result of severe weather in the US Pacific Northwest. Once Linus Torvalds got going, though, he wasted little time; as of this writing, just over 8,600 non-merge changesets have been pulled into the mainline repository for the 5.12 release — over a period of about two days. As one might imagine, that work contains a long list of significant changes.

  • An introduction to lockless algorithms [LWN.net]

    Low-level knowledge of the memory model is universally recognized as advanced material that can scare even the most seasoned kernel hackers; our editor wrote (in the July article) that "it takes a special kind of mind to really understand the memory model". It's been said that the Linux kernel memory model (and in particular Documentation/memory-barriers.txt) can be used to frighten small children, and the same is probably true of just the words "acquire" and "release". At the same time, mechanisms like RCU and seqlocks are in such widespread use in the kernel that almost every developer will sooner or later encounter fundamentally lockless programming interfaces. For this reason, it is a good idea to equip yourself with at least a basic understanding of lockless primitives. Throughout this series I will describe what acquire and release semantics are really about, and present five relatively simple patterns that alone can cover most uses of the primitives.

  • How useful should copy_file_range() be? [LWN.net]

    Its job is to copy len bytes of data from the file represented by fd_in to fd_out, observing the requested offsets at both ends. The flags argument must be zero. This call first appeared in the 4.5 release. Over time it turned out to have a number of unpleasant bugs, leading to a long series of fixes and some significant grumbling along the way. In 2019 Amir Goldstein fixed more issues and, in the process, removed a significant limitation: until then, copy_file_range() refused to copy between files that were not located on the same filesystem. After this patch was merged (for 5.3), it could copy between any two files, falling back on splice() for the cross-filesystem case. It appeared that copy_file_range() was finally settling into a solid and useful system call. Indeed, it seemed useful enough that the Go developers decided to use it for the io.Copy() function in their standard library. Then they ran into a problem: copy_file_range() will, when given a kernel-generated file as input, copy zero bytes of data and claim success. These files, which include files in /proc, tracefs, and a large range of other virtual filesystems, generally indicate a length of zero when queried with a system call like stat(). copy_file_range(), seeing that zero length, concludes that there is no data to copy and the job is already done; it then returns success. But there is actually data to be read from this kind of file, it just doesn't show in the advertised length of the file; the real length often cannot be known before the file is actually read. Before 5.3, the prohibition on cross-filesystem copies would have caused most such attempts to return an error code; afterward, they fail but appear to work. The kernel is happy, but some users can be surprisingly stubborn about actually wanting to copy the data they asked to be copied; they were rather less happy.