Language Selection

English French German Italian Portuguese Spanish

Debian

Syndicate content
Planet Debian - https://planet.debian.org/
Updated: 12 hours 35 min ago

Vincent Sanders: A very productive weekend

Tuesday 19th of February 2019 09:54:48 PM
I just hosted a NetSurf Developer weekend which is an opportunity for us to meet up and make use of all the benefits of working together. We find the ability to plan work and discuss solutions without loosing the nuances of body language generally results in better outcomes for the project.
Due to other commitments on our time the group has not been able to do more than basic maintenance activities in the last year which has resulted in the developer events becoming a time to catch up on maintenance rather than making progress on features.
Because of this the July and November events last year did not feel terribly productive, there were discussions about what we should be doing and bugs considered but a distinct lack of commuted code.
As can be seen from our notes this time was a refreshing change. We managed to complete a good number of tasks and actually add some features while still having discussions, addressing bugs and socialising.
We opened on the Friday evening by creating a list of topics to look at over the following days and updating the wiki notes. We also reviewed the cross compiler toolchains which had been updated to include the most recent releases for things like openssl, curl etc.
As part of this review we confirmed the decision to remove the Atari platform from active support as its toolchain builds have remained broken for over two years with no sign of any maintainer coming forward.
While it is a little sad to see a platform be removed it has presented a burden on our strained resources by requiring us to maintain a CI worker with a very old OS using tooling that can no longer be replicated. The tooling issue means a developer cannot test changes locally before committing so testing changes that affected all frontends was difficult.
Saturday saw us clear all the topics from our list which included:
  • Fixing a bug preventing compiling our reference counted string handling library.
  • Finishing the sanitizer work started the previous July
  • Fixing several bugs in the Framebuffer frontend installation.
  • Making the Framebuffer UI use the configured language for resources.
The main achievement of the day however was implementing automated system testing of the browser. This was a project started by Daniel some eight years ago but worked on by all of us so seeing it completed was a positive boost for the whole group.
The implementation consisted of a frontend named monkey. This frontend to the browser takes textural commands to perform operations (i.e. open a window or navigate to a url) and generates results in a structured text format. Monkey is driven by a python program named monkeyfarmer which runs a test plan ensuring the results are as expected.
This allows us to run a complete browsing session in an automated way, previously someone would have to manually build the browser and check the tests by hand. This manual process was tedious and was rarely completed across our entire test corpus generally concentrating on just those areas that had been changed such as javascript output.
We have combined the monkey tools and our test corpus into a CI job which runs the tests on every commit giving us assurance that the browser as a whole continues to operate correctly without regression. Now we just have the task of creating suitable plans for the remaining tests. Though I remain hazy as to why, we became inordinately amused by the naming scheme for the tools.
We rounded the Saturday off by going out for a very pleasant meal with some mutual friends. Sunday started by adding a bunch of additional topics to consider and we made good progress addressing these. 
We performed a bug triage and managed to close several issues and commit to fixing a few more. We even managed to create a statement of work of things we would like to get done before the next meetup.
My main achievement on the Sunday was to add WEBP image support. This uses the Google libwebp library to do all the heavy lifting and adding a new image content handler to NetSurf is pretty straightforward.

Sylvain Beucler: RenPyWeb - Ren'Py in your HTML5 web browser

Tuesday 19th of February 2019 06:38:01 PM

I like the Ren'Py project, a popular game engine aimed at Visual Novels - that can also be used as a portable Python environment.

One limitation was that it required downloading games, while nowadays people are used to Flash- or HTML5- based games that play in-browser without having to (de)install.

Can this fixed? While maintaining compatibility with Ren'Py's several DSLs? And without rewriting everything in JavaScript?
Can Emscripten help? While this is a Python/Cython project?
After lots of experimenting, and full-stack patching/contributing, it turns out the answer is yes!

Live demo:
https://renpy.beuc.net/

At last I finished organizing and cleaning-up, published under a permissive free software / open source license, like Python and Ren'Py themselves.
Python port:
https://www.beuc.net/python-emscripten/python/dir?ci=tip
Build system:
https://github.com/renpy/renpyweb

Development in going on, consider supporting the project!
https://www.patreon.com/Beuc

Reproducible builds folks: Reproducible Builds: Weekly report #199

Tuesday 19th of February 2019 12:03:50 PM

Here’s what happened in the Reproducible Builds effort between Sunday February 10th and Saturday February 16th 2019:

  • strip-nondeterminism is our tool that post-processes files to remove known non-deterministic output. This week, Chris Lamb adjusted its behaviour to deduplicate hardlinks via stat(2) before processing to avoid issues when handling files in parallel; as the per-filetype handlers are yet currently guaranteed to be atomic, one process could temporarily truncate a file which can cause errors in other processes operating on the “same” file under a different pathname. This was thus causing package build failures in packages that de-duplicate hardlinks in their build process such as the Debian Administrator’s Handbook (#922168).

  • There was a brief update from the Debian Ruby maintainers on whether the language might need to strip -fdebug-prefix-map from the tools used to build extensions.

  • On our mailing list, Holger Levsen re-raised a question regarding uploading the “official” .buildinfo files to buildinfo.debian.net.

  • On Tuesday 26th February Chris Lamb will speak at Speck&Tech 31 “Open Security” on Reproducible Builds in Trento, Italy.

  • Jelle van der Waa fixed some spelling mistakes on the reproducible-builds.org project website. []

  • 6 Debian package reviews were added, 4 were updated and 16 were removed in this week, adding to our knowledge about identified issues.

diffoscope development

diffoscope is our in-depth “diff-on-steroids” utility which helps us diagnose reproducibility issues in packages. This week:

  • Chris Lamb:
    • Add support for comparing .crx Chrome browser extensions. (#41)
    • Add support for comparing MP3 and files with similar metadata. (#43)
    • Replace the literal xxd(1) output (!)(!) in tests/data/hello.wasm with its binary equivalent (#47) and ensure both WebAssembly test data files are actually unique. (#42)
    • Catch tracebacks when mounting invalid filesystem images under guestfs. []
    • Fix tests when using Ghostscript 9.20 vs 9.26 for Debian stable and for stable with the security repositories enabled. [][]
    • Temporarily drop ubuntu-devel from internal test matrix due to a linux-firmware package installation issue. []
  • Ed Maste:
    • Include relocations in objdump disassembly. (#48)
  • Graham Christensen:
    • Clarify “no file-specific differences” message when we fallback to a binary diff. (!19)
  • Mattia Rizzolo:
    • Make test_ps.test_text_diff pass with Ghostscript version 9.26. []

In addition, Vagrant Cascadian updated diffoscope in GNU Guix [] and went on to upload disorderfs [] and trydiffoscope [] too.

Packages reviewed and fixed, and bugs filed Test framework development

We operate a comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org.

  • Hans-Christoph Steiner:
    • Set the LANG and LC_ALL environment variables for F-Droid builds to workaround an unsolved issue in Java/Gradle. [][]
    • Modernise some dependencies. []
    • Node maintenance. []
  • Holger Levsen:
    • Increased the diskspace for the two OSU Open Source Lab Arch Linux build nodes from 50GB to 350GB.
    • Upgraded all 47 nodes running Debian to the newly-released Debian 9.8.
    • Fix the version checking for diffoscope in Arch Linux. []
    • Install kernels as a separate step to ignore failures when installing/upgrading Debian backports’ kernels. []
    • Fix a number of issues with our Munin diskspace plugin. [][]
    • Correct grammar of Arch Linux IRC message. []
  • Mattia Rizzolo:

This week’s edition was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Russ Allbery: INN 2.6.3

Monday 18th of February 2019 10:30:00 PM

INN 2.6.3 has been released. This is a bug fix and minor feature release over INN 2.6.2, and the upgrade should be painless. The main ISC downloads page will be updated shortly; in the meantime, you can download the new release from ftp.isc.org or my personal INN pages. The latter also has links to the full changelog and the other INN documentation.

The big change in this release is support for Python 3. Embedded Python filtering and authentication hooks for innd and nnrpd can now use version 3.3.0 or later of the Python interpreter. Python 2.x is still supported (2.3.0 or later).

Also fixed in this release are several issues with TLS: fixed selection of elliptic curve selection, a new configuration parameter to fine-tune the cipher suites with TLS 1.3, and better logging of failed TLS session negotiation. This release also returns the error message from Python and Perl filter hook rejects to CHECK and TAKETHIS commands and fixes various other, more minor bugs.

As always, thanks to Julien ÉLIE for preparing this release and doing most of the maintenance work on INN!

Chris Lamb: Book Review: Painting the Sand

Monday 18th of February 2019 06:43:21 PM

Painting the Sand (2017)

Kim Hughes

Staff Sargeant Kim Hughes is a bomb disposal operator in the British Army undergoing a gruelling six-month tour of duty in Afghanistan during which he defuses over 100 improvised explosives devices, better known as IEDs.

Cold opening in the heat of the desert, it begins extremely strongly with a set piece of writing that his editor should be proud of. The book contains colourful detail throughout and readers will quickly feel like "one of the lads" with the shibboleth military lingo and acronyms. Despite that, this brisk and no-nonsense account — written in that almost-stereotypical squaddie's tone of voice — is highly accessible and furthermore is refreshing culturally-speaking given the paucity of stories in the zeitgest recounting British derring-do in "Afghan", to adopt Hughes' typical truncation of the country.

However, apart from a few moments (such as when the Taliban adopt devices undetectable with a metal detector) the tension deflates slowly rather than mounting like a lit fuse. For example, I would have wished for a bit more improvised suspense and drama when they were being played, trapped or otherwise manipulated by the Taliban for once, and only so much of that meekness can be attributed to Hughes' self-effacing and humble manner. Indeed, the book was somewhat reminiscent of The Secret Barrister in that the ending is a little too quick for my taste, rushing through "current" events and becoming a little too self-referential in parts, this comparison between the two words being aided by both writers having somewhat of a cynical affect about them.

One is left with the distinct impression that Hughes went to Afghan a job. He gets it done with minimal fuss and no unnecessary nonsense … and that's what exactly what he does as a memorialist.

Raphaël Hertzog: Freexian’s report about Debian Long Term Support, January 2019

Monday 18th of February 2019 02:05:38 PM

Like each month, here comes a report about the work of paid contributors to Debian LTS.

Individual reports

In January, about 204.5 work hours have been dispatched among 13 paid contributors. Their reports are available:

  • Abhijith PA did 12 hours (out of 12 hours allocated).
  • Antoine Beaupré did 9 hours (out of 20.5 hours allocated, thus keeping 11.5h extra hours for February).
  • Ben Hutchings did 24 hours (out of 20 hours allocated plus 5 extra hours from December, thus keeping one extra hour for February).
  • Brian May did 10 hours (out of 10 hours allocated).
  • Chris Lamb did 18 hours (out of 18 hours allocated).
  • Emilio Pozuelo Monfort did 42.5 hours (out of 20.5 hours allocated + 25.25 extra hours, thus keeping 3.25 extra hours for February).
  • Hugo Lefeuvre did 20 hours (out of 20 hours allocated).
  • Lucas Kanashiro did 5 hours (out of 4 hours allocated plus one extra hour from December).
  • Markus Koschany did 20.5 hours (out of 20.5 hours allocated).
  • Mike Gabriel did 10 hours (out of 10 hours allocated).
  • Ola Lundqvist did 4.5 hours (out of 8 hours allocated + 6.5 extra hours, thus keeping 8 extra hours for February, as he also gave 2h back to the pool).
  • Roberto C. Sanchez did 10.75 hours (out of 20.5 hours allocated, thus keeping 9.75 extra hours for February).
  • Thorsten Alteholz did 20.5 hours (out of 20.5 hours allocated).
Evolution of the situation

In January we again managed to dispatch all available hours (well, except one) to contributors. We also still had one new contributor in training, though starting in February Adrian Bunk has become a regular contributor. But: we will lose another contributor in March, so we are still very much looking for new contributors. Please contact Holger if you are interested to become a paid LTS contributor.

The security tracker currently lists 40 packages with a known CVE and the dla-needed.txt file has 42 packages needing an update.

Thanks to our sponsors

New sponsors are in bold.

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

Thomas Lange: Netplan support in FAI

Monday 18th of February 2019 01:34:00 PM

The new version FAI 5.8.1 now generates the configuration file for Ubuntu's netplan tool. It's a YAML description for setting up the network devices, replacing the /etc/network/interfaces file. The FAI CD/USB installation image for Ubuntu now offers two different variants to be installed, Ubuntu desktop and Ubuntu server without a desktop environment. Both are using Ubuntu 18.04 aka Bionic Beaver.

FAI 5.8.1 also improves UEFI support for network installations. UEFI boot is still missing for the ISO images.

The FAI ISO images are available from [1]. The FAIme build service [2] for customized cloud and installation images also uses the newest FAI version.

[1] https://fai-project.org/fai-cd/

[2] https://fai-project.org/FAIme

FAI Ubuntu

Niels Thykier: Making debug symbols discoverable and fetchable

Sunday 17th of February 2019 09:03:31 PM

Michael wrote a few days ago about the experience of debugging programs on Debian.  And he is certainly not the only one, who found it more difficult to find debug symbols on Linux systems in general.

But fortunately, it is a fixable problem.  Basically, we just need a service to map a build-id to a downloadable file containing that build-id.  You can find the source code to my (prototype) of such a dbgsym service on salsa.debian.org.

It exposes one API endpoint, “/api/v1/find-dbgsym”, which accepts a build-id and returns some data about that build-id (or HTTP 404 if we do not know the build-id).  An example:

$ curl --silent http://127.0.0.1:8000/api/v1/find-dbgsym/5e625512829cfa9b98a8d475092658cb561ad0c8/ | python -m json.tool { "package": { "architecture": "amd64", "build_ids": [ "5e625512829cfa9b98a8d475092658cb561ad0c8" ], "checksums": { "sha1": "a7a38b49689031dc506790548cd09789769cfad3", "sha256": "3706bbdecd0975e8c55b4ba14a481d4326746f1f18adcd1bd8abc7b5a075679b" }, "download_size": 18032, "download_urls": [ "https://snapshot.debian.org/archive/debian-debug/20161028T101957Z/pool/main/6/6tunnel/6tunnel-dbgsym_0.12-1_amd64.deb" ], "name": "6tunnel-dbgsym", "version": "1:0.12-1" } }

Notice how it includes a download URL and a SHA256 checksum, so with this you can download the package containing the build-id directly from this and verify the download. The sample_client.py included in the repo does that and might be a useful basis for others interested in developing a client for this service.

 

To seed the database, so it can actually answer these queries, there is a bulk importer that parses Packages files from the Debian archive (for people testing: the ones from debian-debug archive are usually more interesting as they have more build-ids).

Possible improvements
  • Have this service deployed somewhere on the internet rather than the loopback interface on my machine.
  • The concept is basically distribution agnostic (Michael’s post in fact links to a similar service) and this could be a standard service/tool for all Linux distributions (or even just organizations).  I am happy to work with people outside Debian to make the code useful for their distribution (including non-Debian based distros).
    • The prototype was primarily based around Debian because it was my point of reference (plus what I had data for).
  • The bulk importer could (hopefully) be faster on new entries.
  • The bulk importer could import the download urls as well, so we do not have to fetch the relevant data online the first time when people are waiting.
  • Most of the django code / setup could probably have been done better as this has been an excuse to learn django as well.

Patches and help welcome.

Kudos

This prototype would not have been possible without python3, django, django’s restframework, python’s APT module, Postgresql, PoWA and, of course, snapshot.debian.org (including its API).

Birger Schacht: Sway in experimental

Sunday 17th of February 2019 07:52:51 PM

A couple of days ago the 1.0-RC2 version of Sway, a Wayland compositor, landed in Debian experimental. Sway is a drop in replacement for the i3 tiling window manager for wayland. Drop in replacement means that, apart from minor adaptions, you can reuse your existing i3 configuration file for Sway. On the Website of sway you can find a short introduction video that shows the most basic concepts of using Sway, though if you have worked with i3 you will feel at home soon.

In the video the utility swaygrab is mentioned, but this tool is not part of Sway anymore. There is another screenshot tool now though, called grim which you can combine with the tool slurp if you want to select regions for screenshots. The video also mentions swaylock, which is a screen locking utility similar to i3lock. It was split out of the main Sway release a couple of weeks ago but there also exists a Debian package by now. And there is a package for swayidle, which is a idle management daemon, which comes handy for locking the screen or for turning of your display after a timeout. If you need clipboard manager, you can use wl-clipboard. There is also a notification daemon called mako (the Debian package is called mako-notifier and is in NEW) and if you don’t like the default swaybar, you can have a look at waybar (not yet in Debian, see this RFS). If you want to get in touch with other Sway users there is a #sway IRC channel on freenode. For some tricks setting up Sway you can browse the wiki.

If you want to try Sway, beware that is is a release candiate and there are still bugs. I’m using Sway since a couple of month and though i had crashes when it still was the 1.0-beta.1 i hadn’t any since beta.2. But i’m using a pretty conservative setup.

Sway was started by Drew DeVault who is also the upstream maintainer of wlroots, the Wayland compositor library Sway is using and who some might now from his sourcehut project (LWN Article). He also just published an article about Wayland misconceptions. The upstream of grim, slurp and mako is Simon Ser, who also contributes to sway. A lot of thanks for the Debian packaging is due to nicoo who did most of the heavy lifting and to Sean for having patience when reviewing my contributions. Also thanks to Guido for maintaining wlroots!

Andrew Cater: Debian 9.8 released : Desktop environments and power settings

Sunday 17th of February 2019 03:12:40 PM
Debian 9.8 - the latest update to Debian Stretch - was released yesterday. Updated installation media can be found at the Debian CD Netnstall page, for example, at As part of the testing, I was using a very old i686 laptop which powers down at random intervals because the battery is old. It tends to suspend almost immediately. I found that the power management settings for the Cinnamon desktop were hart to find: using a Mate disktop allowed me to find the appropriate settings in the Debian menu layout much more easily Kudos to Steve McIntyre and Andy Simpkins (amongst others) for such a good job producing and testing the Debian CDs

Jonathan Dowland: embedding Haskell in AsciiDoc

Saturday 16th of February 2019 10:50:42 PM

I'm a fan of the concept of Literate Programming (I explored it a little in my Undergraduate Dissertation a long time ago) which can be briefly (if inadequately) summarised as follows: the normal convention for computer code is by default the text within a source file is considered to be code; comments (or, human-oriented documentation) are exceptional and must be demarked in some way (such as via a special symbol). Literate Programming (amongst other things) inverts this. By default, the text in a source file is treated as comments and ignored by the compiler, code must be specially delimited.

Haskell has built-in support for this scheme: by naming your source code files .lhs, you can make use of one of two conventions for demarking source code: either prefix each source code line with a chevron (called Bird-style, after Richard Bird), or wrap code sections in a pair of delimiters \begin{code} and \end{code} (TeX-style, because it facilitates embedding Haskell into a TeX-formatted document).

For various convoluted reasons I wanted to embed Haskell into an AsciiDoc-formatted document and I couldn't use Bird-style literate Haskell, which would be my preference. The AsciiDoc delimiter for a section of code is a line of dash symbols, which can be interleaved with the TeX-style delimiters:

------------ \begin{code} next a = if a == maxBound then minBound else succ a \end{code} ------------

Unfortunately the Tex-style delimiters show up in the output once the AsciiDoc is processed. Luckily, we can swap the order of the AsciiDoc and Literate-Haskell delimiters, because the AsciiDoc ones are treated as a source-code comment by Haskell and ignored. This moves the visible TeX-style delimiters out of the code block, which is a minor improvement:

\begin{code} ------------ next a = if a == maxBound then minBound else succ a ------------ \end{code}

We can disguise the delimiters outside of the code block further by defining an empty AsciiDoc macro called "code". Macros are marked up with surrounding braces, leaving just stray \begin and \end tokens in the text. Towards the top of the AsciiDoc file, in the pre-amble:

= Document title Document author :code:

This could probably be further improved by some AsciiDoc markup to change the style of the text outside of the code block immediately prior to the \begin token (perhaps make the font 0pt or the text colour the same as the background colour) but this is legible enough for me, for now.

The resulting file can be fed to an AsciiDoc processor (like asciidoctor, or intepreted by GitHub's built-in AsciiDoc formatter) and to a Haskell compiler. Unfortunately GitHub insists on a .adoc extension to interpret the file as AsciiDoc; GHC insists on a .lhs extension to interpret it as Literate Haskell (who said extensions were semantically meaningless these days…). So I commit the file as .adoc for GitHub's benefit and maintain a local symlink with a .lhs extension for my own.

Finally, I am not interested in including some of the Haskell code in my document that I need to include in the file in order for it to work as Haskell source. This can be achieved by changing from the code delimiter to AsciiDoc comment delimeters on the outside:

//////////// \begin{code} utilityFunction = "necessary but not interesting for the document" \end{code} ////////////

You can see an example of a combined AsciiDoc-Haskell file here (which is otherwise a work in progress):

https://github.com/jmtd/striot/blob/0f40d110f366ccfe8c4f07b76338ce215984113b/writeup.adoc

Vasudev Kamath: Note to Self: Growing the Root File System on First Boot

Saturday 16th of February 2019 04:43:00 PM

These 2 are the use cases I came across for expanding root file system.

  1. RaspberryPi images which comes in smaller image size like 1GB which you write bigger size SD cards like 32GB but you want to use full 32GB of space when system comes up.
  2. You have a VM image which is contains basic server operating system and you want to provision the same as a VM with much larger size root file system.

My current use case was second but I learnt the trick from 1, that is the RaspberryPi3 image spec by Debian project.

Idea behind the expanding root file system is first expanding the root file system to full available size and then run resize2fs on the expanded partition to grow file system. resize2fs is a tool specific for ext2/3/4 file system. But this needs to be done before the file system is mounted.

Here is my modified script from raspi3-image-spec repo. Only difference is I've changed the logic of extracting root partition device to my need, and of course added comments based on my understanding.

#!/bin/sh # Just extracts root partition and removes partition number to get the device # name eg. /dev/sda1 becomes /dev/sda roottmp=$(lsblk -l -o NAME,MOUNTPOINT | grep '/$') rootpart=/dev/${roottmp%% */} rootdev=${rootpart%1} # Use sfdisk to extend partition to all available free space on device. flock $rootdev sfdisk -f $rootdev -N 2 <<EOF ,+ EOF sleep 5 # Wait for all pending udev events to be handled udevadm settle sleep 5 # detect the changes to partition (we extended it). flock $rootdev partprobe $rootdev # remount the root partition in read write mode mount -o remount,rw $rootpart # Finally grow the file system on root partition resize2fs $rootpart exit 0fs

raspi3-image-spec uses sytemd service file to execute this script just before any file system is mounted. This is done by a making service execute before local-fs.pre target. From the man page for systemd.special

local-fs.target
systemd-fstab-generator(3) automatically adds dependencies of type Before= to all mount units that refer to local mount points for this target unit. In addition, it adds dependencies of type Wants= to this target unit for those mounts listed in /etc/fstab that have the auto mount option set.

Service also disables itself on executing to avoid re-runs on every boot. I've used the service file from raspi3-image-spec as is.

Testing with VM

raspi3-image-spec is well tested, but I wanted to make sure this works with my use case for VM. Since I didn't have any spare physical disks to experiment with I used kpartx with raw file images. Here is what I did

  1. Created a stretch image using vmdb2 with grub installed. Image size is 1.5G
  2. I created another raw disk using fallocate of 4G size.
  3. I created a partition on 4G disk.
  4. Loop mounted the disk and wrote 1G image on it using dd
  5. Finally created a VM using virt-install with this loop mounted device as root disk.

Below is my vmdb configuration yml again derived from raspi3-image-spec one with some modifications to suit my needs.

# See https://wiki.debian.org/RaspberryPi3 for known issues and more details. steps: - mkimg: "{{ output }}" size: 1.5G - mklabel: msdos device: "{{ output }}" - mkpart: primary device: "{{ output }}" start: 0% end: 100% tag: / - kpartx: "{{ output }}" - mkfs: ext4 partition: / label: RASPIROOT - mount: / - unpack-rootfs: / - debootstrap: stretch mirror: http://localhost:3142/deb.debian.org/debian target: / variant: minbase components: - main - contrib - non-free unless: rootfs_unpacked # TODO(https://bugs.debian.org/877855): remove this workaround once # debootstrap is fixed - chroot: / shell: | echo 'deb http://deb.debian.org/debian buster main contrib non-free' > /etc/apt/sources.list apt-get update unless: rootfs_unpacked - apt: install packages: - ssh - parted - dosfstools - linux-image-amd64 tag: / unless: rootfs_unpacked - grub: bios tag: / - cache-rootfs: / unless: rootfs_unpacked - shell: | echo "experimental" > "${ROOT?}/etc/hostname" # '..VyaTFxP8kT6' is crypt.crypt('raspberry', '..') sed -i 's,root:[^:]*,root:..VyaTFxP8kT6,' "${ROOT?}/etc/shadow" sed -i 's,#PermitRootLogin prohibit-password,PermitRootLogin yes,g' "${ROOT?}/etc/ssh/sshd_config" install -m 644 -o root -g root fstab "${ROOT?}/etc/fstab" install -m 644 -o root -g root eth0 "${ROOT?}/etc/network/interfaces.d/eth0" install -m 755 -o root -g root rpi3-resizerootfs "${ROOT?}/usr/sbin/rpi3-resizerootfs" install -m 644 -o root -g root rpi3-resizerootfs.service "${ROOT?}/etc/systemd/system" mkdir -p "${ROOT?}/etc/systemd/system/systemd-remount-fs.service.requires/" ln -s /etc/systemd/system/rpi3-resizerootfs.service "${ROOT?}/etc/systemd/system/systemd-remount-fs.service.requires/rpi3-resizerootfs.service" install -m 644 -o root -g root rpi3-generate-ssh-host-keys.service "${ROOT?}/etc/systemd/system" mkdir -p "${ROOT?}/etc/systemd/system/multi-user.target.requires/" ln -s /etc/systemd/system/rpi3-generate-ssh-host-keys.service "${ROOT?}/etc/systemd/system/multi-user.target.requires/rpi3-generate-ssh-host-keys.service" rm -f ${ROOT?}/etc/ssh/ssh_host_*_key* root-fs: / # Clean up archive cache (likely not useful) and lists (likely outdated) to # reduce image size by several hundred megabytes. - chroot: / shell: | apt-get clean rm -rf /var/lib/apt/lists # TODO(https://github.com/larswirzenius/vmdb2/issues/24): remove once vmdb # clears /etc/resolv.conf on its own. - shell: | rm "${ROOT?}/etc/resolv.conf" root-fs: /

I could not run with vmdb2 installed from Debian archive, so I cloned raspi3-image-spec and used vmdb2 submodule from it. And here are rest of commands used for testing the script.

fallocate -l 4G rootdisk.img # Create one partition with full disk sfdisk -f rootdisk.img <<EOF ,+ EOF kpartx -av rootdisk.img # mounts on /dev/loop0 for me dd if=vmdb.img of=/dev/loop0 sudo virt-install --name experimental --memory 1024 --disk path=/dev/loop0 --controller type=scsi,model=virtio-scsi --boot hd --network bridge=lxcbr0

Once VM booted I could see the root file system is 4G of size instead of 1.5G it was after using dd to write image on to it. So success!.

Ben Hutchings: Debian LTS work, January 2019

Saturday 16th of February 2019 04:01:28 PM

I was assigned 20 hours of work by Freexian's Debian LTS initiative and carried over 5 hours from December. I worked 24 hours and so will carry over 1 hour.

I prepared another stable update for Linux 3.16 (3.16.63), but did not upload a new release yet.

I also raised the issue that the installer images for Debian 8 "jessie" would need to be updated to include a fix for CVE-2019-3462.

David Moreno: Dell XPS 13 9380

Saturday 16th of February 2019 03:17:47 PM

Got myself a XPS 13” 9380 that Dell just released as “Developer Edition” with Ubuntu 18.04 LTS pre-installed. They just released it on January 2019.

Ubuntu didn’t last long though. I prefer OS X Mojave than any of the Ubuntu installations. It’s okay though, it’s just not for me.

Big big big shoutout to Jérémy Garrouste for his documentation only a few days ago of the Debian installation for Buster on the laptop which helped me figure out I needed to upgrade the BIOS to fix a couple of issues and be able to run the d-i alpha 5 with the Atheros drivers. Time to dust a few things here and there :)

Well that’s not a picture of the alpha d-i but from the first time I tried to install before upgrading the BIOS.

Steve Kemp: Updated myy compiler, and bought a watch.

Saturday 16th of February 2019 02:45:46 PM

The simple math-compiler I introduced in my previous post has had a bit of an overhaul, so that now it is fully RPN-based.

Originally the input was RPN-like, now it is RPN for real. It handles error-detection at run-time, and generates a cleaner assembly-language output:

In other news I bought a new watch, which was a fun way to spend some time.

I love mechanical watches, clocks, and devices such as steam-engines. While watches are full of tiny and intricate parts I like the pretence that you can see how they work, and understand them. Steam engines are seductive because their operation is similar; you can almost look at them and understand how they work.

I've got a small collection of watches at the moment, ranging from €100-€2000 in price, these are universally skeleton-watches, or open-heart watches.

My recent purchase is something different. I was looking at used Rolexs, and found some from 1970s. That made me suddenly wonder what had been made the same year as I was born. So I started searching for vintage watches, which had been manufactured in 1976. In the end I found a nice Soviet Union piece, made by Raketa. I can't prove that this specific model was actually manufactured that year, but I'll keep up the pretence. If it is +/- 10 years that's probably close enough.

My personal dream-watch is the Rolex Oyster (I like to avoid complications). The Oyster is beautiful, and I can afford it. But even with insurance I'd feel too paranoid leaving the house with that much money on my wrist. No doubt I'll find a used one, for half that price, sometime. I'm not in a hurry.

(In a horological-sense a "complication" is something above/beyond the regular display of time. So showing the day, the date, or phase of the moon would each be complications.)

Molly de Blanc: pancakes

Saturday 16th of February 2019 01:31:24 PM

My father and Fanny Farmer taught me how to make pancakes. Fanny Farmer provided a basic recipe, and Peter de Blanc filled in the details and the important things she missed — the texture of the batter, how you know when it’s time to flip them, the repeatedly emphasized importance of butter.

Ingredients
  • 1 1/2 cup flour
  • 2 1/2 teaspoons baking powder
  • 3 tablespoons sweetener (optional, see note below)
  • 3/4 teaspoon salt
  • 1 egg (slightly beaten)
  • 2 cups milk
  • 3 tablespoons butter (melted)

Why is the sweetener optional? I think if you sweeten the pancake recipe, it’s enough that you don’t want to cover it in maple syrup, jam, etc, etc. So, I usually go without sugar or honey or putting the maple right in, unless I’m making these to eat on the go. ALSO! If you don’t use sugar, you can make them savory.

Make the batter
  1. Mix all the dry ingredients (flour, baking powder, salt, and sweetener if you’re using sugar).
  2. Add most of the wet ingredients (egg, milk, and sweetener if you’re using honey or maple syrup).
  3. Mix together.
  4. Melt the butter in your pan, so the pan gets full of butter. Yum.
  5. While stirring your batter, add in the melted butter slowly.
Cooking the pancakes Bubbles slowly forming

Okay, this is the hardest part for me because it requires lots of patience, which I don’t have enough of.

A  few tips:

  • Shamelessly use lots of butter to keep your pancakes from sticking. It will also help them taste delicious.
  • After  you put the pancakes in the pan, they need to cook at a medium temperature for a fairly long time. You want them to be full of little holes and mostly solid before you flip them over. (See photos.)
  • I recommend listening to music and taking dancing breaks.

How to cook pancakes:

Almost ready to flip!
  1. Over a medium heat, use your nice, hot, buttery pan.
  2. Use a quarter cup of batter for each pancake. Based on the size of your pan, you should make three – four pancakes per batch.
  3. Let cook for a while. As mentioned above, they should be mostly cooked on the top as well. There ought to be a ring of crispy pancake around the outside (see pictures), but if there’s not it’s still okay!
  4. Flip the pancakes!
  5. Let cook a little bit longer, until the bottom has that pretty golden brown color to match the top.

And that’s it. Eat your pancakes however you’d like. The day I wrote this recipe I heated some maple syrup with vanilla, bourbon, and cinnamon. Yummmmm.

P.S. I tagged this post “free software” so it would show up in Planet Debian because I was asked to include a baking post there. This isn’t quite baking, but it’s still pretty good.

Gunnar Wolf: Debian on the Raspberryscape: Great news!

Saturday 16th of February 2019 03:25:53 AM

I already mentioned here having adopted and updated the Raspberry Pi 3 Debian Buster Unofficial Preview image generation project. As you might know, the hardware differences between the three families are quite deep — The original Raspberry Pi (models A and B), as well as the Zero and Zero W, are ARMv6 (which, in Debian-speak, belong to the armel architecture, a.k.a. EABI / Embedded ABI). Raspberry Pi 2 is an ARMv7 (so, we call it armhf or ARM hard-float, as it does support floating point instructions). Finally, the Raspberry Pi 3 is an ARMv8-A (in Debian it corresponds to the ARM64 architecture).

The machines are quite different, but being the CPUs provided by Broadcom, they all share a strange bootloader requirement, as the GPU must be initialized to kickstart the CPU (and only then can Linux be started), hence, they require non-free firmware

Anyway, the image project was targetted at model 3 Raspberries. However...

Thanks (again!) to Romain Perier, I got word that the "lesser" Raspberries can be made to boot from Debian proper, after they are initialized with this dirty, ugly firmware!

I rebuilt the project, targeting armhf instead of arm64. Dropped an extra devicetree blob on the image, to help Linux understand what is connected to the RPI2. Flashed it to my so-far-trusty SD. And... Behold! On the photo above, you can appreciate the Raspberry Pi 2 booting straight Debian, no Raspbian required!

As for the little guy, the Zero that sits atop them, I only have to upload a new version of raspberry3-firmware built also for armel. I will add to it the needed devicetree files. I have to check with the release-team members if it would be possible to rename the package to simply raspberry-firmware (as it's no longer v3-specific).

Why is this relevant? Well, the Raspberry Pi is by far the most popular ARM machine ever. It is a board people love playing with. It is the base for many, many, many projects. And now, finally, it can run with straight Debian! And, of course, if you don't trust me providing clean images, you can prepare them by yourself, trusting the same distribution you have come to trust and love over the years.

Wheee!

AttachmentSize IMG_20190215_194614.jpg4.11 MB

Chris Lamb: Book Review: Stories of Your Life and Others

Friday 15th of February 2019 11:10:33 PM

Stories of Your Life and Others (2002)

Ted Chiang

This compilation has been enjoying a renaissance in recent years due the success of the film Arrival (2016) which based on on the fourth and titular entry in this amazing collection. Don't infer too much from that however as whilst this is prima facie just another set of sci-fi tales, it is science fiction in the way that Children of Men is, rather than Babylon 5.

A well-balanced mixture of worlds are evoked throughout with a combination of tales that variously mix the Aristotelian concepts of spectacle (opsis), themes (dianoia), character (ethos) and dialogue (lexis), perhaps best expressed practically in that some stories were extremely striking at the time — one even leading me to rebuff an advance at a bar — and a number were not as remarkable at the time yet continue to occupy my idle thoughts.

The opening tale which reworks the Tower of Babel into a construction project probably remains my overall favourite, but the Dark Materials-esque world summoned in Seventy-Two Letters continues to haunt my mind and lips of anyone else who has happened to come across it, perhaps becoming the quite-literal story of my life for a brief period. Indeed it could be said that, gifted as a paperback, whilst the whole collection followed me around across a number of locales, it continues to follow me — figuratively speaking that is — to this day.

Highly recommended to all readers but for those who enjoy discussing books with others it would more than repay any investment.

Gunnar Wolf: Spinning rust woes: Cowbuilder

Friday 15th of February 2019 06:37:39 PM

Wow.
I use a traditional, spinning rust hard drive as my main volume:

  1. $ /dev/sda2 on / type btrfs (rw,relatime,compress=zlib:3,space_cache,subvolid=5,subvol=/)

I just adopted Lars' vmdb2. Of course, I was eager to build and upload my first version and... Hit a FTBFS bug due to missing dependencies... Bummer!
So I went to my good ol' cowbuilder (package cowdancer)to fix whatever needed fixing. But it took long! Do note that my /var/cache/pbuilder/base.cow was already set up and updated.
  1. # time cowbuilder --build /home/gwolf/vcs/build-area/vmdb2_0.13.2+git20190215-1.dsc
  2. (...)
  3. real 15m55.403s
  4. user 0m53.734s
  5. sys 0m23.138s

But... What if I take the spinning rust out of the equation?
  1. # mkdir /var/cache/pbuilder
  2. # mount none -t tmpfs /var/cache/pbuilder
  3. # time rsync -a /var/cache/pbuilderbk/* /var/cache/pbuilder
  4.  
  5. real 0m5.363s
  6. user 0m2.333s
  7. sys 0m0.709s
  8. # time cowbuilder --build /home/gwolf/vcs/build-area/vmdb2_0.13.2+git20190215-1.dsc
  9. (...)
  10. real 0m52.586s
  11. user 0m53.076s
  12. sys 0m8.277s

Close to ¹/₁₆th of the running time — Even including the copy of the base.cow!

OK, I cheated a bit before the rsync, as my cache was already warm... But still, convenient!

Michael Stapelberg: Debugging experience in Debian

Friday 15th of February 2019 12:11:01 PM

Recently, a user reported that they don’t see window titles in i3 when running i3 on a Raspberry Pi with Debian.

I copied the latest Raspberry Pi Debian image onto an SD card, booted it, and was able to reproduce the issue.

Conceptually, at this point, I should be able to install and start gdb, set a break point and step through the code.

Enabling debug symbols in Debian

Debian, by default, strips debug symbols when building packages to conserve disk space and network bandwidth. The motivation is very reasonable: most users will never need the debug symbols.

Unfortunately, obtaining debug symbols when you do need them is unreasonably hard.

We begin by configuring an additional apt repository which contains automatically generated debug packages:

raspi# cat >>/etc/apt/sources.list.d/debug.list <<'EOT' deb http://deb.debian.org/debian-debug buster-debug main contrib non-free EOT raspi# apt update

Notably, not all Debian packages have debug packages. As the DebugPackage Debian Wiki page explains, debhelper/9.20151219 started generating debug packages (ending in -dbgsym) automatically. Packages which have not been updated might come with their own debug packages (ending in -dbg) or might not preserve debug symbols at all!

Now that we can install debug packages, how do we know which ones we need?

Finding debug symbol packages in Debian

For debugging i3, we obviously need at least the i3-dbgsym package, but i3 uses a number of other libraries through whose code we may need to step.

The debian-goodies package ships a tool called find-dbgsym-packages which prints the required packages to debug an executable, core dump or running process:

raspi# apt install debian-goodies raspi# apt install $(find-dbgsym-packages $(which i3))

Now we should have symbol names and line number information available in gdb. But for effectively stepping through the program, access to the source code is required.

Obtaining source code in Debian

Naively, one would assume that apt source should be sufficient for obtaining the source code of any Debian package. However, apt source defaults to the package candidate version, not the version you have installed on your system.

I have addressed this issue with the pk4 tool, which defaults to the installed version.

Before we can extract any sources, we need to configure yet another apt repository:

raspi# cat >>/etc/apt/sources.list.d/source.list <<'EOT' deb-src http://deb.debian.org/debian buster main contrib non-free EOT raspi# apt update

Regardless of whether you use apt source or pk4, one remaining problem is the directory mismatch: the debug symbols contain a certain path, and that path is typically not where you extracted your sources to. While debugging, you will need to tell gdb about the location of the sources. This is tricky when you debug a call across different source packages:

(gdb) pwd Working directory /usr/src/i3. (gdb) list main 229 * the main loop. */ 230 ev_unref(main_loop); 231 } 232 } 233 234 int main(int argc, char *argv[]) { 235 /* Keep a symbol pointing to the I3_VERSION string constant so that 236 * we have it in gdb backtraces. */ 237 static const char *_i3_version __attribute__((used)) = I3_VERSION; 238 char *override_configpath = NULL; (gdb) list xcb_connect 484 ../../src/xcb_util.c: No such file or directory.

See Specifying Source Directories in the gdb manual for the dir command which allows you to add multiple directories to the source path. This is pretty tedious, though, and does not work for all programs.

Positive example: Fedora

While Fedora conceptually shares all the same steps, the experience on Fedora is so much better: when you run gdb /usr/bin/i3, it will tell you what the next step is:

# gdb /usr/bin/i3 […] Reading symbols from /usr/bin/i3...(no debugging symbols found)...done. Missing separate debuginfos, use: dnf debuginfo-install i3-4.16-1.fc28.x86_64

Watch what happens when we run the suggested command:

# dnf debuginfo-install i3-4.16-1.fc28.x86_64 enabling updates-debuginfo repository enabling fedora-debuginfo repository […] Installed: i3-debuginfo.x86_64 4.16-1.fc28 i3-debugsource.x86_64 4.16-1.fc28 Complete!

A single command understood our intent, enabled the required repositories and installed the required packages, both for debug symbols and source code (stored in e.g. /usr/src/debug/i3-4.16-1.fc28.x86_64). Unfortunately, gdb doesn’t seem to locate the sources, which seems like a bug to me.

One downside of Fedora’s approach is that gdb will only print all required dependencies once you actually run the program, so you may need to run multiple dnf commands.

In an ideal world

Ideally, none of the manual steps described above would be necessary. It seems absurd to me that so much knowledge is required to efficiently debug programs in Debian. Case in point: I only learnt about find-dbgsym-packages a few days ago when talking to one of its contributors.

Installing gdb should be all that a user needs to do. Debug symbols and sources can be transparently provided through a lazy-loading FUSE file system. If our build/packaging infrastructure assured predictable paths and automated debug symbol extraction, we could have transparent, quick and reliable debugging of all programs within Debian.

NixOS’s dwarffs is an implementation of this idea: https://github.com/edolstra/dwarffs

Conclusion

While I agree with the removal of debug symbols as a general optimization, I think every Linux distribution should strive to provide an entirely transparent debugging experience: you should not even have to know that debug symbols are not present by default. Debian really falls short in this regard.

Getting Debian to a fully transparent debugging experience requires a lot of technical work and a lot of social convincing. In my experience, programmatically working with the Debian archive and packages is tricky, and ensuring that all packages in a Debian release have debug packages (let alone predictable paths) seems entirely unachievable due to the fragmentation of packaging infrastructure and holdouts blocking any progress.

My go-to example is rsync’s debian/rules, which intentionally (!) still has not adopted debhelper. It is not a surprise that there are no debug symbols for rsync in Debian.

More in Tux Machines

Server: HTTP Clients, IIS DDoS and 'DevOps' Hype From Red Hat

  • What are good command line HTTP clients?
    The whole is greater than the sum of its parts is a very famous quote from Aristotle, a Greek philosopher and scientist. This quote is particularly pertinent to Linux. In my view, one of Linux’s biggest strengths is its synergy. The usefulness of Linux doesn’t derive only from the huge raft of open source (command line) utilities. Instead, it’s the synergy generated by using them together, sometimes in conjunction with larger applications. The Unix philosophy spawned a “software tools” movement which focused on developing concise, basic, clear, modular and extensible code that can be used for other projects. This philosophy remains an important element for many Linux projects. Good open source developers writing utilities seek to make sure the utility does its job as well as possible, and work well with other utilities. The goal is that users have a handful of tools, each of which seeks to excel at one thing. Some utilities work well independently. This article looks at 4 open source command line HTTP clients. These clients let you download files over the internet from the command line. But they can also be used for many more interesting purposes such as testing, debugging and interacting with HTTP servers and web applications. Working with HTTP from the command-line is a worthwhile skill for HTTP architects and API designers. If you need to play around with an API, HTTPie and curl will be invaluable.
  • Microsoft publishes security alert on IIS bug that causes 100% CPU usage spikes
    The Microsoft Security Response Center published yesterday a security advisory about a denial of service (DOS) issue impacting IIS (Internet Information Services), Microsoft's web server technology.
  • 5 things to master to be a DevOps engineer
    There's an increasing global demand for DevOps professionals, IT pros who are skilled in software development and operations. In fact, the Linux Foundation's Open Source Jobs Report ranked DevOps as the most in-demand skill, and DevOps career opportunities are thriving worldwide. The main focus of DevOps is bridging the gap between development and operations teams by reducing painful handoffs and increasing collaboration. This is not accomplished by making developers work on operations tasks nor by making system administrators work on development tasks. Instead, both of these roles are replaced by a single role, DevOps, that works on tasks within a cooperative team. As Dave Zwieback wrote in DevOps Hiring, "organizations that have embraced DevOps need people who would naturally resist organization silos."

Purism's Privacy and Security-Focused Librem 5 Linux Phone to Arrive in Q3 2019

Initially planned to ship in early 2019, the revolutionary Librem 5 mobile phone was delayed for April 2019, but now it suffered just one more delay due to the CPU choices the development team had to make to deliver a stable and reliable device that won't heat up or discharge too quickly. Purism had to choose between the i.MX8M Quad or the i.MX8M Mini processors for their Librem 5 Linux-powered smartphone, but after many trials and errors they decided to go with the i.MX8M Quad CPU as manufacturer NXP recently released a new software stack solving all previous power consumption and heating issues. Read more

Qt Creator 4.9 Beta released

We are happy to announce the release of Qt Creator 4.9 Beta! There are many improvements and fixes included in Qt Creator 4.9. I’ll just mention some highlights in this blog post. Please refer to our change log for a more thorough overview. Read more

Hack Week - Browsersync integration for Online

Recently my LibreOffice work is mostly focused on the Online. It's nice to see how it is growing with new features and has better UI. But when I was working on improving toolbars (eg. folding menubar or reorganization of items) I noticed one annoying thing from the developer perspective. After every small change, I had to restart the server to provide updated content for the browser. It takes few seconds for switching windows, killing old server then running new one which requires some tests to be passed. Last week during the Hack Week funded by Collabora Productivity I was able to work on my own projects. It was a good opportunity for me to try to improve the process mentioned above. I've heard previously about browsersync so I decided to try it out. It is a tool which can automatically reload used .css and .js files in all browser sessions after change detection. To make it work browsersync can start proxy server watching files on the original server and sending events to the browser clients if needed. Read more