Domain > www.happyassassin.net

More information on this domain is in AlienVault OTX

DNS Resolutions

Date	IP Address
2023-03-18	89.117.9.26 (ClassC)
2025-01-17	172.67.150.236 (ClassC)

HTTP/1.1 200 OKDate: Fri, 17 Jan 2025 23:45:25 GMTContent-Type: text/html; charsetutf-8Transfer-Encoding: chunkedConnection: keep-aliveAccess-Control-Allow-Origin: *Cache-Control: public, max-age0, must-revalidatereferrer-policy: strict-origin-when-cross-originx-content-type-options: nosniffReport-To: {endpoints:{url:https:\/\/a.nel.cloudflare.com\/report\/v4?seejI60LhXou%2BSLLT%2FAYBnBPZ21d4hjBM%2FM2sI4zSABCGu44znRZw7vxy2JBWdzeUQFfXuZvXGL716jIib0Kl2qGG0lTjK9V23aBQZGja20m%2B%2B5na5FLW6EwdrRd0RZafU2ipItYFhTA%3D},group:cf-nel,max_age:604800}NEL: {success_fraction:0,report_to:cf-nel,max_age:604800}Vary: Accept-Encodingcf-cache-status: DYNAMICServer: cloudflareCF-RAY: 903a38086e47ebee-SEAalt-svc: h3:443; ma86400server-timing: cfL4;desc?protoTCP&rtt9189&min_rtt9088&rtt_var2623&sent5&recv6&lost0&retrans0&sent_bytes2855&recv_bytes732&delivery_rate318661&cwnd252&unsent_bytes0&cid2f73cdd4ce3df229&ts106&x0

!DOCTYPE html>html prefixog: http://ogp.me/ns# article: http://ogp.me/ns/article# langen>head>meta charsetutf-8>meta namedescription contentAdamW on Linux and more>meta nameviewport contentwidthdevice-width, initial-scale1>title>AdamW on Linux and more/title>link hrefassets/css/bootstrap.min.css relstylesheet typetext/css>link hrefassets/css/baguetteBox.min.css relstylesheet typetext/css>link hrefassets/css/rst.css relstylesheet typetext/css>link hrefassets/css/code.css relstylesheet typetext/css>link hrefassets/css/theme.css relstylesheet typetext/css>link hrefassets/css/bootblog.css relstylesheet typetext/css>link hrefhttps://fonts.googleapis.com/css?familyPlayfair+Display:700,900 relstylesheet>meta nametheme-color content#5670d4>meta namegenerator contentNikola (getnikola.com)>link relalternate typeapplication/rss+xml titleRSS hreflangen hrefrss.xml>link relalternate typeapplication/atom+xml titleAtom hreflangen hreffeed.atom>link relcanonical hrefhttps://www.happyassassin.net/>link relnext hrefindex-104.html typetext/html>!--if lt IE 9>script srcassets/js/html5.js>/script>!endif-->link relprefetch hrefposts/2025/01/14/new-laptop-and-silverblue-update/ typetext/html>/head>body>a href#content classsr-only sr-only-focusable>Skip to main content/a>!-- Header and menu bar -->div classcontainer>      header classblog-header py-3>div classrow nbb-header align-items-center>          div classcol-md-3 col-xs-2 col-sm-2 stylewidth: auto;>            button classnavbar-toggler navbar-light bg-light nbb-navbar-toggler typebutton data-togglecollapse data-target.bs-nav-collapsible aria-controlsbs-navbar aria-expandedfalse aria-labelToggle navigation>                span classnavbar-toggler-icon>/span>            /button>            div classcollapse bs-nav-collapsible bootblog4-search-form-holder>                            /div>        /div>          div classcol-md-6 col-xs-10 col-sm-10 bootblog4-brand stylewidth: auto;>            a classnavbar-brand blog-header-logo text-dark href.>            span idblog-title>AdamW on Linux and more/span>        /a>          /div>            div classcol-md-3 justify-content-end align-items-center bs-nav-collapsible collapse flex-collapse bootblog4-right-nav>            nav classnavbar navbar-light bg-white>ul classnavbar-nav bootblog4-right-nav>/ul>/nav>/div>    /div>/header>nav classnavbar navbar-expand-md navbar-light bg-white static-top>div classcollapse navbar-collapse bs-nav-collapsible idbs-navbar>            ul classnavbar-nav nav-fill d-flex w-100>li classnav-item>a hrefarchive.html classnav-link>Archives/a>                /li>li classnav-item>a hrefcategories/index.html classnav-link>Tags/a>                /li>li classnav-item>a hrefpages/about classnav-link>About/a>                /li>li classnav-item>a hrefpages/privacy-and-legal classnav-link>Privacy / Legal/a>                /li>li classnav-item>a hrefpages/contact classnav-link>Contact/a>                /li>li classnav-item>a hrefhttps://fosstodon.org/@adamw classnav-link>Mastodon @span class__cf_email__ data-cfemail0c6d686d617b4c6a637f7f786368636222637e6b>email protected/span>/a>                /li>li classnav-item>a hrefhttps://pagure.io/user/adamwill classnav-link>Pagure.io/a>                /li>li classnav-item>a hrefhttps://github.com/AdamWill classnav-link>Github/a>                /li>li classnav-item>a hrefrss.xml classnav-link>RSS feed/a>                /li>li classnav-item>a hreffeed.atom classnav-link>Atom feed/a>                            /li>/ul>/div>!-- /.navbar-collapse -->/nav>/div>div classcontainer idcontent rolemain>    div classbody-content>        !--Body content-->                            div classpostindex>            article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2025/01/14/new-laptop-and-silverblue-update/ classu-url>New laptop and Silverblue update/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2025/01/14/new-laptop-and-silverblue-update/ relbookmark>            time classpublished dt-published datetime2025-01-14T22:25:14Z itempropdatePublished title2025-01-14 22:25>2025-01-14 22:25/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>Figured Id post an update on how things are going with the new laptop (HP Omnibook Ultra 14, AMD Ryzen AI 9 365 Strix Point, for the searchers) and with Silverblue./p>p>I managed to work around the hub issue by swapping out the fancy $300 Thunderbolt hub for a a hrefhttps://www.amazon.ca/dp/B0DF7MF2D7>$40 USB-C hub off Amazon/a>. This comes with limitations - youre only going to get a single 4k 60Hz external display, and limited bandwidth for anything else - but its sufficient for my needs, and makes me regret buying the fancy hub in the first place. It seems to work 100% reliably on startup, reboot and across suspend/resume. Theres still clearly something wrong with Thunderbolt handling in the kernel, but its not my problem any more./p>p>The poor performance of some sites in Firefox turned out to be tied to the hanging problem - Id disabled graphics acceleration in Firefox, which helped with the hanging, but was causing the appalling performance on Google sites and others. Ive now cargo-culted a set of kernel args - code>amdgpu.dcdebugmask0x800 amdgpu.lockup_timeout100000 drm.vblankoffdelay0/code> - which em>seem/em> to be helping; I turned graphics acceleration back on in Firefox and it hasnt started hanging again. At least, I havent had random hangs for the last few days, and this morning I played a video on youtube and the system has not hung since then. Ive no idea how bad they are for battery life, but hey, they seem to be keeping things stable. So, the system is pretty workable at this point. Ive been using it full-time, havent had to go back to the old one./p>p>Im also feeling better about Silverblue as a main OS this time. A lot of things seem to have got better. The toolbox container experience is pretty smooth now. I managed to get adb working inside a container by putting a hrefhttps://github.com/M0Rf30/android-udev-rules>these udev rules/a> in /etc/udev/rules.d. It seems like I have to kill and re-start the adb server any time the phone disconnects or reboots - usually adb would keep seeing the phone just fine across those events - but its a minor inconvenience. I had to print something yesterday, was worried for a moment that Id have to figure out how to get hp-setup to do its thing, but then...Silverblue saw my ancient HP printer on the network, let me print to it, and it worked, all without any manual setup at all. It seems to be working over IPP, but Im a bit surprised, as the printer is from 2010 or 2011 and I dont think it worked before. But Im not complaining!/p>p>I havent had any real issues with app availability so far. All the desktop apps I need to use are available as flatpaks, and the toolbox container handles CLI stuff. Im running Firefox (baked-in version), Evolution, gedit, ptyxis (built-in), liferea, nheko, slack and vesktop (for discord) without any trouble. LibreOffice and GIMP flatpaks also work fine. Everythings really been pretty smooth./p>p>I do have a couple of tweaks in my bashrc (I put them in a file in code>~/.bashrc.d/code>, which is a neat invention) that other Atomic users might find useful.../p>div classcode>pre classcode literal-block>span classk>if/span>span classw> /span>span classp>/span>span classw> /span>span classo>-/span>span classn>n/span>span classw> /span>span classs2>$container/span>span classw> /span>span classp>/span>span classn>then/span>span classw>  /span>span classn>alias/span>span classw> /span>span classn>gedit/span>span classo>/span>span classs2>flatpak-spawn --host /var/lib/flatpak/exports/bin/org.gnome.gedit/span>span classw>  /span>span classn>alias/span>span classw> /span>span classn>xdg/span>span classo>-/span>span classn>open/span>span classo>/span>span classn>flatpak/span>span classo>-/span>span classn>xdg/span>span classo>-/span>span classn>open/span>span classk>else/span>span classw>  /span>span classn>alias/span>span classw> /span>span classn>gedit/span>span classo>//span>span classk>var/span>span classo>//span>span classn>lib/span>span classo>//span>span classn>flatpak/span>span classo>//span>span classn>exports/span>span classo>//span>span classn>bin/span>span classo>//span>span classn>org/span>span classo>./span>span classn>gnome/span>span classo>./span>span classn>gedit/span>span classn>fi/span>/pre>/div>p>the gedit aliases let me do code>gedit somefile/code> either inside or outside a container, and the file just opens in my existing gedit instance. Cant really live without that. You can adapt it for anything thats a flatpak app on the host. The xdg-open alias within containers similar makes code>xdg-open somefile/code> within the container do the same as it would outside the container./p>p>So its still early days, but Im optimistic Ill keep this setup this time. I might try rebasing to the bootc build soon./p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2025/01/08/new-laptop-experience-fedora-on-hp-omnibook-ultra-14-ryzen-ai-365-strix-point/ classu-url>New laptop experience (Fedora on HP Omnibook Ultra 14 - Ryzen AI 365, Strix Point)/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2025/01/08/new-laptop-experience-fedora-on-hp-omnibook-ultra-14-ryzen-ai-365-strix-point/ relbookmark>            time classpublished dt-published datetime2025-01-08T00:39:00Z itempropdatePublished title2025-01-08 00:39>2025-01-08 00:39/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>New year, new blog post! Fedoras going great...41 came out and seems to be getting good reviews, theres exciting stuff going on with atomic/bootc, were getting a a hrefhttps://communityblog.fedoraproject.org/fedora-chooses-forgejo/>new forge/a>, its an exciting time to be alive.../p>p>Personally Ive spent a large chunk of the last few weeks bashing my head against two awkward bugs - one a hrefhttps://bugzilla.redhat.com/show_bug.cgi?id2329581>kernel bug/a>, one a hrefhttps://github.com/systemd/systemd/issues/35499>systemd bug/a>. Its a bit of a slog, but hey. Also now working up to our next big openQA project, a hrefhttps://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/351>extending test coverage for the new installer/a>./p>p>But also! I bought myself a new laptop. For the last couple of years Ive been using a Dell XPS 13 9315, the Alder Lake generation. Ive been using various generations of XPS 13 ever since Sony stopped making laptops (previously I used a 2010 Vaio Z) - I always found it to be the best thin-and-light design, and this one was definitely that. But over time it felt really underpowered. Some of this is the fault of modern apps. I have to run a dumb amount of modern chat apps, and while theyre much nicer than IRC, they sure use a lot more resources than hexchat. Of course I have a browser with about 50 tabs open at all times, Evolution uses quite a lot of memory for my email setup for some reason, and I have to run VMs quite often for my work obviously. Put all that together, and...I was often running out of RAM despite having 16GB, which is pretty ridiculous. But even aside from that, you could tell the CPU was just struggling with everything. Just being in a video chat was hard work for it (if I switched apps too much while in a meeting, my audio would start chopping up for others on the call). Running more than two VMs tend to hang the system irretrievably. Just normal use often caused the fan to spin up pretty high. And the battery life wasnt great. It got better with kernel updates over time, but still only 3-4 hours probably./p>p>So I figured Id throw some hardware at the problem. Ive been following all the chipset releases over the last couple of years, and decided I wanted to get something with AMDs latest silicon, codenamed Strix Point, the Ryzen AI 3xx chips. Theyre not massively higher-performing than the previous gen, but the battery life seems to be improved, and they have somewhat better GPUs. That pretty much brought it down to the Asus Vivobook S 14, HP Omnibook Ultra 14, and Lenovo T14S gen 6 AMD. The Asus is stuck with 24GB of RAM max and Im not a huge Asus fan in general, and the HP came in like $600 cheaper than the Thinkpad with equivalent specs, and had a 3 year warranty included. So I went with the HP, with 1TB of storage and 32GB of RAM./p>p>I really like the system as a whole. Its heavier than the XPS 13, obviously, the bezels are a little bigger, and the screen is glossier. But the screen is pretty nice, I like the keyboard, and the overall build quality feels pretty solid. The trackpad seems fine./p>p>As for running Fedora (and Linux in general) on it...well, its em>almost/em> great. Everything more or less works out of the box, except the fingerprint reader. I dont care about that because I set up the reader on the XPS 13 and kinda hated it; its nowhere near as nice as a fingerprint reader on a phone. Even if it worked on the HP Id leave it off. The performance is fantastic (except that Google office sites perform weirdly terribly on Firefox, havent tried on Chromium yet)./p>p>But...after using it for a while, the issues become apparent. The first one I hit is that the system seems to hang pretty reproducibly playing video in browsers. This seems to be affecting pretty much everyone with a Strix Point system, and the only fix is to turn off hardware video acceleration in the browser, which isnt really great (it means playing video will use the CPU, hurting battery life and performance). Then I found even with that workaround applied the system would hang occasionally. Looking at a hrefhttps://gitlab.freedesktop.org/drm/amd/-/issues/?label_name%5B%5DStrix%20%2F%20Kraken>the list of Strix Point issues/a> on the AMD issue tracker, I found a few that recommended kernel parameters to disable various features of the GPU to work around this; Im running with code>amdgpu.dcdebugmask0x800/code>, which disables idle power states for the GPU, which probably hurts battery life pretty bad. Havent had a hang with that yet, but well see. But aside from that, Im em>also/em> having issues with docks. I have a a hrefhttps://www.caldigit.com/ts3-plus/>Caldigit TS3+/a>, which was probably overpowered for what I really need, but worked great with the XPS 13. I have a keyboard, camera, headset, ethernet and monitor connected to it. With the HP, I find that at encryption passphrase entry during boot (so, in the initramfs) the keyboard works fine, but once I reach the OS proper, only the monitor works. Nothing else attached to the dock works at all. A couple of times, suspending the system and resuming it seemed to make it start working - but then I tried that a couple em>more/em> times and it didnt work. I have another Caldigit dock in the basement; tried that, same deal. Then I tried my cheap no-name travel hub (which just has power pass-through, an HDMI port, and one USB-A port on it) with a USB-A hub, and...at first it worked fine! But then I suspended and resumed and the camera and headset stopped working. Keyboard still worked. Sigh. Ive ordered a mid-range hub with HDMI, ethernet, a card reader and four USB-A ports on it off Amazon, so I wont need the USB-A hub daisychain any more...Im hoping thatll work well enough. If not, its a bit awkward./p>p>So, so far its a bit of a frustrating experience. It could clearly be a fantastic Linux laptop, but it isnt quite one yet. Id probably recommend holding off for a bit while the upstream devs (hopefully) shake out all the bugs.../p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2024/11/19/adamws-debugging-adventures-inadvertent-extreme-optimization/ classu-url>AdamWs Debugging Adventures: Inadvertent Extreme Optimization/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2024/11/19/adamws-debugging-adventures-inadvertent-extreme-optimization/ relbookmark>            time classpublished dt-published datetime2024-11-19T07:10:29Z itempropdatePublished title2024-11-19 07:10>2024-11-19 07:10/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>Its time for that rarest of events: a blog post! And its another debugging adventure. Settle in, folks!/p>p>Recently I got interested in improving the time it takes to do a full compose of Fedora. This is when we start from just the packages and a few other inputs (basically image recipes and package groups), and produce a set of repositories and boot trees and images and all that stuff. For a long time this took somewhere between 5 and 10 hours. Recently weve managed to get it down to 3-4, then I figured out a hrefhttps://pagure.io/pungi/pull-request/1790>a change/a> which has got it under 3 hours./p>p>After that I a hrefhttps://pagure.io/releng/issue/12463>re-analyzed the process/a> and figured out that the biggest remaining point to attack is something called the pkgset phase, which happens near the start of the process, not in parallel with anything else, and takes 35 minutes or so. So I started digging into that to see if it can be improved./p>p>I fairly quickly a hrefhttps://pagure.io/releng/issue/12463#comment-944539>found/a> that it spends about 20 minutes in one relatively small codepath. Its created one giant package set (this is just a concept in memory at the time, it gets turned into an actual repo later) with every package in the compose in it. During this 20 minutes, it creates subsets of that package set per architecture, with only the packages relevant to that architecture in it (so packages for that arch, plus noarch packages, plus source packages, plus compatible arches, like including i686 for x86_64)./p>p>I poked about at that code a bit and decided I could maybe make it a bit more efficient. The current version works by creating each arch subset one at a time by looping over the big global set. Because every arch includes noarch and src packages, it winds up looping over the noarch and src lists once per arch, which seemed inefficient. So I went ahead and a hrefhttps://pagure.io/pungi/pull-request/1794>rewrote it to create them all at once/a>, to try and reduce the repeated looping./p>p>Today I was testing that out, which unfortunately has to be done more or less in production, so if you like you can watch me a hrefhttps://kojipkgs.fedoraproject.org/compose/adamwtest-ignore/>here/a>, where youll see composes appearing and disappearing every fifteen minutes or so. At first of course my change didnt work at all because Id made the usual few dumb mistakes with wrong variable names and stuff. After fixing all that up, I timed it, and it turned out a hrefhttps://pagure.io/pungi/pull-request/1794#comment-211855>about 7 minutes faster/a>. Not earth shattering, but hey./p>p>So I started checking it was accurate (i.e. created the same package sets as the old code). It turned out it wasnt quite (a subtle bug with noarch package exclusions). While fixing that, I ran across some lines in the code that had bugged me since the first time I started looking at it:/p>div classcode>pre classcode literal-block>                if i.file_path in self.file_cache:                    # TODO: test if it really works                    continue/pre>/div>p>These were extra suspicious to me because, not much later, theyre followed by this:/p>div classcode>pre classcode literal-block>                self.file_cache.file_cachei.file_path  i/pre>/div>p>that is, we check if the thing is in code>self.file_cache/code> and move on if it is, but if its not, we add it to code>self.file_cache.file_cache/code>? That didnt look right at all. But up till now Id left it alone, because hey, it had been this way for years, right? Must be OK. Well, this afternoon, in passing, I thought eh, lets try changing it./p>p>Then things got weird./p>p>I was having trouble getting the compose process to actually run exactly as it does in production, but once I did, I was getting what seemed like really bizarre results. The original code was taking 22 minutes in my tests. My earlier test of my branch had taken about 14 minutes. Now it was taking em>three seconds/em>./p>p>I thought, this cant possibly be right! So I spent a few hours running and re-running the tests, adding debug lines, trying to figure out how (surely) I had completely broken it and it was just bypassing the whole block, or something./p>p>Then I thought...what if I go back to the original code, but change the cache thing?/p>p>So I went back to unmodified pungi code, commented out those three lines, ran a compose...and it took three seconds. Tried again with the check corrected to code>self.file_cache.file_cache/code> instead of code>self.file_cache/code>...three seconds./p>p>I repeated this enough times that it em>must/em> be true, but it still bugged me. So I just spent a while digging into it, and I em>think/em> I know why. These file caches are code>kobo.pkgset.FileCache/code> instances; see the a hrefhttps://github.com/release-engineering/kobo/blob/57ba5c6b25c41c2797a4a5174ec3ba806a6c171f/kobo/pkgset.py#L220>source code here/a>. So, whats the difference between code>foo in self.file_cache/code> and code>foo in self.file_cache.file_cache/code>? Well, a code>FileCache/code> instances own code>file_cache/code> is a dict. code>FileCache/code> instances also implement code>__iter__/code>, returning code>iter(self.file_cache)/code>. I think this is why code>foo in self.file_cache/code> works em>at all/em> - it actually does do the right thing. But the key is, I think, that it does it em>inefficiently/em>./p>p>Pythons preferred way to do code>foo in bar/code> is to call code>bar.__contains__(foo)/code>. If that doesnt work, it falls back on iterating over code>bar/code> until it either hits foo or runs out of iterations. If code>bar/code> doesnt support iteration it just raises an exception./p>p>Python dictionaries have a very efficient implementation of code>__contains__/code>. So when we do code>foo in self.file_cache.file_cache/code>, we hit that efficient algorithm. But code>FileCache/code> does not implement code>__contains__/code>, so when we do code>foo in self.file_cache/code>, we fall back to iteration and wind up using that iterator over the dictionarys keys. This em>works/em>, but is massively less efficient than the dictionarys code>__contains__/code> method would be. And because these package sets are absolutely fracking huge, that makes a very significant difference in the end (because we hit the cache check a huge number of times, and every time it has to iterate over a huge number of dict keys)./p>p>So...heres the a hrefhttps://pagure.io/pungi/pull-request/1796>pull request/a>./p>p>Turns out I could have saved the day and a half it took me to get my rewrite correct. And if anyone had ever got around to TODOing the TODO, we couldve saved about 20 minutes out of every Fedora compose for the last nine years.../p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2023/06/20/devconfcz-2023-rawhide-update-test-gating-eln-testing-and-more/ classu-url>DevConf.CZ 2023, Rawhide update test gating, ELN testing and more!/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2023/06/20/devconfcz-2023-rawhide-update-test-gating-eln-testing-and-more/ relbookmark>            time classpublished dt-published datetime2023-06-20T11:17:53Z itempropdatePublished title2023-06-20 11:17>2023-06-20 11:17/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>Im in Brno, working from the office for a few days after the end of DevConf.CZ. It was a great conference, good to see people and feel some positive energy after all the stuff with RH layoffs and so on. It was really well attended, and there were a lot of useful talks. I presented on the current state of openQA and Fedora CI, with Miroslav Vadkerti kindly covering the Fedora CI stuff (thanks to Miro for that). The segmented talk video hasnt been updated yet, but you can watch it from the recorded live stream starting a hrefhttps://youtu.be/detokEIR4jU?t21872>here/a> (at 6:04:32). I think it went pretty well, Im much happier with this latest version of the talk than the versions I did at DevConf.US and LinuxCon (sorry to anyone who saw those ones!)/p>p>The a hrefhttps://devconfcz2023.sched.com/event/1MYbo/estimators-of-the-lost-arc-advance-recon-crew>talk by Aoife Moloney and Michal Konecny/a> on how the CPE team (which handles Fedora infra and apps) has started doing organized pre-scoping for projects instead of just diving in was really interesting and informative. The anaconda meetup wound up being just the anaconda team and myself and Sumantro from QA, but it was very useful as we were able to talk about the plans for moving forward with the new a hrefhttps://fedoramagazine.org/anaconda-web-ui-preview-image-now-public/>anaconda webUI/a> and how we can contribute testing for that - look out for Test Weeks coming soon. Davide Cavalcas a hrefhttps://devconfcz2023.sched.com/event/1MYlp/fedora-eln-at-meta-a-testbed-for-fleet-upgrades>talk on Fedora ELN usage at Meta/a> was great, and inspired me to work on stuff (more on that later)./p>p>There were a lot of random conversations as always - thanks to it being June, the hallway track mostly evolved into the shadow track, under the shade of a big tree in the courtyard, with beanbags and ice cream! Thats a definite improvement. The social event was in a great location - around an outdoor swimming pool (although we couldnt swim - apparently we couldnt serve drinks if swimming was allowed, so that seems like the best choice!) All in all, a great conference. Im very much looking forward to a hrefhttps://flocktofedora.org/>Flock in Cork/a> now, and will be doing my talk there again if its accepted./p>p>Tomorrow will be an exciting day, because (barring any unforeseen issues) well be a hrefhttps://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/message/G2K2SMYN7ONOJEFQEMCDR7GK72MZQFYB/>turning on gating of Rawhide updates/a>! Ive been working towards this for some time now - improving the reliability of the tests, implementing test re-run support from Bodhi, implementing the critical path group stuff, and improving the Bodhi web UI display of test results and gating status - so Im really looking forward to getting it done (and hoping it goes well). This should mean Rawhides stability improves even more, and Kevin and I dont have to scramble quite so much to shadow gate Rawhide any more (by untagging builds that fail the tests)./p>p>Davide mentioned during his ELN talk that they ran into an issue that openQA would have caught if it ran on ELN, so I asked if that would be useful, and he said yes. So, yesterday I did it. This required changes to a hrefhttps://pagure.io/fedora-qa/fedfind/c/967349c5e2f81bc95f8f617457119c95ef1f067d?branchmain>fedfind/a>, the a hrefhttps://pagure.io/fedora-qa/os-autoinst-distri-fedora/c/b0fb6911f3d74721a2ca70ee0db309d619146df6?branchmain>openQA tests/a>, and the a hrefhttps://pagure.io/fedora-qa/fedora_openqa/c/2e0500d7ce0d9becb3f59186c1f007898aba76ad?branchmain>openQA scheduler/a> - and then after that all worked out well and I deployed it, I realized it also needed changes to the result reporting code and a couple of other things too, which I had to do in rather a hurry! But its all sorted out now, and we have new ELN composes automatically tested in production when they land. Initially only a couple of default-install-and-boot tests were running, Im now working to extend the test set and the tested images./p>p>Other than that Ive been doing a lot of work on the usual things - keeping openQA updated and running smoothly, investigating and fixing test failures, improving stuff in Bodhi and Greenwave, and reviewing new tests written by lruzicka. Ill be on vacation for a week or so from Friday, which will be a nice way to decompress from DevConf, then back to work on a bunch of ideas that came out of it!/p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2023/02/02/thoughts-on-a-pile-of-laptops/ classu-url>Thoughts on a pile of laptops/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2023/02/02/thoughts-on-a-pile-of-laptops/ relbookmark>            time classpublished dt-published datetime2023-02-02T21:42:51Z itempropdatePublished title2023-02-02 21:42>2023-02-02 21:42/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>Hi folks! For the first post of 2023, I thought Id do something a bit different. If you want to keep up with what Ive been working on, these days a hrefhttps://fosstodon.org/@adamw>Mastodon/a> is the best place - Ive been posting a quick summary at the end of every working day there. Seems to be working out well so far. The biggest thing lately is that grouped critical path, which I wrote about in my a hrefhttps://www.happyassassin.net/posts/2022/11/18/fedora-37-openqa-news-mastodon-and-more/>last post/a>, is deployed in production now. This has already reduced the amount of tests openQA has to run, and Im working on some further changes to optimize things more./p>p>So instead of that, I want to rhapsodize on this pile of laptops:/p>p>img altA pile of laptops srcimages/laptops.jpg>/p>p>On the top is the one I used as my main laptop for the last six years, and my main system for the last couple, since I got rid of my desktop. Its a Dell XPS 13 9360, the Kaby Lake generation. Not pictured (as its over here being typed on, not in the pile) is its replacement, a 2022 XPS 13 (9315), which I bought in December and have been pretty happy with so far. On the bottom of the pile is a Lenovo tester (with AMD Ryzen hardware) which I tried to use as my main system for a bit, but it didnt work out as it only has 8G of RAM and that turns out to be...not enough. Second from bottom is a terrible budget Asus laptop with Windows on it that I keep around for the occasional time I need to use Windows - mainly to strip DRM from ebooks. Not pictured is the older XPS 13 I used before the later two, which broke down after a few years./p>p>But the hidden star of the show is the one second from top. It has a high-resolution 13 display with pretty slim bezels and a built-in webcam. It has dual NVIDIA and Intel GPUs. It has 8G of RAM, SSD storage and a multicore CPU, and runs Fedora 36 just fine, with decent (3-4hr) battery life. It weighs 3.15lb (1.43kg) and has USB, HDMI and ethernet outs./p>p>It also has a built-in DVD drive, VGA out and an ExpressCard slot (anyone remember those?) Thats because its from strong>2010/strong>./p>p>Its a Sony Vaio Z VPC-Z11, and I still use it as a backup/test system. It barely feels outdated at all (until you remember about the DVD drive, which is actually pretty damn useful sometimes still). Every time I open it Im still amazed at what a ridiculous piece of kit it is/was. Just do an image search for 2010 laptop and youll see stuff like, well, a hrefhttps://www.notebookcheck.net/NBC-Onsite-HP-Notebook-Lineup-05-2010.30296.0.html>this/a>. Thats what pretty much every laptop looked like in 2010. They had 4G of RAM if you were lucky, and hard disks. They weighed 2kg+. They had huge frickin bezels. The Macbook Air had come out in 2008, but it was an underpowered thing with a weak CPU and HDD storage. The 2010 models had SSDs, but maxed out at 4G RAM and still had pretty weak CPUs (and way bigger bezels, and worse screens, and they certainly didnt have DVD drives). Theyd probably feel pretty painful to use now, but the Vaio still feels fine. Heres a glamour shot:/p>p>img altOne very cool laptop srcimages/vaioz.jpg>/p>p>Ive only had to replace its battery twice and its SSDs (it came from the factory with two SSDs configured RAID-0, because weird Sony is like that) once in 12 years. Probably one day it will finally not be really usable any more, but who the heck knows how long that will be./p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2022/11/18/fedora-37-openqa-news-mastodon-and-more/ classu-url>Fedora 37, openQA news, Mastodon and more/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2022/11/18/fedora-37-openqa-news-mastodon-and-more/ relbookmark>            time classpublished dt-published datetime2022-11-18T22:34:59Z itempropdatePublished title2022-11-18 22:34>2022-11-18 22:34/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>Hey, time for my now-apparently-annual blog post, I guess? First, a quick note: I joined the herd showing up on a hrefhttps://joinmastodon.org/>Mastodon/a>, on the a hrefhttps://fosstodon.org>Fosstodon/a> server, as a hrefhttps://fosstodon.org/@adamw>@span class__cf_email__ data-cfemail5d3c393c302a1d3b322e2e293239323373322f3a>email protected/span>/a>.  So, you know, follow me or whatever. I posted to Twitter even less than I post here, but well see what happens!/p>p>The big news lately is of course that a hrefhttps://fedoramagazine.org/announcing-fedora-37/>Fedora 37 is out/a>. Pulling this release together was a bit more painful than has been the norm lately, and it does have at least a hrefhttps://ask.fedoraproject.org/t/f37-install-media-dont-boot-in-uefi-mode-on-certain-motherboards/28364>one bug Im sad we didnt sort out/a>, but unless you have one of a very few motherboards from six years ago and want to do a reinstall, everything should be great!/p>p>Personally Ive been running Fedora Silverblue this cycle, as an experiment to see how it fares as a daily driver and a a hrefhttps://en.wikipedia.org/wiki/Eating_your_own_dog_food>dogfooding/a> base. Overall its been working fine; there are still some awkward corners if you are strict about avoiding RPM overlays, though. Im definitely interested in a hrefhttps://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable>Colins big native container rework proposal/a>, which would significantly change how the rpm-ostree-based systems work and make package layering a more accepted thing to do. I also found that sourcing apps feels slightly odd - Id kinda like to use Fedora Flatpaks for everything, from a dogfooding perspective, but not everything I use is available as one, so I wound up with kind of a mix of things sourced from Flathub and from Fedora Flatpaks. I was also surprised that Fedora Flatpaks arent generally updated terribly often, and dont seem to have development branches - while Fedora 37 was in development, I couldnt get Flatpak builds of apps that matched the Fedora 37 RPM builds, I was stuck running Fedora 36-based Flatpaks. So it actually impeded my ability to test the latest versions of everything. Itd be nice to see some improvement here going forward./p>p>My biggest project this year has been working towards gating Rawhide critical path updates on the a hrefhttps://openqa.fedoraproject.org>openQA/a> tests, as we do for stable and Branched releases. This has been a deceptively large effort; ensuring all the tests work OK on Rawhide was a relatively small job, but the experience of actually having the tests running has been interesting. There are, overall, a lot more updates for Rawhide than any other release, and obviously, they tend to break things more often. First I turned the tests on for the staging instance, then after a few months trying to get on top of things there, turned them on for the production instance. I planned to run this way for a month or two to see if I could stay on top of keeping the tests running smoothly and passing when they should, and dealing with breakage. On the whole, its been possible...but just barely. The increased workload means tests can take several hours to complete after an update is submitted, which isnt ideal. Because we dont have the gating turned on, when somebody does submit an update that breaks the tests, I have to ensure it gets fixed em>right away/em> or else get it untagged before the next Rawhide compose happens, or else the test will fail for every subsequent update too; that can be stressful. We also have had quite a lot of fun with intermittent problems like a hrefhttps://bugzilla.redhat.com/show_bug.cgi?id2133829>systemd-oomd killing things it shouldnt/a>. This can result in a lot of time spent manually restarting failed tests, coming up with awkward workarounds, and trying to debug the problems./p>p>So, I kinda felt like things arent quite solid enough yet to turn the gating on, and I wound up working down a path intended to help with the too many jobs take too long and intermittent failures angles. This actually started out when I a hrefhttps://pagure.io/fedora-comps/pull-request/764>added a proper critical path definition for KDE/a>. This rather increased the openQA workload, as it added a bunch of packages to critical path that werent there before. There was especially a fun moment when a couple hundred KDE package updates got submitted separately as Rawhide updates, and openQA spent a day running 55 tests on all of them, including all the GNOME and Server tests./p>p>As part of getting the KDE stuff added to the critical path, I wound up doing a a hrefhttps://pagure.io/releng/pull-request/11000>big update/a> to the script that actually generates the critical path definition, and working on that made me realize it wouldnt be difficult to track the critical path package set by group, not just as one big flat list. That, in turn, could allow us to only run relevant openQA tests for an update: if the update is only in the KDE critical path, we dont need to run the GNOME and Server tests on it, for instance. So for the last few weeks Ive been working on what turned out to be quite a lot of pieces relevant to that./p>p>First, I a hrefhttps://pagure.io/releng/pull-request/11044>added the fundamental support in the critical path generation script/a>. Then I had to make a hrefhttps://bodhi.fedoraproject.org>Bodhi/a> work with this. Bodhi decides whether an update is critical path or not, and openQA gets that information from Bodhi. Bodhi, as currently configured, actually gets this information from a hrefhttps://pdc.fedoraproject.org/>PDC/a>, which seems to me an unnecessary layer of indirection, especially as were hoping to retire PDC; Bodhi could just as easily itself be the source of truth for the critical path. So I a hrefhttps://github.com/fedora-infra/bodhi/pull/4755>made Bodhi capable of reading critpath information directly from the files output by the script/a>, then a hrefhttps://github.com/fedora-infra/bodhi/pull/4759>made it use the group information for Greenwave queries and show it in the web UI and API query results/a>. Thats all a hard requirement for running fewer tests on some updates, because without that, we would still always gate on all the openQA tests for every critical path update - so if we didnt run all the tests for some update, it would always fail gating. I also a hrefhttps://pagure.io/fedora-infra/ansible/c/18817b175aca65c4c1c8167eaadad231ef7a0449?branchmain>changed the Greenwave policies accordingly/a>, to only require the appropriate set of tests to pass for each critical path group, once our production Bodhi is set up to use all this new stuff - until then, the combined policy for the non-grouped decision contexts Bodhi still uses for now winds up identical to what it was before./p>p>Once a new Bodhi release is made and deployed to production, and we configure it to use the new grouped-critpath stuff instead of the flat definition from PDC, all of the groundwork is in place for me to actually change the openQA scheduler to check which critical path group(s) an update is in, and only schedule the appropriate tests. But along the way, I noticed this change meant Bodhi was querying Greenwave for em>even more/em> decision contexts for each update. Right now for critical path updates Bodhi usually sends two queries to Greenwave (if there are more than seven packages in the update, it sends 2*((number of packages in update+1)/8) queries). With these changes, if an update was in, say, three critical path groups, it would send 4 (or more) queries. This slows things down, and also produces a hrefhttps://github.com/fedora-infra/bodhi/issues/4320>rather awkward and hard-to-understand output in the web UI/a>. So I decided to fix that too. I made it so the gating status displayed in the web UI is a hrefhttps://github.com/fedora-infra/bodhi/pull/4784>combined from however many queries Bodhi has to make/a>, instead of just displaying the result of each query separately. Then I tweaked greenwave to allow a hrefhttps://github.com/release-engineering/greenwave/pull/95>querying multiple decision contexts together/a>, and a hrefhttps://github.com/fedora-infra/bodhi/pull/4821>had Bodhi make use of that/a>. With those changes combined, Bodhi should only have to query once for most updates, and for updates with more than seven packages, the displayed gating status wont be confusing any more!/p>p>Im hoping all those Bodhi changes can be deployed to stable soon, so I can move forward with the remaining work needed, and ultimately see how much of an improvement we see. Im hoping well wind up having to run rather fewer tests, which should reduce the wait time for tests to complete and also mitigate the problem of intermittent failures a bit. If this works out well enough, we might be able to move ahead with actually turning on the gating for Rawhide updates, which Im really looking forward to doing./p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2022/01/11/adamws-debugging-adventures-bootloaders-and-machine-ids/ classu-url>AdamWs Debugging Adventures: Bootloaders and machine IDs/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2022/01/11/adamws-debugging-adventures-bootloaders-and-machine-ids/ relbookmark>            time classpublished dt-published datetime2022-01-11T22:08:00Z itempropdatePublished title2022-01-11 22:08>2022-01-11 22:08/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>Hi folks! Well, it looks like I forgot to blog for...em>checks watch/em>....em>checks calendar/em>...a year. Wow. Whoops. Sorry about that. Im still here, though! We released, uh, lots of Fedoras since the last time I wrote about that. Fedora 35 is the current one. Its, uh, a hrefhttps://fedoraproject.org/wiki/Common_F35_bugs>mostly great!/a> Go a hrefhttps://getfedora.org/>get a copy/a>, why dont you?/p>p>And while thats downloading, you can get comfy and listen to another of Crazy Uncle Adams Debugging Adventures. In this episode, well be uncomfortably reminded just how much of the code that causes your system to actually boot at all consists of fragile shell script with no tests, so thisll be fun!/p>p>Last month, booting a system installed from Rawhide live images stopped working properly. You could boot the live image fine, run the installation fine, but on rebooting, the system would fail to boot with an error: code>dracut: FATAL: Dont know how to handle rootlive:CDLABELFedora-WS-Live-rawh-20211229-n-1/code>. openQA caught this, and so did one of our QA community members - Ahed Almeleh - who a hrefhttps://bugzilla.redhat.com/show_bug.cgi?id2036199>filed a bug/a>. After the end-of-year holidays, I got to figuring out what was going wrong./p>p>As usual, I got a bit of a head start from pre-existing knowledge. I happen to know that error message is referring to kernel arguments that are set in the bootloader configuration of em>the live image itself/em>. dracut is the tool that handles an early phase of boot where we boot into a temporary environment thats loaded entirely into system memory, set up the em>real/em> system environment, and boot that. This early environment is contained in the code>initrd/code> files you can find alongside the kernel on most Linux distributions; thats what theyre for. Part of dracuts job is to be run when a kernel is installed to em>produce/em> this environment, and then other parts of dracut are included em>in the environment itself/em> to handle initializing things, finding the real system root, preparing it, and then switching to it. The initrd environments on Fedora live images are built to contain a dracut module (called code>90dmsquash-live/code>) that knows to interpret code>rootlive:CDLABELFedora-WS-Live-rawh-20211229-n-1/code> as meaning go look for a live system root on the filesystem with that label and boot that. Installed systems dont contain that module, because, well, they dont need to know how to do that, and you wouldnt really ever want an installed system to em>try/em> and do that./p>p>So the short version here is: the installed system has the wrong kernel argument for telling dracut where to find the system root. It em>should/em> look something like code>root/dev/mapper/fedora-root/code> (where were pointing to a system root on an LVM volume that dracut will set up and then switch to). So the obvious next question is: why? Why is our installed system getting this wrong argument? It seemed likely that it leaked from the live system to the installed system somehow, but I needed to figure out how./p>p>From here, I had kinda two possible ways to investigate. The easiest and fastest would probably be if I happened to know exactly how we deal with setting up bootloader configuration when running a live install. Then Id likely have been able to start poking the most obvious places right away and figure out the problem. But, as it happens, I didnt at the time remember exactly how that works. I just remembered that I wind up having to figure it out every few years, and its complicated and scary, so I tend to forget again right afterwards. I kinda knew where to start looking, but didnt really want to have to work it all out again from scratch if I could avoid it./p>p>So I went with the other possibility, which is always: figure out when it broke, and figure out what changed between the last time it worked and the first time it broke. This usually makes life much easier because now you know one of the things on that list is the problem. The shorter and simpler the list, the easier life gets./p>p>I looked at the openQA result history and found that the bug was introduced somewhere between 20211215.n.0 and 20211229.n.1 (unfortunately kind of a wide range). The good news is that only a few packages could plausibly be involved in this bug; the most likely are dracut itself, grub2 (the bootloader), grubby (a Red Hat / Fedora-specific grub configuration...thing), anaconda (the Fedora installer, which obviously does some bootloader configuration stuff), the kernel itself, and systemd (which is of course involved in the boot process itself, but also - perhaps less obviously - is where a script called a hrefhttps://github.com/systemd/systemd/tree/main/src/kernel-install>code>kernel-install/code>/a> that is used (on Fedora and many other distros) to install kernels lives (this was another handy thing I happened to know already, but really - its always a safe bet to include systemd on the list of potential suspects for anything boot-related)./p>p>Looking at what changed between 2021-12-15 and 2021-12-29, we could let out grub2 and grubby as they didnt change. There were some kernel builds, but nothing in the scriptlets changed in any way that could be related. dracut got a build with a hrefhttps://src.fedoraproject.org/rpms/dracut/c/76eb28fc2ef2f9e43b5ea66d0b9c96f83e124d4b?branchrawhide>one change/a>, but again it seemed clearly unrelated. So I was down to anaconda and systemd as suspects. On an initial quick check during the vacation, I thought anaconda had not changed, and took a brief look at systemd, but didnt see anything immediately obvious./p>p>When I came back to look at it more thoroughly, I realized anaconda did get a new version (36.12) on 2021-12-15, so that initially interested me quite a lot. I spent some time going through the changes in that version, and there were some that really could have been related - it changed how running things during install inside the installed system worked (which is definitely how we do some bootloader setup stuff during install), and it had interesting commit messages like Remove the dracut_args attribute and Remove upd-kernel. So I spent an afternoon fairly sure itd turn out to be one of those, reviewed all those changes, mocked up locally how they worked, examined the logs of the actual image composes, and...concluded that none of those seemed to be the problem at all. The installer seemed to still be doing things the same as it always had. There werent any tell-tale missing or failing bootloader config steps. However, this time wasnt entirely wasted: I was reminded of exactly what anaconda does to configure the bootloader when installing from a live image./p>p>When we install from a live image, we dont do what the traditional installer does and install a bunch of RPM packages using dnf. The live image does not contain any RPM packages. The live image itself was em>built/em> by installing a bunch of RPM packages, but it is the em>result/em> of that process. Instead, we essentially set up the filesystems on the drive(s) were installing to and then just dump the contents of the live image filesystem em>itself/em> onto them. Then we run a few tweaks to adjust anything that needs adjusting for this now being an installed system, not a live one. One of the things we do is re-generate the code>initrd/code> file for the installed system, and then re-generate the bootloader configuration. This involves running code>kernel-install/code> (which places the kernel and initrd files onto the boot partition, and writes some bootloader configuration snippet files), and then running code>grub2-mkconfig/code>. The main thing code>grub2-mkconfig/code> does is produce the main bootloader configuration file, but thats not really why we run it at this point. Theres a hrefhttps://github.com/rhinstaller/anaconda/blob/anaconda-36.12-1/pyanaconda/modules/storage/bootloader/utils.py#L244>a very interesting comment/a> explaining why in the anaconda source:/p>div classcode>pre classcode literal-block>span classc1># Update the bootloader configuration to make sure that the BLS/span>span classc1># entries will have the correct kernel cmdline and not the value/span>span classc1># taken from /proc/cmdline, that is used to boot the live image./span>/pre>/div>p>Which is exactly what we were dealing with here. The BLS entries were talking about here are the things I called snippet files above, they live in code>/boot/loader/entries/code> on Fedora systems. These are where the kernel arguments used at boot are specified, and indeed, thats where the problematic code>rootlive:.../code> arguments were specified in broken installs - in the BLS entries in code>/boot/loader/entries/code>. So it seemed like, somehow, this mechanism just wasnt working right any more - we were expecting this run of code>grub2-mkconfig/code> in the installed system root after live installation to correct those snippets, but it wasnt. However, as I said, I couldnt establish that any change to anaconda was causing this./p>p>So I eventually shelved anaconda at least temporarily and looked at systemd. And it turned out that systemd had changed too. During the time period in question, wed gone from systemd 250~rc1 to 250~rc3. (If you check the build history of systemd the dates dont seem to match up - by 2021-12-29 the 250-2 build had happened already, but in fact the 250-1 and 250-2 builds were untagged for causing a different problem, so the 2021-12-29 compose had 250~rc3). By now I was obviously pretty focused on code>kernel-install/code> as the most likely related part of systemd, so I went to my systemd git checkout and ran:/p>div classcode>pre classcode literal-block>git log v250-rc1..v250-rc3 src/kernel-install//pre>/div>p>which shows all the commits under code>src/kernel-install/code> between 250-rc1 and 250-rc3. And that gave me another juicy-looking, yet thankfully short, set of commits:/p>p>a hrefhttps://github.com/systemd/systemd/commit/641e2124de6047e6010cd2925ea22fba29b25309>641e2124de6047e6010cd2925ea22fba29b25309/a> kernel-install: replace 00-entry-directory with K_I_LAYOUT in k-ia hrefhttps://github.com/systemd/systemd/commit/357376d0bb525b064f468e0e2af8193b4b90d257>357376d0bb525b064f468e0e2af8193b4b90d257/a> kernel-install: Introduce KERNEL_INSTALL_MACHINE_ID in /etc/machine-infoa hrefhttps://github.com/systemd/systemd/commit/447a822f8ee47b63a4cae00423c4d407bfa5e516>447a822f8ee47b63a4cae00423c4d407bfa5e516/a> kernel-install: Remove Default from list of suffixes checked/p>p>So I went and looked at all of those. And again...I got it wrong at first! This is I guess a good lesson from this Debugging Adventure: you dont always get the right answer at first, but thats okay. You just have to keep plugging, and always keep open the possibility that youre wrong and you should try something else. I spent time thinking the cause was likely a change in anaconda before focusing on systemd, then focused on the wrong systemd commit first. I got interested in 641e212 first, and had even written out a whole Bugzilla comment blaming it before I realized it wasnt the culprit (fortunately, I didnt post it!) I thought the problem was that the new check for code>$BOOT_ROOT/$MACHINE_ID/code> would not behave as it should on Fedora and cause the install scripts to do something different from what they should - generating incorrect snippet files, or putting them in the wrong place, or something./p>p>Fortunately, I decided to test this before declaring it was the problem, and found out that it wasnt. I did this using something that turned out to be invaluable in figuring out the real problem./p>p>You may have noticed by this point - harking back to our intro - that this critical code>kernel-install/code> script, key to making sure your system boots, is...a shell script. That calls other shell scripts. You know what else is a big pile of shell scripts? code>dracut/code>. You know, that critical component that both builds and controls the initial boot environment. Big pile of shell script. The install script - the code>dracut/code> command itself - is shell. All the dracut code>modules/code> - the bits that do most of the work - are shell. Theres a bit of C in the source tree (Im not entirely sure what that bit does), but most of its shell./p>p>Critical stuff like this being written in shell makes me shiver, because shell is very easy to get wrong, and quite hard to test properly (and in fact neither dracut nor kernel-install has good tests). But one good thing about it is that its quite easy to em>debug/em>, thanks to the magic of code>sh -x/code>. If you run some shell script via code>sh -x/code> (whether thats really code>sh/code>, or code>bash/code> or some other alternative pretending to be code>sh/code>), it will run as normal but print out most of the logic (variable assignments, tests, and so on) that happen along the way. So on a VM where Id run a broken install, I could do code>chroot /mnt/sysimage/code> (to get into the root of the installed system), find the exact code>kernel-install/code> command that anaconda ran from one of the logs in code>/var/log/anaconda/code> (I forget which), and re-run it through code>sh -x/code>. This showed me all the logic going on through the run of code>kernel-install/code> itself and all the scripts it sources under code>/usr/lib/kernel/install.d/code>. Using this, I could confirm that the check I suspected had the result I suspected - I could see that it was deciding that code>layoutother/code>, not code>layoutbls/code>, a hrefhttps://github.com/systemd/systemd/blob/v250-rc3/src/kernel-install/kernel-install#L134>here/a>. But I could em>also/em> figure out a way to override that decision, confirm that it worked, and find that it didnt solve the problem: the config snippets were still wrong, and running code>grub2-mkconfig/code> didnt fix them. In fact the config snippets got wronger - it turned out that we do em>want/em> code>kernel-install/code> to pick other rather than bls here, because Fedora doesnt really implement BLS according to the upstream specs, so if we let kernel-install think we do, the config snippets we get are wrong./p>p>So now Id been wrong twice! But each time, I learned a bit more that eventually helped me be right. After I decided that commit wasnt the cause after all, I finally spotted the problem. I figured this out by continuing with the code>sh -x/code> debugging, and noticing an inconsistency. By this point Id thought to find out what bit of code>grub2-mkconfig/code> should be doing the work of correcting the key bit of configuration here. Its in a a hrefhttps://src.fedoraproject.org/rpms/grub2/blob/b25606806096b51e9d920f50b9bb47773641644d/f/0062-Add-BLS-support-to-grub-mkconfig.patch#_230>Fedora-only downstream patch to one of the scriptlets in code>/etc/grub.d/code>/a>. It replaces the code>options/code> line in any snippet files it finds with what it reckons the kernel arguments should be. So I got curious about what exactly was going wrong there. I tweaked code>grub2-mkconfig/code> slightly to run those scriptlets using code>sh -x/code> by changing these lines in code>grub2-mkconfig/code>:/p>div classcode>pre classcode literal-block>span classn>echo/span> span classs>### BEGIN $i ###/span>span classs>$i/span>span classn>echo/span> span classs>### END $i ###/span>/pre>/div>p>to read:/p>div classcode>pre classcode literal-block>span classn>echo/span> span classs>### BEGIN $i ###/span>span classn>sh/span> span classo>-/span>span classn>x/span> span classs>$i/span>span classn>echo/span> span classs>### END $i ###/span>/pre>/div>p>Now I could re-run code>grub2-mkconfig/code> and look at what was going on behind the scenes of the scriptlet, and I noticed that it wasnt em>finding/em> any snippet files at all. But why not?/p>p>a hrefhttps://src.fedoraproject.org/rpms/grub2/blob/b25606806096b51e9d920f50b9bb47773641644d/f/0062-Add-BLS-support-to-grub-mkconfig.patch#_211>The code that looks for the snippet files/a> reads the file code>/etc/machine-id/code> as a string, then looks for files in code>/boot/loader/entries/code> whose names start with that string (and end in code>.conf/code>). So I went and looked at my sample system and...found that the files in code>/boot/loader/entries/code> did em>not/em> start with the string in code>/etc/machine-id/code>. The files in code>/boot/loader/entries/code> started with code>a69bd9379d6445668e7df3ddbda62f86/code>, but the ID in code>/etc/machine-id/code> was code>b8d80a4c887c40199c4ea1a8f02aa9b4/code>. This is why everything was broken: because those IDs didnt match, code>grub2-mkconfig/code> couldnt find the files to correct, so the argument was wrong, so the system didnt boot./p>p>Now I knew what was going wrong and I only had two systemd commits left on the list, it was pretty easy to see the problem. It was in a hrefhttps://github.com/systemd/systemd/commit/357376d0bb525b064f468e0e2af8193b4b90d257>357376d/a>. That changes how code>kernel-install/code> names these snippet files when creating them. It names them by finding a machine ID to use as a prefix. Previously, it used whatever string was in code>/etc/machine-id/code>; if that file didnt exist or was empty, it just used the string Default. After that commit, it also looks for a value specified in code>/etc/machine-info/code>. If theres a code>/etc/machine-id/code> but not code>/etc/machine-info/code> when you run code>kernel-install/code>, it uses the value from code>/etc/machine-id/code> and writes it to code>/etc/machine-info/code>./p>p>When I checked those files, it turned out that on the live image, the ID in both code>/etc/machine-id/code> and code>/etc/machine-info/code> was code>a69bd9379d6445668e7df3ddbda62f86/code> - the problematic ID on the installed system. When we generate the live image itself, code>kernel-install/code> uses the value from code>/etc/machine-id/code> and writes it to code>/etc/machine-info/code>, and both files wind up in the live filesystem. But em>on the installed system/em>, the ID in code>/etc/machine-info/code> was that same value, but the ID in code>/etc/machine-id/code> was different (as we saw above)./p>p>Remember how I mentioned above that when doing a live install, we essentially dump the live filesystem itself onto the installed system? Well, one of the tweaks we make when doing this is to a hrefhttps://github.com/rhinstaller/anaconda/blob/612bfee7d37e16d7bfab329b44182a71d04a3344/pyanaconda/modules/storage/bootloader/utils.py#L41-L48>re-generate code>/etc/machine-id/code>/a>, because that ID is meant to be unique to each installed system - we dont want every system installed from a Fedora live image to have the same machine ID as the live image itself. However, as this code>/etc/machine-info/code> file is new, we dont strip it from or re-generate it in the installed system, we just install it. The installed system has a code>/etc/machine-info/code> with the same ID as the live images machine ID, but a new, different ID in code>/etc/machine-id/code>. And this (finally) was the ultimate source of the problem! When we run them on the installed system, the new version of code>kernel-install/code> writes config snippet files using the ID from code>/etc/machine-info/code>. But Fedoras patched code>grub2-mkconfig/code> scriptlet doesnt know about that mechanism at all (since its brand new), and expects the snippet files to contain the ID from code>/etc/machine-id/code>./p>p>There are various ways you could potentially solve this, but after consulting with systemd upstream, the one we chose is to a hrefhttps://github.com/rhinstaller/anaconda/commit/fe652add9733943a3476128a83b24b9a8c63b335>have anaconda exclude code>/etc/machine-info/code>/a> when doing a live install. The changes to systemd here arent wrong - it does potentially make sense that code>/etc/machine-id/code> and code>/etc/machine-info/code> could both exist and specify different IDs in some cases. But for the case of Fedora live installs, it doesnt make sense. The sanest result is for those IDs to match and both be the fresh machine ID thats generated at the end of the install process. By just not including code>/etc/machine-info/code> on the installed system, we achieve this result, because now when code>kernel-install/code> runs at the end of the install process, it reads the ID from code>/etc/machine-id/code> and writes it to code>/etc/machine-info/code>, and both IDs are the same, code>grub2-mkconfig/code> finds the snippet files and edits them correctly, the installed system boots, and I can move along to the next debugging odyssey.../p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2020/11/24/site-and-blog-migration/ classu-url>Site and blog migration/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2020/11/24/site-and-blog-migration/ relbookmark>            time classpublished dt-published datetime2020-11-24T00:36:54Z itempropdatePublished title2020-11-24 00:36>2020-11-24 00:36/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>So Ive been having an adventurous week here at HA Towers: I decided, after something more than a decade, Im going to get out of the self-hosting game, as far as I can. It makes me a bit sad, because its been kinda cool to do and I think its worked pretty well, but Im getting to a point where it seems silly that a small part of me has to constantly be concerned with making sure my web and mail servers and all the rest of it keep working, when the services exist to do it much more efficiently. Its cool that its still possible to do it, but I dont think I need to em>actually do it/em> any more./p>p>So, if youre reading this...and I didnt do something really weird...its not being served to you by a Fedora system three feet from my desk any more. Its being served to you by a server owned by a commodity web hoster...somewhere in North America...running Lightspeed (boo) on who knows what OS. I pre-paid for four years of hosting before realizing they were running proprietary software, and I figured what the hell, its just a web serving serving static files. If it starts to really bug me Ill move it, and hopefully youll never notice./p>p>All the redirects for old Wordpress URLs should still be in place, and also all URLs for software projects I used to host here (fedfind etc) should redirect to appropriate places in Pagure and/or Pypi. Please yell if you see something that seems to be wrong. I moved a hrefhttps://openqa.fedoraproject.org/nightlies.html>nightlies/a> and a hrefhttps://openqa.fedoraproject.org/testcase_stats/>testcase_stats/a> to the Fedora openQA server for now; thats still a slightly odd place for them to be, but at least its in the Fedora domain not on my personal domain, and it was easiest to do since I have all the necessary permissions, putting them anywhere else would be more work and require other people to do stuff, so this is good enough for now. Redirects are in place for those too./p>p>Ive been working on all the other stuff I self-host, too. Today I set up all the IRC channels I regularly read in my a hrefhttps://matrix.org/>Matrix/a> account and Im going to try using that setup for IRC instead of my own proxy (which ran a hrefhttps://bip.milkypond.org/>bip/a>). It seems to work okay so far. Im using the a hrefhttps://github.com/quotient-im/Quaternion>Quaternion/a> client for now, as it seems to have the most efficient UI layout and isnt a big heavy wrapper around a web client. Matrix is a really cool thing, and itd be great to see more F/OSS projects adopting it to lower barriers to entry without compromising F/OSS principles; IRC really is getting pretty creaky these days, folks. Theres some talk about both Fedora and GNOME adopting Matrix officially, and I really hope that happens./p>p>I also set up a a hrefhttps://kolabnow.com/>Kolab Now/a> account and switched my contacts and calendar to it, which was nice and easy to do (download the ICS files from Radicale, upload them to Kolab, switch my accounts on my laptops and phone, shut down the Radicale server, done). I also plan to have it serve my mail, but that migration is going to be the longest and most complicated as Ill have to move several gigs of mail and re-do all my filters. Fun!/p>p>I also refreshed my desktop setup; after (again) something more than a decade having a dedicated desktop PC Im trying to roll without one again. Back when I last did this, I got to resenting the clunky nature of docking at the time, and also I still ran quite a lot of local code compiles and laptops arent ideal for that. These days, though, docking is getting pretty slick, and I dont recall the last time I built anything really chunky locally. My current laptop (a 2017 XPS 13) should have enough power anyhow, for the occasional case. So I got me a a hrefhttps://www.apple.com/ca/shop/product/HMX12ZM/A/caldigit-ts3-plus-dock>fancy Thunderbolt dock/a> - yes, from the Apple store, because apparently no-one else has it in stock in Canada - and a a hrefhttps://www.techradar.com/reviews/benq-ew3270u-monitor>32 4K monitor/a> and plugged the things into the things and waited a whole night while all sorts of gigantic things I forgot I had lying around my home directory synced over to the laptop and...hey, it works. Probably in two months Ill run into something weird thats only set up on the old desktop box, but hey./p>p>So once I have all this wrapped up Im aiming to have substantially fewer computers lying around here and fewer Sysadmin Things taking up space in my brain. At the cost of being able to say I run an entire domain out of a $20 TV stand in my home office. Ah, well./p>p>Oh, I also bought a a hrefhttps://blueradius.ca>new domain/a> as part of this whole thing, as a sort of backup / staging area for transitions and also possibly as an alternative vanity domain. Because it is sometimes awkward telling people yes, my email address is happyassassin.net, no, Im not an assassin, dont worry, its a name based on a throwaway joke from university which I probably wouldnt have picked if I knew Id be signing up for bank accounts with it fifteen years later. So if I do start using it for stuff, here is your advance notice that yeah, its me. This name I just picked to be vaguely memorable and hopefully to be entirely inoffensive, vaguely professional-sounding, and composed of sounds that are unambiguous when read over an international phone line to a call centre in India. It doesnt mean anything at all./p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2020/06/17/on-inclusive-language-an-extended-metaphor-involving-parties-because-why-not/ classu-url>On inclusive language: an extended metaphor involving parties because why not/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2020/06/17/on-inclusive-language-an-extended-metaphor-involving-parties-because-why-not/ relbookmark>            time classpublished dt-published datetime2020-06-17T00:57:49Z itempropdatePublished title2020-06-17 00:57>2020-06-17 00:57/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>So theres been some discussion within Red Hat about inclusive language lately, obviously related to current events and the worldwide protests against racism, especially anti-Black racism. I dont want to get into any internal details, but in one case we got into some general debate about the validity of efforts to use more inclusive language. I thought up this florid party metaphor, and I figured instead of throwing it at an internal list, Id put it up here instead. If you have constructive thoughts on it, go ahead and mail me or start a twitter thread or something. If you have non-constructive thoughts on it, keep em to yourself!/p>p>Before we get into my pontificating, though, heres some useful practical resources if you just want to read up on how you can make the language in your projects and docs more inclusive:/p>ul>li>a hrefhttps://developers.google.com/style/inclusive-documentation>Google inclusive documentation style guide/a>/li>li>a hrefhttps://lethargy.org/~jesus/writes/a-guide-to-nomenclature-selection/>A Guide to Nomenclature Selection, by Theo Schlossnagle/a>/li>li>a hrefhttps://tools.ietf.org/id/draft-knodel-terminology-00.html>Terminology, Power and Oppressive Language RFC draft by M. Knodel/a>/li>/ul>p>To provide a bit of context: I was thinking about a suggestion that people promoting the use of more inclusive language are trying to be offended. And heres where my mind went!/p>p>Imagine you are throwing a party. You send out the invites, order in some hors douevres (did I spell that right? I never spell that right), queue up some Billie Eilish (everyone loves Billie Eilish, its a scientific fact), set out the drinks, and wait for folks to arrive. In they all come, the rooms buzzing, everyone seems to be having a good time, its going great!/p>p>But then you notice (or maybe someone else notices, and tells you) that most of the people at your party seem to be straight white dudes and their wives and girlfriends. Thats weird, you think, Im an open minded modern guy, Id be happy to see some Black folks and maybe a cute gay couple or something! What gives? I dont want people to think Im some kind of racist or sexist or homophobe or something!/p>p>So you go and ask some non-white folks and some non-straight folks and some non-male folks whats going on. What is it? Is it me? What did I do wrong?/p>p>Well, they say, look, its a hugely complex issue, I mean, we could be here all night talking about it. And yes, fine, that broken pipeline outside your house might have something to do with it (IN-JOKE ALERT). But since you ask, look, let us break this one part of it down for you./p>p>You know how youve got a bouncer outside, and every time someone rolls up to the party he looks them up and down and says well hi there! Whats your name? Is it on the BLACKLIST or the WHITELIST? Well...I mean...that might put some folks off a bit. And you know how you made the theme of the party masters and slaves? You know, that might have something to do with it too. And, yeah, you see how you sent all the invites to men and wrote if your wife wants to come too, just put her name in your reply? I mean, you know, that might speak to some people more than others, you hear what Im saying?/p>p>Now...this could go one of two ways. On the Good Ending, you might say hey, you know what? I didnt think about that. Thanks for letting me know. I guess next time Ill maybe change those things up a bit and maybe itll help. Hey thanks! I appreciate it!/p>p>and that would be great. But unfortunately, you might instead opt for the Bad Ending. In the Bad Ending, you say something like this:/p>p>Wow. I mean, just wow. I feel so em>attacked/em> here. Its not like I called it a blacklist because Im em>racist/em> or something. I dont have a racist bone in my body, why do you have to read it that way? You know a hrefhttps://github.com/graphite-project/graphite-web/issues/1569#issuecomment-237320189>blacklist doesnt even MEAN that/a>, right? And jeez, look, the whole masters and slaves thing was just a bit of fun, its not like we made all the Black people the slaves or something! And besides that whole thing was so long ago! And I mean look, most people are straight, right? Its just easier to go with whats accurate for most people. Its so inconvenient to have to think about EVERYBODY all the time. Its not like Im em>homophobic/em> or anything. If gay people would just write back and say actually I have a husband or whatever theyd be TOTALLY welcome, Im all cool with that. God, why do you have to be so EASILY OFFENDED? Why do you want to make me feel so guilty?/p>p>So, I mean. Out of Bad Ending Person and Good Ending Person...whose next party do we think is gonna be more inclusive?/p>p>So obviously, in this metaphor, Party Throwing Person is Red Hat, or Google, or Microsoft, or pretty much any company that says hey, we accept this industry has a problem with inclusion and were trying to do better, and the party is our software and communities and events and so on. If you are looking at your communities and wondering why they seem to be pretty white and male and straight, and you ask folks for ideas on how to improve that, and they give you some ideas...just em>listen/em>. And try to take them on board. You asked. Theyre trying to help. They are not saying you are a BAD PERSON who has done BAD THINGS and OFFENDED them and you must feel GUILTY for that. Theyre just trying to help you make a positive change that will help more folks feel more welcome in your communities./p>p>You know, in a weird way, if our Party Throwing Person wasnt quite Good Ending Person or Bad Ending person but instead said hey, you know what, I dont care about women or Black people or gays or whatever, this is a STRAIGHT WHITE GUY PARTY! WOOOOO! SOMEONE TAP THAT KEG!...thats almost em>not as bad/em>. At least you know where you em>stand/em> with that. You dont feel like youre getting gaslit. You can just write that idiot and their party off and try and find another. The kind of Bad Ending Person who keeps insisting theyre not racist or sexist or homophobic and they totally want more minorities to show up at their party but they em>just cant figure out/em> why they all seem to be so awkward and easily offended and why they want to make poor Bad Ending Person feel so guilty...you know...that gets pretty tiring to deal with sometimes./p>                /div>            /article>article classh-entry post-text itemscopeitemscope itemtypehttp://schema.org/Article>header>h1 classp-name entry-title>a hrefposts/2020/06/03/fedora-coreos-test-day-coming-up-on-2020-06-08/ classu-url>Fedora CoreOS Test Day coming up on 2020-06-08/a>/h1>                div classmetadata>                    p classbyline author vcard>span classbyline-name fn itempropauthor>                        Adam Williamson                    /span>/p>            p classdateline>            a hrefposts/2020/06/03/fedora-coreos-test-day-coming-up-on-2020-06-08/ relbookmark>            time classpublished dt-published datetime2020-06-03T22:45:27Z itempropdatePublished title2020-06-03 22:45>2020-06-03 22:45/time>/a>            /p>                /div>            /header>div classe-content entry-content>                    p>Mark your calendars for next Monday, folks: 2020-06-08 will be the a hrefhttps://fedoraproject.org/wiki/Test_Day:Fedora_32_CoreOS_2020-06-08>very first Fedora CoreOS test day/a>! Fedora QA and the CoreOS team are collaborating to bring you this event. Well be asking participants to test the bleeding-edge em>next/em> stream of Fedora CoreOS, run some a hrefhttps://fedoraproject.org/wiki/Category:CoreOS_Test_Cases>test cases/a>, and also read over the a hrefhttps://docs.fedoraproject.org/en-US/fedora-coreos/>documentation/a> and give feedback./p>p>All the details are on the a hrefhttps://fedoraproject.org/wiki/Test_Day:Fedora_32_CoreOS_2020-06-08>Test Day page/a>. You can join in on the day on Freenode IRC, well be using a hrefhttps://webchat.freenode.net/#fedora-coreos>#fedora-coreos/a> rather than #fedora-test-day for this event. Please come by and help out if you have the time!/p>                /div>            /article>/div>            ul classpager postindexpager clearfix>li classnext>a hrefindex-104.html relnext>Older posts/a>/li>        /ul>!--End of body content-->footer idfooter>            Contents © 2025         a href/cdn-cgi/l/email-protection#fe9f9a9f9389d59c929199b0b1adaebfb3be969f8e8e879f8d8d9f8d8d9790d0909b8a>Adam Williamson/a> - Powered by         a hrefhttps://getnikola.com relnofollow>Nikola/a>                             /footer>/div>/div>            script data-cfasyncfalse src/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js>/script>script srcassets/js/jquery.min.js>/script>script srcassets/js/popper.min.js>/script>script srcassets/js/bootstrap.min.js>/script>script srcassets/js/baguetteBox.min.js>/script>script>    baguetteBox.run(div#content, {        ignoreClass: islink,        captions: function(element){var ielement.getElementsByTagName(img)0;return iundefined?:i.alt;}});    /script>/body>/html>

View on OTX | View on ThreatMiner

Data with thanks to AlienVault OTX, VirusTotal, Malwr and others. [Sitemap]

Domain > www.happyassassin.net

Is this malicious?

DNS Resolutions

Port 80

Port 443