Operating systems - 6

I wrote couple of posts about Windows 10 upgrades, but there are no more free upgrades. See:

From now on, you'll just have to pay for it.

There are couple of issues I bumped into while upgrading about dozen PCs last month alone and I'll be blogging about those later. There is no rush, as the free upgrade window closed (pun intended).

Before release of Windows 10 build 10565 in October 2015, it was pretty much impossible to do a clean install for upgrading Windows 7 or 8 without first running the upgrade on the target machine. The announcement said:

Device activation improvements: Microsoft has received a lot of feedback from Insiders on making it easier to activate Windows 10 on devices that take advantage of the free upgrade offer to genuine Windows by using existing Windows 7, Windows 8 or Windows 8.1 product keys. If you install this build of the Windows 10 Insider Preview on a PC and it doesn’t automatically activate, you can enter the product key from a qualifying Windows 7, Windows 8 or Windows 8.1 used to activate the prior Windows version on the same device to activate Windows 10 by going to Settings > Update & security > Activation and selecting Change Product Key.

In short: You can whip up your install USB and do a clean install on a machine. If its not an OEM-machine, just enter your Windows 7 or Windows 8 key at install, and the thing should activate. Nice an easy.

And to clarify the exact build number situation your box is running, go to Settings, System, About. It should have something like this:

As suggested by Ilpo in a comment he left in my blog, there is an alternative method described by article How to Directly Clean Install Windows 10 without having to Upgrade First. Of course I had to try that! And thanks Ilpo, for the suggestion.

What you need is a Windows 10 install USB-stick. Go create one with your favorite method, and if you don't have a favorite, just go for Media creation tool, it can download an install image and store it to your USB-stick for later booting.

When your install-stick is ready, go find <your USB-stick drive>:\support\gatherosstate.exe into the target machine to be upgraded. Do NOT run in at the USB-stick, COPY it. I just dragged the file into my desktop. When ran, the application will create a XML-file containing a fingerprint from the machine ran. That fingerprint can be used to active the upgraded Windows 10 installation later. Needless to say, this type of activation will work only for valid upgrade paths. You cannot upgrade Windows 8.1 with Bing into Windows 10 Pro (I tried).

Second important thing: Run the gatherosstate.exe as an Administrator. I tried to validate the run-as-Admin -requirement after the upgrade was done, but it was too late. The file generated will differ. My thinking is, that it is not necessary to run as Admin, but that's what the original instructions said.

Take a copy of the generated GenuineTicket.xml-file. You will need that later. At this point you're good to go with a clean install. Replace hard drives or just re-partition the original one, whatever your install plan is. I'm guessing you wouldn't be doing the upgrade this way if it wasn't absolutely necessary.

During Windows 10 installation do NOT enter a license key. It is possible to "skip this" and "do this later" and ultimately when the install ends run a perfectly good non-activated Windows 10.

Now its time to go activate the newly installed Windows 10. You need to locate your saved GenuineTicket.xml-file and copy it to directory %ProgramData%\Microsoft\Windows\ClipSVC\GenuineTicket. Like this:

Now the activation is only a reboot away. Go ahead and boot the thing, and confirm:

Done! Simple as pie.

One of my own boxes runs a Fedora Linux. A while back my upgrade failed miserably due to Fedora installer not getting an IP-address from my ISP, Elisa. I had a minor skirmish for an hour or so with the installer, but no avail, Fedora installer beat me on that one. As I love to have that box up and running, I gave up and decided to investigate that later. Now that day came and I'm victorious!

Basics

DHCP is what pretty much everybody has for getting an IPv4 address in 2016. Mobile connections have something different, but everything else including Wi-Fi hotspots, ADSL-routers, Fiber-to-the-Home -connections, etc. etc. issue an IP-address (mostly IPv4, sometimes IPv6) to any well-behaving customer of theirs. Today, the de-facto is that the IP-address is allocated out of a well known broadband address range, or pool. Lists of those are generally available, so that home customers can be differentiated from data centers and companies.

To put DHCP simply, it is a mechanism for allocating an unique address for your Internet connection. The Wikipedia definition for Dynamic Host Configuration Protocol uses more words and isn't as concise as mine, but you'll get the idea.

Details of the problem

In case of mis-use or unpaid internet bill, they'd naturally decline any DHCP-requests for an IP-address. Since everything I tested, including various Windowses, OS Xes and Linuxes worked it wasn't about that. The connection was ok, the DHCP server issued a valid DHCP-lease as it had been doing for couple years, but not for my Fedora installer. Duh?

At this point I went to google for the symptoms and quite soon I landed into RedHat Bugzilla. It contains bug 1154200 which is titled as "not getting a dhcp address assigned". Mr. Krovich reports that his Fedora 21 installer won't get an IP-address from ISP. I pulled up a Fedora 20 installer for the previous version. It worked ok! Yep, they changed something into Fedora 21. The change affects Fedora 22, 23 and the latest 24. It does not affect RedHat nor CentOS (yet).

Fix (aka. trial and error)

In the comments of bug 1154200 they're talking about Option 61 commit which was introduced for Fedora 21 release. A possible fix would be to use DHCP-configuration: send dhcp-client-identifier = hardware;

I downloaded Fedora 24 installer and tested it out. It didn't help any. After a lot of wiresharking the traffic, I isolated this:
In the DHCP Discover -packet, there was an Option 61 present.

More poking for man 5 dhclient-options revealed that it was possible to specify a fixed string for identifier. So, again I edited /etc/dhcp/dhclient.conf (btw. the file didn't exist to begin with, I had to create one) to contain:
send dhcp-client-identifier = "";

That did the trick! Now even Fedora 24 installer got a a valid IP-address and it was possible to install.

Specs

In the early days, all IP-addresses were assigned manually. Everybody was given an IP-address and they punched all the details manually. That was frustrating and error-prone, so somebody invented BOOTP to automate the entire setup. Quite soon, that evolved into DHCP, defined by RFC 1531. When DHCP gained traction and more and more vendors joined the dynamically allocated -game, couple of clarifying iterations later, we're at RFC 2131 for current breed of DHCPing. It is basically the original BOOTP, but with most wrinkles ironed out.

The options are defined at RFC 1533 for DHCP Options and BOOTP Vendor Extensions. Looks like nobody supported Option 61 for a long time. Windowses don't, Apple doesn't, most Linux Distros don't, but RFC 4361 for Node-specific Client Identifiers for Dynamic Host Configuration Protocol Version Four (DHCPv4) must have done it for Fedora-guys. They chose to implement request 560361 (Dhclient doesn't use client-identifier; may cause issues in certain bridged environments) and make sure everybody uses it, with assumption that all ISPs that won't support Client Identifiers will merrily ignore the option. Nice!

Afterwards

I'll target equal blame to my ISP. The Option 61 is well-defined and it should be possible to ignore it. Looks like they're running Alcatel-Lucent hardware there and for some reason it is configured to spit on Option 61 requests.

Naturally I reported the error to my ISP, but you can assume how well that goes. Any regular customer facing clerk won't know DHCP or what it does, nor any options of it. So all I got back is the classic "we'll investigate" -style response. I'm not keeping my hopes up. I have more hope on my own Bugzilla request 1357469 to have an option to enable or disable usage of Option 61 on Fedora. They might even implement that one day.

Anyway, I'm hoping that this post will help somebody strugging to install their Fedora.

I'd like to be clear on this: I did not figure this one out myself, I just bumped into Philip Yip's page about this. All the credit should go to him, I just tested this and confirmed it to be working.

One of the upgrades I did earlier this month was for a Windows Vista. Computer geeks point out two things immediately: 1) Vista! Really? Who would use that voluntarily? 2) well, it cannot be upgraded into Windows 10. Both very true. It just so happened, that there was an OEM Vista running on a computer. Vista will cease to have security upgrades on 10th April 2017, so I suggested to upgrade the thing into a 10 while I did some other work on the thing. Obviously the person owning the poor computer didn't want to pay for the upgrade, so I chose to go with Mr. Yip's method on it.

Prerequisites

For this to happen you will need:

• An USB-stick, minimum size 4 GiB
  • This will be used to host the Windows 10 installation image
  • I went to a supermarket and got a brand new 16 GiB USB3 stick for the speed improvements
• Ability to boot the target computer from the above USB-stick
  • Most computers I've worked with can do this
• Windows 10 ISO, build 10130
  • Ok, this one is harder one to come by, but it is generally available in the Big Net. I'm talking about software piracy here.
  • You will need this specific build. Older or newer ones won't do the trick.
• A piece of software to write this ISO-file into the above USB-stick and make it bootable.
  • My favorite thing for doing that is Rufus. It is a solid piece of software to write images into USB-sticks.
• Windows 10 media creation tool
  • You can get one from Microsoft's website
• Possibility to disable internet connection from the target machine and enable it when needed
• Finally: Possiblity to change the BIOS/UEFI date on the target computer.
  • This trick solely relies on the fact, that you can pretend the date to be in the history.

Also please note, that there is NO UPGRADE possibility here. This is effectively a fresh install. If you choose to re-install on top of your existing Vista installation, please take care no to lose any precious data. I installed a bigger hard drive to the computer, so I made sure that the old drive was intact during this process.

Steps

This is how the entire process goes:

1. Use Rufus to create Windows 10 build 10130 USB-stick
   
2. Boot into BIOS/UEFI setup, set date to anything around September 2015. I just bumped back the year and did my installation in "July 2015".
3. Make sure, there is no Internet connection during the installation. THIS IS IMPORTANT!
4. Boot from Windows 10 build 10130 USB. Make sure there is no Internet connectivity on the machine, so that it is impossible for it to update date via NTP or talk to Microsoft.
   1. Run Windows 10 install (for the first time)
   2. During installation, don't bother installing any security upgrades, you will re-install entire Windows 10 after this one completes.
      
   3. Wait couple of hours. This depends on your hardware speed.
   4. Remove the USB
5. Create a recent Windows 10 install USB
   • I had my own computer for media creation and didn't have to use the target computer for this. This can be a parallel process.
   • When asked about the edition, choose Win 10 Pro
6. Turn off internet time when install done. I found that one to be little tricky, but the setting is there.
   
7. Set a specific license key. For this you'll need to open CMD.exe as Administrator. If you attempt this as a regular user, ie. not right clicking the cmd.exe and selecting "run as administrator", you'll notice that soon. The command to execute is:
   slmgr /ipk 6P99N-YF42M-TPGBG-9VMJP-YKHCF
   
8. Connect to internet. THIS is the first time the install machine is allowed to get there. If your box did connect to The Net before this, you have failed.
9. Activate Windows (remember: Admin prompt):
   slmgr /ato
   
10. Now you have an activated Windows 10 running. You can go to My Computer and Properties to confirm.
    
    However, it's an old edition and you don't want to keep that. The important thing is, that now your computer's fingerprint has been stored into Microsoft's server and any subsequent Windows 10 installations will proceed without entering any license key.
11. Just insert the USB install stick with the newer Windows 10 installer (the one you created on step 5) and run setup.exe on that. There is no need for reboot or anything, just run the installer.
12. Install it again! This is your second installation.
    • I choose to keep nothing. This just makes sure, that all the old garbage will be gone after you're done.
    • Please note, that when asked for license key, don't enter any. Always select "Do this later" or "Skip". This is not needed and your newly installed Windows 10 will be activated at the end.
    • Wait 15 minutes
13. You're done! Your Windows is ready to be used.
14. Remove the first installation stored into C:\Windows.old, you don't need to keep any of that. However, the files are protected by operating system and deleting the entire directory prooved to be little tricky. I ended up booting from the USB-installation stick into rescue-mode and chose the cmd.exe from that. On prompt I entered two commands:
    del /S /F /Q c:\Windows.old
rd /S /Q c:\Windows.old

Finally

This process worked for me on 5th July 2016. I don't think this has anything to do with July 29 free upgrade -limitation.

Also I'm sure guys at Microsoft keep track of their installations. They'll know exactly how many licenses they gave out using this method. My thinking is, that they simply don't care. Pretty much all of the competition is giving out operating systems for free and they cannot for some strange reason. Anybody choosing this method needs to be aware, that in an unfortunate scenario they would deactivate all licenses distributed this way.

Happy installing! I know I created one ex-Vista user very happy with this.

Recently I've been doing a lot of Windows 10 upgrades. The free upgrade offer is about to end in less than 15 days:

There is a counter in Windows.com.

My own adoption to Win10 is pretty high, only one box left to upgrade. I'll have to resort to virtual machines for things, that I really, really need Windows 7 for.

The real-time Windows 10 adoption gauge @ gosquared.com shows one third:

That's pretty good, but not the number Microsoft guys are aiming for.

I regularily follow Mr. Gordon Kelly, a writer who publishes @ forbes.com. In December 2015 he wrote a piece with a title 'Free' Windows 10 Has An Expensive Secret. There he pretty much speculates, that Microsoft really wants to get to one billion (1.000.000.000) Windows 10 installations as fast as possible. Not giving away free upgrades to pretty much everybody wouldn't help in that. However, his post after Microsoft's May 2016 press release (Three Reasons Microsoft Stopped Free Windows 10 Upgrades) indicates, that even he gave up hope on prolonged free upgrades being available. I guess, they didn't want to get to one billion that bad. That's something Mr. Richard Hay @ winsupersite.com predicted last December.

It seems that it is possible to prolong your free upgrade offer by enabled assistive technologies. See Microsoft's blog post about that. But unless you do that, it's time to act on it or do some trickery after July 29th.

For my Windows 10 weeks, I'll be posting couple of articles what I did earlier this month with my various upgrades.

I was about to upgrade an old CentOS 6 box into 7. It was all planned, backups taken, necessary information gathered and USB stick prepared with 7.2 DVD image in it. A shutdown and boot from the installation USB, bunch of settings, date/time, keyboard, network, but Däng! No dice.

My initial attempt was to install from USB, but for some reason the server didn't see the USB volume as a valid installation source. No problem, I thought, let's go for network-install then. The interface was already up and the box could reach Internet ok. Installing from a mirror shouldn't take too long. But no. All I could accomplish was a "Error setting up base repository". I went googling about this and found CentOS 7.2 Netinstall Guide – Network Installation Screenshots.

First I set up installation source as On the network: http://mirror.centos.org/centos/7.2.1511/os/x86_64/ and then This URL refers to a mirror list: Checked. No avail. It took about 8 minutes to get the error, but this approach failed miserably. What /tmp/packaging.log had was:

ERR packaging: failed to grab repo metadata for anaconda: Cannot find a valid baseurl for repo: anaconda
ERR packaging: metadata download for repo anaconda failed after 10 retries

Argh! 8 minutes to determine, that the thing didn't work.

There was plenty of time to plan for the next move. I went to see CentOS mirror list, and picked the local Finnish mirror at nic.FUNET. Setting that as source: http://ftp.funet.fi/pub/mirrors/centos.org/7.2.1511/os/x86_64/ and with This URL refers to a mirror list: set as Unchecked got me butkus, /tmp/packaging.log had:

ERR packaging: failed to grab repo metadata for anaconda: failure: repodata/6990209f63a9fd811f13e830ac3c6de4c5d70a42b1c6873e4329b523d394c3bd-primary.xml.gz from anaconda: [Errno 256] No more mirrors to try.
http://ftp.funet.fi/pub/mirrors/centos.org/7.2.1511/os/x86_64/repodata/
6990209f63a9fd811f13e830ac3c6de4c5d70a42b1c6873e4329b523d394c3bd-primary.xml.gz: [Errno 14] HTTP Error 404 - Not Found

Finally a tangible result. Obviously the HTTP/404 was correct. There is no such file in that directory. It took me about 15 seconds to determine, that the URL should be http://ftp.funet.fi/pub/mirrors/centos.org/7.2.1511/os/x86_64/repodata/
0e54cd65abd3621a0baf9a963eafb1a0ffd53603226f02aadce59635329bc937-primary.xml.gz. Something was off in the installer metadata. But where?

I checked treeinfo at http://ftp.funet.fi/pub/mirrors/centos.org/7.2.1511/os/x86_64/.treeinfo, but no avail. Then my poking around landed at /var/run/install/repo/repodata. It has among others, a file named repomd.xml. Looking at the network version from http://ftp.funet.fi/pub/mirrors/centos.org/7.2.1511/os/x86_64/repodata/repomd.xml made everything clear as crystal.

• Drive had:
  • revision 1449702798
  • 6990209f63a9fd811f13e830ac3c6de4c5d70a42b1c6873e4329b523d394c3bd, the file that doesn't exist in the mirror
• Network had:
  • revision 1449700451
  • 0e54cd65abd3621a0baf9a963eafb1a0ffd53603226f02aadce59635329bc937, the file that does exist

But how to fix this?

My initial attempt was to wget http://ftp.funet.fi/pub/mirrors/centos.org/7.2.1511/os/x86_64/repodata/repomd.xml into /var/run/install/repo/repodata and retry, but that didn't change anything, still the same frustrating error after 10 minute delay.

I rebooted the box and relized, that my change persisted on the USB-drive. Whoa! Anyway, I got things cooking this time. Finally the base repository was accepted, I got to go make installation selection and got the install forward.

What the hell was going on there? Where did the incorrect repomd.xml come from? It isn't in the installation image. Or it is, but it comes from a place I didn't find. Whatever it is, there is something seriously off in the process. Why doesn't the installer try to get the most recent version from the network. It is a network install, after all!! After frustrating couple hours later than anticipated, I finally got the box upgraded. Hopefully this information saves you that time.

Microsoft's policy to annoy every Windows 7 and 8.1 user to the point every single one of them will be crazy is something I've addressed earlier. My previous posts are here and here.

Couple days ago I decided to go and upgrade one of my existing Windows 7 boxes. It's roughtly 2 months time left for me to get my free upgrade, also I had been staring enough the re-releases of Update for Windows 7 for x64-based Systems (KB3035583), which every single time I see the upgrade being offered makes me laugh out loud. As you can see:

It reads:

Recommended update

Install this update to resolve issues in Windows. For a complete listing of the issues that are included in this update, see the associated Microsoft KnowledgeBase article for more information. After you install this item, you may have to restart your computer.

Ok. I chose to "resolve issues" in my Windows. In reality that's just crap that forces me to look at Windows 10 being forced down my throat. That one causes more issues than it resolves. That's a complete lie!

I carefully made sure I have backups and all necessary information and decided that I'm good to go for upgrade. But quite soon I realized, that ... there is no easy way for me to upgrade. As I wrote in my previous blog posts, I've taken drastic measures to deflect any/every possible Microsoft attempt to inject anything related to the upgrade, upgrade widgets, upgrade tools or upgrade pre-loaded files.

So, I decided to go google "windows 10 upgrade", landed at page https://www.microsoft.com/en-us/windows/windows-10-upgrade which very helpfully offers to download a tool called GetWindows10-Web_Default_Attr.exe. Downloading and runnin that does absolutely nothing! I took a peek what it does and it simply runs GWX.exe. Which of course, as a result of my deflector shields again does absolutely nothing. It just fails quietly and exits.

A new plan was needed.

Then I landed on another Microsoft page: https://www.microsoft.com/en-us/software-download/windows10. That one prooved to be much more useful. On that page, there is a download for Windows10Upgrade9252.exe which actually could upgrade my Windows.

The upgrade went fine, I didn't lose too many applications and my user profile survived the upgrade well too. The only thing I had to do was to disable IPv6 privacy randomization. This is my recipe which I run on every one of my Windowses:

netsh interface ipv6 set privacy state=disabled store=active
netsh interface ipv6 set privacy state=disabled store=persistent
netsh interface ipv6 set global randomizeidentifiers=disabled store=active
netsh interface ipv6 set global randomizeidentifiers=disabled store=persistent

Other than that I didn't do much else. And as a result I get to see forced advertisements for Office365. Crap!

Those pushhy bastards at Microsoft really, really want to upgrade every Windows 7, 8 and 9 into a 10. (Yes, Windows 9 doesn't exist.) They even re-released KB3035583 to make sure that any previous blocking since March 2015 wouldn't be affected anymore. This is what my Windows 7 started doing:

This wasn't supposed to be happening. See details of Part 1. I downloaded latest version of GWX Control Panel, I think it is 1.7.3, but it showed nothing special:

There were some deleted Windows 10 files and a single process running, but I re-zapped them with GWX CP. That didn't make the KB3035583 disappear from the updates list.

A solution to make it gone was simpler than I originally thought:

Just selecting Hide update will make it stay gone. I guess MS-guys will be re-re-releasing it eventually, but for the time being, I'm happy with the result.

The problem

Ok. Microsoft has amped up their "upgrade now" campaign to a ridiculous level.

First Microsoft Marketing chief Chris Capossela "warned" about Windows 7 being insecure. Warned in quotes because my initial reaction was: "Oh really!? Is it really possible, that a Windows can not be secure operating system."

While this machine is eligble for a free Win 10 upg, I just don't want to do that yet. I will upgrade eventually, but at the time I choose, not some ignorant corporate chose for me. So, Mr. Capossela explained that users who choose Windows 7 do so “at your own risk, at your own peril” and he revealed Microsoft has concerns about its future software and hardware compatibility, security and more.

Now the latest development is, that Windows 10 upg is a recommended update as they're now aggressively pushing Windows 10 upgrades. Meaning, that have to actively dodge it every time I'll upgrade this box of mine.

Further details about KB2952664 a Compatibility update for upgrading Windows 7 patch @ Softpedia article.

The information

The annoyance looks like this:

In the process list of the victim machine, there is a GWXUX.exe producing that. Actually there are other GWX-prefixed processes capturing your machine, but for the sake of this blog post I'm ignoring that crap.

Ages ago Microsoft release KB article ID 3080351, How to manage Windows 10 notification and upgrade options. It contains all kinds of useless information about modifying registry entries like HKLM\Software\Microsoft\Windows\CurrentVersion\WindowsUpdate\OSUpgrade value ReservationsAllowed.

What the article doesn't mention is, that it also hijacks your disk space at C:\$Windows.~BT and C:\$Windows.~WS. I got a tip from an article and found 5 GiB of crap already loaded to my computer. I ran cmd.exe (with administrator credentials) to free up my SSD-space:

C:\Windows\system32>attrib -h "C:\$Windows.~BT"
C:\Windows\system32>attrib -h "C:\$Windows.~WS"

C:\Windows\system32>cd "C:\$Windows.~BT"
C:\$Windows.~BT>takeown /f . /R /D Y
C:\$Windows.~BT>icacls . /grant Administrators:(OI)(CI)F /T

C:\Windows\system32>cd "C:\$Windows.~WS"
C:\$Windows.~WS>takeown /f . /R /D Y
C:\$Windows.~WS>icacls . /grant Administrators:(OI)(CI)F /T

After doing all that, reading all the information, changing all the registry values and removing pre-loaded waste...   It didn't work. What the article doesn't mention, that one of the GWX-processes runs in background and snoops those registry values and changes them back! Yes. You read it right. The fuckers go to your computer and make better choices for your registery. Their flawed reasoning is, that you obviously have to be some sort of moron not to go for their ultimate product right now, or preferably yesterday. And as you (the moron), the owner of your computer running a lesser "insecure" OS, need Microsoft's help to make better decisions.

OH, COME ON! Not cool.

I'm NOT upgrading to Windows 10 yet, because I (as in me) am in control here. Not Microsoft (as in them).

The solution

After spending countless hours and experiencing a number of setbacks, when the timers kicked on I stumbled into somebody having exactly the same problem as I do. He also had a solution for it. Here is the article Using GWX Control Panel to Permanently Remove the 'Get Windows 10' Icon.

GWX Control Panel looks like this:

With this application, no need to game of Whack-a-Mole anymore. The not wanted and not needed crap will just fly out with a click of a button. Example of getting rid of pre-loaded Windows 10 installation files:

Ultimately the cleaned up machine will report:

List of goodies include:

• Is 'Get Windows 10' icon app running? App not found
• Is 'Get Windows 10' icon app enabled? App not found
• Windows 10 Download folders found? No

Oh yes!

The application has an option to stay as a background process to keep monitoring, that evil Microsoft processes stay gone. I didn't test that, as just kicking all the crap out of my computer did the trick. Now I was back on the drivers' seat. Now I can decide when to do the upgrade.

In my hugely popular article about Transferring Windows 7 OEM license to a new hard drive, I kept insisting, that you absolutely positively need to know your SLP-key before processing. Couple days ago I was working on a recovery of a failed hard drive and realized, that it's not completely true.

As the troubled drive had already been replaced with a brand new SSD, I had the HDD in an USB-dock for investigation purposes:

The new Windows 7 OEM had already been installed, but not activated. As I didn't know the SLP-key I was planning to do the phone activation with COA-key. At that point I realized, that it was possible to use the Magical Jelly Bean Keyfinder from another drive:

It wasn't much simpler than that! A simple copy/paste -operation to get the key in and Windows activated.

The requirement seems to be to point to a %windir% and system32\config\ is assumed on top of that.

Problem

My home Linux-box was outputting more-or-less useless lm_sensor output. Example:

coretemp-isa-0000
Adapter: ISA adapter
Physical id 0:  +36.0°C (high = +80.0°C, crit = +98.0°C)
Core 0:         +34.0°C (high = +80.0°C, crit = +98.0°C)
Core 1:         +31.0°C (high = +80.0°C, crit = +98.0°C)
Core 2:         +36.0°C (high = +80.0°C, crit = +98.0°C)
Core 3:         +33.0°C (high = +80.0°C, crit = +98.0°C)

nct6776-isa-0290
Adapter: ISA adapter
Vcore:          +0.97 V  (min =  +0.00 V, max =  +1.74 V)
in1:            +1.02 V  (min =  +0.00 V, max =  +0.00 V)  ALARM
AVCC:           +3.33 V  (min =  +2.98 V, max =  +3.63 V)
+3.3V:          +3.31 V  (min =  +2.98 V, max =  +3.63 V)
in4:            +1.01 V  (min =  +0.00 V, max =  +0.00 V)  ALARM
in5:            +2.04 V  (min =  +0.00 V, max =  +0.00 V)  ALARM
in6:            +0.84 V  (min =  +0.00 V, max =  +0.00 V)  ALARM
3VSB:           +3.42 V  (min =  +2.98 V, max =  +3.63 V)
Vbat:           +3.36 V  (min =  +2.70 V, max =  +3.63 V)
fan1:             0 RPM  (min =    0 RPM)
fan2:           703 RPM  (min =    0 RPM)
fan3:             0 RPM  (min =    0 RPM)
fan4:           819 RPM  (min =    0 RPM)
fan5:             0 RPM  (min =    0 RPM)
SYSTIN:         +36.0°C  (high =  +0.0°C, hyst = +0.0°C)  ALARM  sensor = thermistor
CPUTIN:         -60.0°C  (high = +80.0°C, hyst = +75.0°C) sensor = thermal diode
AUXTIN:         +35.0°C  (high = +80.0°C, hyst = +75.0°C) sensor = thermistor
PECI Agent 0:   +26.0°C  (high = +80.0°C, hyst = +75.0°C)
                         (crit = +88.0°C)
PCH_CHIP_TEMP:   +0.0°C
PCH_CPU_TEMP:    +0.0°C
PCH_MCH_TEMP:    +0.0°C

That's all great and all, but what the heck are in 1, 4-6 and fan 1-5? Are the in 1, 4-6 readings really reliable? Why are there sensors with 0 RPM readings? CPUTIN indicating -60 degrees, really? PCH-temps are all 0, why?

Investigation

In order to get to bottom of all this, let's start from the chip in question. lm_sensors -setup identified it as NCT6776. For some reason Nuvoton doesn't have the data sheet anymore, but by little bit of googling, a PDF with title NCT6776F / NCT6776D Nuvoton LPC I/O popped up.

Analog inputs:

Following information can be found:

It contains following analog inputs:

• AVCC
• VBAT
• 3VSB
• 3VCC
• CPUVCORE
• VIN0
• VIN1
• VIN2
• VIN3

The good thing is, that first 5 of them are clearly labeled, but inputs 0 through 3 are not. They can be pretty much anything.

Revolution Pulse counters:

When it comes to RPM-readings, following information is available:

That lists following inputs:

• SYSFANIN
• CPUFANIN
• AUXFANIN0
• AUXFANIN1
• AUXFANIN2

Looks like all of those have connectors on my motherboard.

Temperature Sources:

For the temperature measurements, the chip has:

The analog temperature inputs are:

• SMIOVT1
• SMIOVT2
• SMIOVT3
• SMIOVT4
• SMIOVT5
• SMIOVT6

According to the above table, they're mapped into AUXTIN, CPUTIN and SYSTIN.

Also on top of those, there is PECI (Platform Environment Control Interface). A definition says "PECI is a new digital interface to read the CPU temperature of Intel® CPUs". So, there aren't any analog pins for that, but there are readings available, when questioned.

Configuration

A peek in to /etc/sensors3.conf at the definition of the chip shows:

chip "w83627ehf-*" "w83627dhg-*" "w83667hg-*" "nct6775-*" "nct6776-*"
    label in0 "Vcore"
    label in2 "AVCC"
    label in3 "+3.3V"
    label in7 "3VSB"
    label in8 "Vbat"

    set in2_min  3.3 * 0.90
    set in2_max  3.3 * 1.10
    set in3_min  3.3 * 0.90
    set in3_max  3.3 * 1.10
    set in7_min  3.3 * 0.90
    set in7_max  3.3 * 1.10
    set in8_min  3.0 * 0.90
    set in8_max  3.3 * 1.10

And that's all. I guess that would be ok for the generic case, but in my particular box that list of settings doesn't cover half of the inputs.

Solution

Configuration changes

I added following settings for temperature into "chip "w83627ehf-*" "w83627dhg-*" "w83667hg-*" "nct6775-*" "nct6776-*""-section:

label in0 "Vcore"
set in0_min 1.1 * 0.9
set in0_max 1.1 * 1.15

label in1 "+12V"
compute in1 @ * 12, @ / 12
set in1_min 12 * 0.95
set in1_max 12 * 1.1

label in2 "AVCC"
set in2_min 3.3 * 0.95
set in2_max 3.3 * 1.1

label in3 "+3.3V"
set in3_min 3.3 * 0.95
set in3_max 3.3 * 1.1

label in4 "+5V"
compute in4 @ * 5, @ / 5
set in4_min 5 * 0.95
set in4_max 5 * 1.1

ignore in5
ignore in6

label in7 "3VSB"
set in7_min 3.3 * 0.95
set in7_max 3.3 * 1.1

label in8 "Vbat"
set in8_min 3.3 * 0.95
set in8_max 3.3 * 1.1

The obvious problem still stands: what are the undocumented in 1, 4, 5 and 6? Mr. Ian Dobson at Ubuntuforums.org discussion about NCT6776 claims, that in1 is for +12 VDC power and in4 is for +5VDC power. I cannot deny nor confirm that for my board. The Novoton-chip only provides the inputs, but there is absolutely no way of telling how the manufacturer chooses to connect them to various parts of the MoBo. I took the same assumption, so all that was necessary, was to multiply the input data by 12 and 5 to get a proper reading. I don't know what in5 and in6 are for, that's why I remove them from the display. All the other ones are min and max boundaries for the known readings.

The fan settings are machine specific, in my case:

label fan2 "CPU fan"
set fan2_min 200
label fan4 "HDD fan"
set fan4_min 200
ignore fan1
ignore fan3
ignore fan5

As I only have fans connected to 2 out of 5, I'll ignore the not connected ones. For the connected, I set a lower limit of 200 RPM.

Temperatures are motherboard-specific. In my case, I did following additions:

label temp1 "MB"
set temp1_max 38
set temp1_max_hyst 35
label temp3 "CPU"
label temp7 "CPU?"
ignore temp2
ignore temp8
ignore temp9
ignore temp10

The easy part is to remove the values not displaying anything. The hard part is to try to figure out what the measurements indicate. Based on the other readings, temp3 is CPU combined somehow. The other sensor is displaying rougly same values for each core I have there. However, the temp7 is for PECI, but it doesn't behave anything like CPU-temps. It should, but it doesn't. That's why I left a question mark after it.

Resulting output

After the additions, following output is available:

coretemp-isa-0000
Adapter: ISA adapter
Physical id 0:  +48.0°C  (high = +80.0°C, crit = +98.0°C)
Core 0:         +48.0°C  (high = +80.0°C, crit = +98.0°C)
Core 1:         +40.0°C  (high = +80.0°C, crit = +98.0°C)
Core 2:         +43.0°C  (high = +80.0°C, crit = +98.0°C)
Core 3:         +39.0°C  (high = +80.0°C, crit = +98.0°C)

nct6776-isa-0290
Adapter: ISA adapter
Vcore:        +1.22 V  (min =  +0.99 V, max =  +1.26 V)
+12V:        +12.29 V  (min = +11.42 V, max = +13.25 V)
AVCC:         +3.33 V  (min =  +3.14 V, max =  +3.63 V)
+3.3V:        +3.31 V  (min =  +3.14 V, max =  +3.63 V)
+5V:          +5.04 V  (min =  +4.76 V, max =  +5.52 V)
3VSB:         +3.42 V  (min =  +3.14 V, max =  +3.63 V)
Vbat:         +3.38 V  (min =  +3.14 V, max =  +3.63 V)
CPU fan:      912 RPM  (min =  200 RPM)
HDD fan:      897 RPM  (min =  200 RPM)
MB:           +35.0°C  (high = +38.0°C, hyst = +35.0°C)  sensor = thermistor
CPU:          +37.0°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor
CPU?:         +37.0°C  (high = +80.0°C, hyst = +75.0°C)
                       (crit = +88.0°C)

Before taking the readings, I ran sensors -s to set the min/max values.

Now my output starts making sense and I can actually monitor any changes.

PS.
At the time of writing this article, website http://www.lm-sensors.org/ was down for multiple days in a row. I can only hope, that project personnel solves the issue with the web site and it is up at the time you're seeing this.

This is part 2 of my hard drive upgrade. My previous part was about failure to partition a replaced hard drive with GNU Parted: It was just emitting an error of: "The resulting partition is not properly aligned for best performance"

When I had the drive partitioned properly, I failed to proceed with my setup in a yet another mysterious error. My drives are always using LVM, so that I get more control over the filesystem sizes. To get the new partition into LVM, it needs to be associated with a Volume Group (VG). First step is to inform LVM about new physical drive:

# pvcreate /dev/sda1
  Can't open /dev/sda1 exclusively.  Mounted filesystem?

Oh really? It's definitely not mounted, but ... somebody is stealing my resource. The root of this problem is obviously on the fact, that there used to be a PV on that partition, but I replaced the drive and partitioned it. It is entirely possible, that LVM likes to fiddle with my new partition somehow.

The device mapper knows about the partition:

# dmsetup ls
Box_vg1-LogVol_wrk2   (253:9)

That's kind of bad. I guess it likes to hold on into it. Further check of:

# pvdisplay

... indicates, that LVM doesn't know about the partition (yet), but Linux kernel does.

An attempt to fix:

# dmsetup remove Box_vg1-LogVol_wrk2

And new attempt:

# pvcreate /dev/sda1
  Can't open /dev/sda1 exclusively.  Mounted filesystem?

No change. Perhaps a strace will provide helpful details of the problem:

# strace pvcreate /dev/sda1
...
stat("/dev/sda1", {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 1), ...}) = 0
stat("/dev/sda1", {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 1), ...}) = 0
open("/dev/sda1", O_RDWR|O_EXCL|O_DIRECT|O_NOATIME) = -1 EBUSY (Device or resource busy)
...

Reading a fragment of OPEN(2) man page:

OPEN(2)
open, openat, creat - open and possibly create a file

       O_EXCL Ensure that this call creates the file: if this flag is
              specified in conjunction with O_CREAT, and pathname already
              exists, then open() will fail.

              In general, the behavior of O_EXCL is undefined if it is used
              without O_CREAT.  There is one exception: on Linux 2.6 and
              later, O_EXCL can be used without O_CREAT if pathname refers
              to a block device.  If the block device is in use by the
              system (e.g., mounted), open() fails with the error EBUSY.

... confirms the suspicion, that somebody is holding a handle to the block device. Running lsof(8) or fuser(1) yield nothing. It's not a file-handle, when kernel has your block device as hostage.

My only idea at this point was to do a wimpy Windows-style reboot. The thing is: Linux-men don't reboot on anything, but this time I was out of ideas. I'm sure somewhere there is an IOCTL-call to release the handle, but I couldn't find it easily. So, a reboot was in order.

After the reboot: yes results:

# pvcreate /dev/sda1
  Physical volume "/dev/sda1" successfully created

Then I could proceed with my build sequence. Next, associate a Volume Group with the new Pysical Volume. The options would be to to add the drive into an existing VG, or create a new one. I chose the latter:

# vgcreate Box_vg1 /dev/sda1
  Volume group "Box_vg1" successfully created

Then create a logical partition, or Logical Volume in LVM-lingo on the newly created VG:

# lvcreate -L 800G -n LogVol_wrk2 Box_vg1
  Logical volume "LogVol_wrk2" created

As a physical partition also a LV needs to have a filesystem on it, to be usable for the operating system:

# mkfs.ext4 /dev/Box_vg1/LogVol_wrk2
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 209715200 4k blocks and 52428800 inodes
Filesystem UUID: 93be6c97-3ade-4a62-9403-789f64ef73d0
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Now the drive was ready to be mounted and I had plenty of completely empty space waiting for data to be stored on it.

I plugged in a SATA-USB -dock and started looking for my old data. I intentionally had created a VG with precisely the same name as the old drive had, so there was an obvious collision. My syslog had entries about the pvscan:

Nov  8 16:03:21 pvscan: device-mapper: create ioctl on Box_vg1-LogVol_wrk2 failed: Device or resource busy
Nov  8 16:03:21 pvscan: 0 logical volume(s) in volume group "Box_vg1" now active
Nov  8 16:03:21 pvscan: Box_vg1: autoactivation failed.

Yes, that one I had coming. No autoactivation, as VG names collided. A check:

# vgdisplay
  --- Volume group ---
  VG Name               Box_vg0
...
  --- Volume group ---
  VG Name               Box_vg1
...
  --- Volume group ---
  VG Name               Box_vg1
...
  VG UUID               trx8sq-2Mtf-2tfa-2m1P-YPGq-cVzA-6fWflU

No surprises there, there were two Volume Groups with exactly same name. To address them, there are unique identifiers or UUIDs. With UUID, it is possible to rename the VG. Like this:

# vgrename trx8sq-2Mtf-2tfa-2m1P-YPGq-cVzA-6fWflU Box_vgold
  Volume group "Box_vg1" successfully renamed to "Box_vgold"

Now it would be possible to activate and it would appear on udev:

# vgchange -ay Box_vgold
  1 logical volume(s) in volume group "Box_vgold" now active

Now the old data was available at /dev/Box_vgold/LogVol_wrk2 and ready to be mounted and files copied out of it.

Done and mission accomplished! Now I had much more space on a fast drive.

On the other day I was cleaning out junk from my shelfs and found a perfectly good WD Caviar Black hard drive. Obviously in the current SSD-era where your only computer is a laptop and most of your data is stashed into a cloud somewhere, no regular Joe User is using spinning platters.

Hey! I'm not a regular, nor joe. I have a Linux-server running with plenty of capacity in it for my various computing needs. So, the natrural thing to do is to pop out one of the old drives and hook this 1,5 TiB high performing storage monster to replace it. The actual hardware installation on an ATX-case isn't anything worth documenting, but what happens afterwards goes pretty much this sequence: 1) partition the drive, 2) copy all/some of the old data back to it and 3) continue living successfully ever after.

The typical scenario is that something always at least hiccups, if not fails. And as expected, I choked on the 1).

Here goes:

Preparation

The drive had been used previously, and I just wasted the beginning of the drive by writing 10k sectors of nothingness. This will remove all traces of possible partition tables, boot sectors and all the critical metadata of the drive you normally value highly:

# dd if=/dev/zero of=/dev/sda bs=512 count=10000

Pay attetion to the details. It would be advisable to target a correct drive. In my case a regular JBOD-drive really appears as /dev/sda on the Linux-side. On your case, I'm pretty sure your operating system runs on /dev/sda, so please don't wipe that.

Then with GNU Parted, create a GUID partition table (or GPT):

# parted /dev/sda
GNU Parted 3.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mktable gpt

That's it for the preparation part.

Attempt 1: The stupid way

Regardless what's on the drive already (in my case, its completely empty), Parted syntax allows an approach, where you create a partition using the maximum allowed capacity from start 0, to end -1. Like this:

(parted) mkpart LVM ext4 0 -1
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? c

That obviously will emit an error about non-optimal partition alignment. But hey, that's what I asked for. I obviously cancelled that attempt.

Attempt 2: The smart way

A smart approach would be to see about the boundaries:

(parted) print free
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 1500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt Disk Flags:

Number Start  End    Size   File system Name Flags
       17.4kB 1500GB 1500GB Free Space

Now we have a range of 17.4 KiB to 1500 GiB which can be used for a new partition. Let's try that:

(parted) mkpart LVM ext4
Start? 17.4kB
End? 1500GB
Warning: You requested a partition from 16.9kB to 1500GB (sectors 33..2929687500).
The closest location we can manage is 17.4kB to 1500GB (sectors 34..2930277134).
Is this still acceptable to you? Yes/No? y
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? c

I have bumped into this number of times earlier. Why in the f**k cannot the Parted tell me what values it wants to see there!! Come on!

This is the part where it hits me like a hammer: enough bullshit, let's solve this once and for all!

Attempt 3: Solution

This is the script I wrote: parted_mkpart_calc.sh.

It is based on the information found from following sources:

• How to align partitions for best performance using parted, somebody else is having the same fight than I do
• I/O Limits: block sizes, alignment and I/O hints, information about the Parted alignment calculation
• https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-block, Linux kernel block-device ABI information

It is a Bash-script to do the math for you. Example usage:

$ ./parted_mkpart_calc.sh sda
Using default 1 MiB default alignment in calc
Calculated alignment for /dev/sda (gpt) is: 2048s

If you would be root, you could create partition with:
# parted /dev/sda mkpart [name] [type] 2048s 2930276351s
Verify partition alignment with:
# parted /dev/sda align-check optimal 1 Should return: 1 aligned

I just enter one argument to the script: sda. From that, the script deduces the alignment, that should be used when partitioning that block-device. In this case it is 2048 sector boundaries (what it doesn't say is, that a sector contains 512 bytes). But it outputs 2 commands which can be copy/pasted (as root):

parted /dev/sda mkpart [name] [type] 2048s 2930276351s

If you would replace [name] with a partition name and [type] with a partition type, it would create a correctly aligned partition to fill up most of the drive. It won't fill up exactly all of the drive, because of the alignment issues.

To help that issue, I added a feature to do the following:

$ ./parted_mkpart_calc.sh sda LVM ext4

Optionally, you can provide the partition name and type on the command line to get:

parted /dev/sda mkpart LVM ext4 2048s 2930276351s

as output. That's ready-to-go copy/paste material.

Finally, you can verify the correct alignment:

# parted /dev/sda align-check optimal 1
1 aligned

That's the proof, that calc worked ok.

Attempt 4: The simple way

It didn't take long, before I got my first comment on this article. It was simply: "Why didn't you use percentages?". What? What percentages.

Example:

(parted) unit s
(parted) print
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 2930277168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End         Size        File system Name Flags
 1     2048s 2930276351s 2930274305s             LVM
(parted) rm 1
(parted) mkpart LVM ext4 0% 100%
(parted) print
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 2930277168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End         Size        File system Name Flags
 1     2048s 2930276351s 2930274305s             LVM

Using range 0% 100% will produce exactly the same results. Amazing!

So, parted knows the alignment and can use it, but not if you don't first do a rain dance and knock three times on a surface sprinkled with holy water.

Final Words

Why does Parted complain about mis-alignment, but offers no help at all? That's just plain stupid!

Of course, I should add the feature to the source code and offer the patch to FSF, but on the other hand. Naah. I don't want to waste any more energy on this madness.

I previously wrote about upgrading OS X El Capitan. After doing couple of boxes I ran into a SNAFU.

If you don't see anything in that page, that's correct! There is nothing there. It would be a safe assumption, that something had gone wrong.

Here is what web browser console says:

Error was: "Failed to load resource: The certificate for this server is invalid." As the errors were emitting from Amazon CloudFront, it didn't make any sense at all. Either Amazon had some sort of security fault happening, or I did. Unfortunately in such situations, the odds are always against me. I had upgraded couple of Macs already and had no problems with them, this box must have had something wrong with it.

My next move was to get a list of trusted root certificates shipping with an OS X. The list is available in Apple knowledgebase article HT205204. Here is what I got:

Another error: 'Safari can't verify the identity of the website "support.apple.com"'. Right. First Amazon was failing on me, then Apple. At this point I whipped up an already upgraded Mac and went for the page, this time it looked ok:

That was the proof, that something was badly off on that Mac.

For fact gathering I went trough the certificate chain of support.apple.com:

As the certificate wasn't trusted, the page looked horrible and there was no lock-icon on the address bar. The important fact here was, that the root certificate of VeriSign Class 3 Public Primary Certification Authority - G5 had version number 3 and serial number of 25 0C E8 E0 30 61 2E 9F 2B 89 F7 05 4D 7C F8 FD. On the working Mac same certificate:

A completely different serial number of 18 DA D1 98 26 7D E8 BB 4A 21 58 CD CC 6B 2B 4A.

Then the relevant question was: Why do they differ? The facts are at OS X certificate store. It so happens, that all certificates can be viewed and altered via Keychain Access -tool. I went to see the System Roots -keychain:

But that didn't solve my problem! VeriSign Class 3 Public Primary Certification Authority - G5 was there and had the proven correct serial number of 18 DA D1 98 26 7D E8 BB 4A 21 58 CD CC 6B 2B 4A. More poking around, and this is what I found:

A set of Verisign certificates on login-keychain. Weird. One of them was:

There was the 25 0C E8 E0 30 61 2E 9F 2B 89 F7 05 4D 7C F8 FD! The only appropriate action was:
And that solved it! Simply letting the weird ones go made all my websites work again.

But where did those certs come from? By googling I found Why is Symantec/Verisign CA appearing as an invalid authority? [closed] and Invalid certificate after Security Update 2015-004 in Mavericks. They both were pointing a finger to April 2015 security update. The release notes About the security content of OS X Yosemite v10.10.3 and Security Update 2015-004 say:

Certificate Trust Policy
Impact: Update to the certificate trust policy
Description: The certificate trust policy was updated. View the complete list of certificates.

I just happened to update the Mac too early and got a flawed upgrade. It is also possible, that on April, when I got bad certs, I may have gone to Verisign and manually loaded the proper root certificates in to fix my problem at the time. However, it just blew up on my face on OS X 11 upg.

If you never encountered any of this: good. I honestly don't think this issue is touching a wide audience. However, I disclosed this information for archive purposes. If something like this happens in the future, you have a clue what to look for.

Now that OS X El Capitan or version 11 is out, I'll do a refresh for the USB installation instructions.

The thing is ... the upgrade free, but it's big. The amount of downloading needed is easily 6+ GiB. I have 3 Macs to update and I don't want to download the huge package on all of them. So, let's figure out something smarter.

Step 0: Prerequisites

You'll need a bootable USB-stick with capacity of 8 GiB or more. All sticks should boot, but I have encountered some that didn't manage that.

USB-booting a Mac is trickier than a PC. The knowledge base article HT1948 states:
Intel-based Macs support starting from an external USB storage device's volume that:

• Has been formatted with a GUID partition type
• Contains an installation of Mac OS X 10.4.5 or later, or Mac OS X 10.5 or later, which is compatible with (or shipped with) the Mac that the USB device is connected to. Note: You should not use a version of Mac OS X that is earlier ("older") than the version your Mac shipped with.

So, if you just bought an USB-stick, the chances are, that it is MBR-partitioned FAT32. That's my experience of getting new ones. They are incompatible at their current state for USB-booting a Mac. Not to worry, that can be fixed!

If you happen to have a ready-made stick for any previous OS X version, that obviously can be loaded with new installer. You can even skip couple of things during the process as you don't have to reformat the stick.

Step 1: Go download

In your Apple menu (the top left apple-shaped thing at every program's menu), go for Software Update. An alternate is to click the App Store link:

Both options land you on the App Store main screen showing you something like this:

Click the image saying Free Upgrade (free as in beer):

Your Apple ID credentials are required for this free package. They'll keep track of who downloaded and what.

This is the part you'll wait for the download to complete:

When it's all on your machine, the installer will automatically kick in.

Step 2: Go USB

Now that you have the thing in your drive, don't proceed with the upgrade.

When you reach this screen:

do not proceed! You can quit the installer, if you want:

It won't delete the files from your drive. It is also possible to continue installing on that Mac, but don't do it yet. Take a copy of the files first.

Take at least a 8 GiB USB-storage. 4 won't do it, but any larger will. In my case, the USB-stick appeared as /dev/disk3. That may vary on your system. Also it is possible to use some GUI-tools on OS X to format your drives, but as a Linux-nerd I don't know about them.

It is very likely, that the disk is mounted and will display an icon on your desktop, and will appear on your Finder. The diskutil will unmount it automatically on partition, but I wanted to make sure and did:

# sudo diskutil umount /Volumes/MyUSBdrive

Next step is to make sure, the stick is in a Mac-format (this needs to be run as root, that's what the sudo is for). This will partition and format the entire stick into Mac-use:

# sudo /bin/bash
root# diskutil partitionDisk /dev/disk3 1 GPT jhfs+ "OS X El Capitan" 0b

It will say something like this as a result:

Started partitioning on disk3
Unmounting disk
Creating the partition map
Waiting for the disks to reappear
Formatting disk3s2 as Mac OS Extended (Journaled) with name OS X El Capitan
Initialized /dev/rdisk3s2 as a 7 GB case-insensitive HFS Plus volume with a 8192k journal
Mounting disk
Finished partitioning on disk3
/dev/disk3
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *8.0 GB disk3
1: EFI EFI 209.7 MB disk3s1
2: Apple_HFS OS X El Capitan 7.7 GB disk3s2

Next thing is to confirm, that the volume with given name will be mounted:

root# ls -l /Volumes/
total 40
lrwxr-xr-x 1 root admin 1 Aug 29 12:24 Macintosh HD -> /
drwxrwxr-x 7 root wheel 306 Oct 10 12:41 OS X El Capitan

If it does, you're ready to go. Copy the thing into it:

root# cd /Applications/Install\ OS\ X\ El\ Capitan.app/Contents/Resources/
root# ./createinstallmedia --volume /Volumes/OS\ X\ El\ Capitan/ \
--applicationpath /Applications/Install\ OS\ X\ El\ Capitan.app/ \
--nointeraction

It will result in a lengthy process saying:

Erasing Disk: 0%... 10%... 20%... 30%...100%...
Copying installer files to disk...
Copy complete.
Making disk bootable...
Copying boot files...
Copy complete.
Done.

Step 3: Go update

Your stick is ready. This is the part you will be replicating to any of your Macs you want to upgrade.

Reboot the Mac and make sure to boot from the USB. This can be achieved by pressing down option-key during boot:

The official Apple instruction at knowlegebase article HT1948 states:
To start from a USB storage device that meets the above requirements:

1. connect the device
2. restart
3. immediately press and hold the Option key to access Startup Manager

If you successfully followed the steps, you will end up in Mac boot manager:

In that, you pretty much select the drive you want to boot from. In this particular case, making a choice for the recently prepared USB-stick will be a good one. Click the orange "Install OS X El Capitan". Both the gray HD and the network selection are there to confuse you. Ignore them and double click the USB-drive.

Most steps in this upgrade will include lot of waiting. Make sure that you have reserved couple of hours for the upgrade. Screens like this will become familiar to you:

Most time estimates are wild guesses. A 9 minute wait in reality is something like 45 minutes. Eventually the USB-stick finishes booting, and you will end up in a screen saying "To set up the installation of OS X, click Continue". Most screens will refer your upgrade as an install. It is nerve-wrecking thing, because you don't know if it is going to wipe your settings and data, or do a nice upgrade what you'd be expecting. My experience is, that it will upgrade nicely, but it won't say it properly.

Then there is a license screen which you must agree to continue. Then land on OS X Utilities. One thing you can do with a bootable stick is to upgrade or install an OS X:

When you select to go for upgrade or install, there is a welcome to OS X El Capitan screen. Click Continue. Yet another license screen appears. Click Agree and for the confirmation dialog: I have read and agree to the terms of the software license agreement, Agree. Select the install/upgrade destination as the one having a hard drive icon Macintosh HD (typical installations have that), then click Install. As the first thing, the installer/upgrader will prepare for the operation and then go for the real thing:

This is the most time-consuming part. After a minute or so, there will be an not-so-accurate estimate of time remaining for the preparation. When it has run its course, the actual installer will start. So if the screen says 1 minute left, do not believe it for a second. In my iMac, one cup of coffee doesn't do it. You can easily cook and eat a meal and then have the coffee while the installer/upgrader runs. Especially the final phase saying "About a second remaining" will take ages. My hardware isn't especially old or slow, but ... the process is.

Step 4: Finishing touches

After you've done your waiting. A reboot will result. At this point you'll need to login into your precious upgraded Mac with your local user account.

On successful login, you will end up in the OOBE (or out-of-box experience). Apple makes an effort to not allow stolen hardware to be used and they pretty much require you to login to Apple ID during install:

Since I have a 2-factor authentication enabled for my Apple ID, a 2FA-screen will appear during the process:

There are steps about sending your usage data to Apple. Then yet another license screen which you have to agree twice. First on the bottom screen and then on the pop-up that will appear. So if you want to complain later, they'll just say "but you did agree to our terms and conditions". It will look like this:

The last question you'll need to answer is about setting up iCloud Keychain. I choose not to share my passwords into any cloud services, but at this point you'll have the option to enable Keychain:

Little bit of setting up ... this one won't take long. And then you're pretty much done. Finally your upgrade is ready!

Step 5: Done!

Login for the first time:

One of the first things I did, was eject my USB-stick (there were couple other Macs to be updated, too):

At this point, you can continue using your precious Mac.

Aftermath

I updated anyway, as a nerd I like the latest stuff running on my computers. I should yield less problems and there needs to be some progress. I find myself stating the same thing in couple of my blog posts, "it wasn't worth it, but I did it anyway". With computers, it never will result any good to stand still and ignore future.