Write Clean Code — or else …
Monday, December 13. 2021
Anybody who has written any significant amount of code into a software project will (painfully) eventually learn how the code should have been written in its early days.This phenomenon has happened to all of us and will keep happening as long as any code is being maintained.
Experienced developers know how to fight this code "rot", or debt. Here is a really good diagram on how to maintain code quality.
Credit: Artem Diashkin, 2020, https://medium.com/litslink/solid-software-design-principles-ac5be34a6cd5
So, four things:
- You must work with you code all the time to improve it. Preferably not the same exact code, but your codebase in general.
- Without automated tests, you'll be lost. Getting 100% coverage isn't easy and not the actual target even. Have SOME coverage at minimum.
- Implement sensible design patterns, have the factory factory if its applicable for what you're doing. Avoid over-kill, you don't need a factory for everything as it makes your #2 difficult requiring more #1.
- S.O.L.I.D.
- Single Responsibility Principle
- Open Closed Principle
- Liskov Substitution Principle
- Interface Segregation Principle
- Dependency Inversion Principle
Especially #4 is very difficult. Nevertheless, you need to aim towards it.
As reality bites, what happens if you don't do all of the above. Well ... shit will hit the fan. A really good example of things going wrong is Log4Shell.
Christine Dodrill in her post "Open Source" is Broken or: Why I Don't Write Useful Software Unless You Pay Me explains how huge corporations use work of non--paid volunteers while failing to direct any money towards them. When things go wrong or seriously wrong, everybody will blame those non-paid people for not doing their jobs properly. Note how I write "jobs". It was never their job, it was their hobby, something they do in their spare time.
Whatever code you work on, eventually your end result will look as this XKCD #2347 depicts:
When that tiny piece crumbles, the poor Nebraskan will get the blame. Nobody offered to help before it happened, though. We had a taste of that in 2016, How one programmer broke the internet by deleting a tiny piece of code. Last month it happened again.
Everybody wanted to use Apache Log4j, nobody wanted to pay for it (actually, three users did). What these non-paid volunteers maintaining the code didn't do was have the discipline. Pyramid of clean code wasn't applied, or wasn't properly applied. They didn't bother, entire project was simply something they did on their free time. With lack of discipline, legacy version's and what not, the worst was not applying S.O.L.I.D.'s S. The same piece of code that should write the requested log entry to a log destination also does parsing of the message. That's two responsibilities, not single.
From proof-of-concept Java-code:
log.info("This is what we got:{}", someData);
Whatever the variable someData contains, by Single Responsibility Principle, no parsing must be done and any input, for example ${jndi:ldap://127.0.0.1/a}, must be taken as literal. Because of test code coverage, lack of refactoring, not using design patterns properly and lack of S.O.L.I.D. design all hell broke loose.
Internet burst into flames!
Next time you, as a software architect, designed, developer, whatever plan on adding an implicit functionality to your existing code, make sure everybody who will ever use your new toy knows of this dual-responsbility. Inform all of this implicit functionaly and how it will effect. Don't hide a parser somewhere where there should be no parser, have it a separate operation even if it will break prople's legacy applications. Doing this will keep your days boring and Internet not bursting into flames. The bad thing, obviously, is your notoriety will reamain low as nobody will ever know of your really cool feature.
My advice is:
Have it that way rather than being the underpaid guy whose code broke everything. Really, broke everything.
Anybody who has written any significant amount of code into a software project will (painfully) eventually learn how the code should have been written in its early days.This phenomenon has happened to all of us and will keep happening as long as any code is being maintained.
Experienced developers know how to fight this code "rot", or debt. Here is a really good diagram on how to maintain code quality.
Credit: Artem Diashkin, 2020, https://medium.com/litslink/solid-software-design-principles-ac5be34a6cd5
So, four things:
- You must work with you code all the time to improve it. Preferably not the same exact code, but your codebase in general.
- Without automated tests, you'll be lost. Getting 100% coverage isn't easy and not the actual target even. Have SOME coverage at minimum.
- Implement sensible design patterns, have the factory factory if its applicable for what you're doing. Avoid over-kill, you don't need a factory for everything as it makes your #2 difficult requiring more #1.
- S.O.L.I.D.
- Single Responsibility Principle
- Open Closed Principle
- Liskov Substitution Principle
- Interface Segregation Principle
- Dependency Inversion Principle
Especially #4 is very difficult. Nevertheless, you need to aim towards it.
As reality bites, what happens if you don't do all of the above. Well ... shit will hit the fan. A really good example of things going wrong is Log4Shell.
Christine Dodrill in her post "Open Source" is Broken or: Why I Don't Write Useful Software Unless You Pay Me explains how huge corporations use work of non--paid volunteers while failing to direct any money towards them. When things go wrong or seriously wrong, everybody will blame those non-paid people for not doing their jobs properly. Note how I write "jobs". It was never their job, it was their hobby, something they do in their spare time.
Whatever code you work on, eventually your end result will look as this XKCD #2347 depicts:
When that tiny piece crumbles, the poor Nebraskan will get the blame. Nobody offered to help before it happened, though. We had a taste of that in 2016, How one programmer broke the internet by deleting a tiny piece of code. Last month it happened again.
Everybody wanted to use Apache Log4j, nobody wanted to pay for it (actually, three users did). What these non-paid volunteers maintaining the code didn't do was have the discipline. Pyramid of clean code wasn't applied, or wasn't properly applied. They didn't bother, entire project was simply something they did on their free time. With lack of discipline, legacy version's and what not, the worst was not applying S.O.L.I.D.'s S. The same piece of code that should write the requested log entry to a log destination also does parsing of the message. That's two responsibilities, not single.
From proof-of-concept Java-code:
log.info("This is what we got:{}", someData);
Whatever the variable someData contains, by Single Responsibility Principle, no parsing must be done and any input, for example ${jndi:ldap://127.0.0.1/a}, must be taken as literal. Because of test code coverage, lack of refactoring, not using design patterns properly and lack of S.O.L.I.D. design all hell broke loose.
Internet burst into flames!
Next time you, as a software architect, designed, developer, whatever plan on adding an implicit functionality to your existing code, make sure everybody who will ever use your new toy knows of this dual-responsbility. Inform all of this implicit functionaly and how it will effect. Don't hide a parser somewhere where there should be no parser, have it a separate operation even if it will break prople's legacy applications. Doing this will keep your days boring and Internet not bursting into flames. The bad thing, obviously, is your notoriety will reamain low as nobody will ever know of your really cool feature.
My advice is:
Have it that way rather than being the underpaid guy whose code broke everything. Really, broke everything.
Decoding EU Digital COVID Certificate into human-readable format
Saturday, December 11. 2021
Passwords, December 2021 Follow up
Monday, December 6. 2021
This year, I've done quite a few posts about passwords. There are still couple days left before 2022 begins, so another piece about passwords is needed.
Most commonly used passwords
For reason not really known to me, at the end of each year there is a reccurring topic: Lists of most commonly used passwords. Here is one example, a Techspot article from November 2021 The most common passwords of 2021 are outright embarrassing. In the article, they're referring to another article by Nordpass Top 200 most common passwords. Nordpass seems to be the password manager software by NordVPN.
In the Nordpass-article, you can select a list to look at. In their back-end the front-app loads the data as JSON from per-country lists. Example: All countries list has URL of https://nordpass.com/json-data/top-worst-passwords/findings/all.json. The one I, as a Finn, was most interested in was the Finnish most used passwords from https://nordpass.com/json-data/top-worst-passwords/findings/fi.json.
Bit of processing with jq to extract top-30 list with use frequency:
jq -r '.[] | "\(.Rank): \(.Password) [\(.User_count)]"' all.json | head -30
Resulting:
1: 123456 [103170552]
2: 123456789 [ 46027530]
3: 12345 [ 32955431]
4: qwerty [ 22317280]
5: password [ 20958297]
6: 12345678 [ 14745771]
7: 111111 [ 13354149]
8: 123123 [ 10244398]
9: 1234567890 [ 9646621]
10: 1234567 [ 9396813]
11: qwerty123 [ 8933334]
12: 000000 [ 8377094]
13: 1q2w3e [ 8204700]
14: aa12345678 [ 8098805]
15: abc123 [ 7184645]
16: password1 [ 5771586]
17: 1234 [ 5544971]
18: qwertyuiop [ 5197596]
19: 123321 [ 5168171]
20: password123 [ 4681010]
21: 1q2w3e4r5t [ 4624323]
22: iloveyou [ 4387925]
23: 654321 [ 4384762]
24: 666666 [ 4329996]
25: 987654321 [ 4239959]
26: 123 [ 3606937]
27: 123456a [ 3493177]
28: qwe123 [ 3284938]
29: 1q2w3e4r [ 3197899]
30: 7777777 [ 3112046]
Ok. Now we learn using 123456 is really common. Nordpass has recorded 103 million times that password being used.
Q: Where does this password data come from?
Yeah, I know the above list contains really commonly used passwords.Or does it?
From which data do we know the above list to contain top-30 most commonly used passwords? Isn't it kinda suspicious for a password vault company to publish this kind of information? Do they know what passwords do you use? How do they know that 123456 is being used over 100 million times?
There is no answer to the set of questions. Nordpass don't say, so I need to speculate and guess.
Harvesting the passwords people do use
According to John Wetzel of Recorded Future passwords are leaked constantly:
Actually, dumps with leaked passwords are easily available in the net. Even I have millions of passwords from various leaks. That could be one source to measure bad passwords, see which ones are leaked the most.
Q: What if only bad passwords leak?
Either Nordpass cheats and they do know which passwords their customers use, OR there is a case of survivorship bias. The bad password ended up being hijacked, put to a database, leaked and picked up by number of people wanting to see what passwords are being used just because it was poor one to begin with. Back-in-the-days there were cases where users' passwords
I don't know if that's the case. Not disclosing the source makes me wonder if Nordpass's ethics is bad or if their password manger is bad allowing them to see what the password is.
Life after passwords
A lot of systems being used allow users to change password and while doing it put a super simple one. Also a big problem are lists of default passwords of devices sold making the password easily guessable or non-existent as they're commonly known.
In UK they're fighting against passwords hard time: Ban on default passwords in new UK law. Nice! As a lot of device manufacturers choose to go the cheapest way, some legislation will be needed to smack some sense to them. If a device has simple default password, they won't allow selling it. We definitely should have that ruling in effect in the EU-side too!
One mechanism to rid passwords is WebAuthN. As it doesn't seem to get the traction, yet one initiative was launched Decentralized Identity Foundation (DIF). What they're proposing is to kinda reverse the authentication problem and let you control your own data allowing you to define which service will verify you're you to a website you're logging into. That would solve bunch of problems if being commonly used. However, DIF is such a new proposal, we don't know yet if that's going to fly or not.
This year, I've done quite a few posts about passwords. There are still couple days left before 2022 begins, so another piece about passwords is needed.
Most commonly used passwords
For reason not really known to me, at the end of each year there is a reccurring topic: Lists of most commonly used passwords. Here is one example, a Techspot article from November 2021 The most common passwords of 2021 are outright embarrassing. In the article, they're referring to another article by Nordpass Top 200 most common passwords. Nordpass seems to be the password manager software by NordVPN.
In the Nordpass-article, you can select a list to look at. In their back-end the front-app loads the data as JSON from per-country lists. Example: All countries list has URL of https://nordpass.com/json-data/top-worst-passwords/findings/all.json. The one I, as a Finn, was most interested in was the Finnish most used passwords from https://nordpass.com/json-data/top-worst-passwords/findings/fi.json.
Bit of processing with jq to extract top-30 list with use frequency:
jq -r '.[] | "\(.Rank): \(.Password) [\(.User_count)]"' all.json | head -30
Resulting:
1: 123456 [103170552]
2: 123456789 [ 46027530]
3: 12345 [ 32955431]
4: qwerty [ 22317280]
5: password [ 20958297]
6: 12345678 [ 14745771]
7: 111111 [ 13354149]
8: 123123 [ 10244398]
9: 1234567890 [ 9646621]
10: 1234567 [ 9396813]
11: qwerty123 [ 8933334]
12: 000000 [ 8377094]
13: 1q2w3e [ 8204700]
14: aa12345678 [ 8098805]
15: abc123 [ 7184645]
16: password1 [ 5771586]
17: 1234 [ 5544971]
18: qwertyuiop [ 5197596]
19: 123321 [ 5168171]
20: password123 [ 4681010]
21: 1q2w3e4r5t [ 4624323]
22: iloveyou [ 4387925]
23: 654321 [ 4384762]
24: 666666 [ 4329996]
25: 987654321 [ 4239959]
26: 123 [ 3606937]
27: 123456a [ 3493177]
28: qwe123 [ 3284938]
29: 1q2w3e4r [ 3197899]
30: 7777777 [ 3112046]
Ok. Now we learn using 123456 is really common. Nordpass has recorded 103 million times that password being used.
Q: Where does this password data come from?
Yeah, I know the above list contains really commonly used passwords.Or does it?
From which data do we know the above list to contain top-30 most commonly used passwords? Isn't it kinda suspicious for a password vault company to publish this kind of information? Do they know what passwords do you use? How do they know that 123456 is being used over 100 million times?
There is no answer to the set of questions. Nordpass don't say, so I need to speculate and guess.
Harvesting the passwords people do use
According to John Wetzel of Recorded Future passwords are leaked constantly:
Actually, dumps with leaked passwords are easily available in the net. Even I have millions of passwords from various leaks. That could be one source to measure bad passwords, see which ones are leaked the most.
Q: What if only bad passwords leak?
Either Nordpass cheats and they do know which passwords their customers use, OR there is a case of survivorship bias. The bad password ended up being hijacked, put to a database, leaked and picked up by number of people wanting to see what passwords are being used just because it was poor one to begin with. Back-in-the-days there were cases where users' passwords
I don't know if that's the case. Not disclosing the source makes me wonder if Nordpass's ethics is bad or if their password manger is bad allowing them to see what the password is.
Life after passwords
A lot of systems being used allow users to change password and while doing it put a super simple one. Also a big problem are lists of default passwords of devices sold making the password easily guessable or non-existent as they're commonly known.
In UK they're fighting against passwords hard time: Ban on default passwords in new UK law. Nice! As a lot of device manufacturers choose to go the cheapest way, some legislation will be needed to smack some sense to them. If a device has simple default password, they won't allow selling it. We definitely should have that ruling in effect in the EU-side too!
One mechanism to rid passwords is WebAuthN. As it doesn't seem to get the traction, yet one initiative was launched Decentralized Identity Foundation (DIF). What they're proposing is to kinda reverse the authentication problem and let you control your own data allowing you to define which service will verify you're you to a website you're logging into. That would solve bunch of problems if being commonly used. However, DIF is such a new proposal, we don't know yet if that's going to fly or not.
EU Digital COVID Certificate, December 2021 edition
Sunday, December 5. 2021
Since my August blog post about vacdec, the utility to take a peek into COVID-19 passport internals, I've got lots of feedback and questions. The topic, obviously, touches all of us. As this global pandemic won't end, I've been following different events, occurrences and incdients around digital COVID certificates.
TV and Print media bloopers
To anybody working with software, computers, data, encoding and transmitting data, especially in the era of GDPR, it's second nature to classify data. There is public data, confidential data, data containing GDPR-covered personal details, to classify by simple three criteria.
What's puzzling on how EU Digital COVID Certificate QR-codes where shown in media. Personally I contacted four different journalists in fashion of "Hey, you published all of your personal details in form of a QR-code. Please, retract and do not ever do that again." One journalist I talked in person and his comment was "I couldn't fathom this black-and-white blob should be kept secret!"
These leaks weren't by small and insignificant media. The codes I saw were portrayed from 9 o'clock evening TV news (cinematographer's cert), website of print media (journalist's own cert), Internet news video (producer's cert) and paper print of news (not sure whose cert that was).
As these bloopers aren't regularily seen anymore, it seems all of the media outlets have issued internal memos for not to display the QR-codes in a readable format anymore. I have facts of one media outlet's internal memo informing people not to publish real certs publicly. One positive change is most of them are using sample certs when they really want to portray a QR-code.
Brute-forcing ECDSA cryptography
In one instance, I bumped into a script-kiddies masterpiece. He was convinced of the possibility of brute-forcing the ECDSA-256 private key of one COVID Certificate issuer. I eyeballed the Python-code and indeed, it approached the problem by doing an nearly-forever loop of picking 256 random bits to match those bits against the public key. Nice! By carefully choosing the bits, that is how the thing works.
However, some suspicions rose. There was a comment in the discussion thread saying: "I don't think this code is working. My home computer has been crunching this for two weeks now and there doesn't seem to be any results."
Really! REALLY!!
These kids really have no clue.
Hints for brute-forcing ECDSA-crypto:
- If it was meant to be that easy, everybody was cracking the private key.
- Don't do random bits, instead go through 2^256 bits in sequence. By going random, there is no mechanism to check if that combination has been attempted already.
- Go parallel, split the sequence into chunks and do it simultaneously with multiple computers.
- Two to 256th power is: 115792089237316195423570985008687907853269984665640564039457584007913129639936. If for some reason you come to a conclusion of that particular number being on the larger scale, your conclusion is correct. IT IS!
- If some magic lottery bounces your way and you happend to find the correct sequence. For the love of god, do not tell anybody else of your findings. It takes you couple gazillion years to brute force the key, for the opposing party couple seconds to revoke it.
- Make sure to put in couple lottery tickets while at it. Chances are you'll become rich before finding the private key.
Forged Digital COVID Certificates
Somebody in Germany got their hands on the actual certificate issuing private key and did throw out couple fully valid and totally verifiable COVID certificates for different names. There were at least three different ones circulating that I know of. Here is one (notice the green color indicating validity):
This one was targeting Finnish customer pool and had first name of "Rokotepassieu" which translates as "Vaccination passport EU". Surname translated: "Contact me via Wickr" (for those who don't do Wickr, it's an AWS owned messaging app with highest standards on security).
According to Swedish government list of valid certificates, Germany has 57 sets of keys in use (disclaimer: at the time of writing this post). What German government had to do is revoke the public key which private key was misplaced. For thousands of people who had a valid COVID cert, they had to get theirs again with different signing key.
This kind of incident/leakage obviously caught attention from tons of officials. Making sellers' business dry up rather swiftly. Unfortunaly no details of this incident were disclosed. The general guess is for some underpaid person to be doing darknet moonlighting on the side with his access to this rare and protected resource.
Leaked Digital COVID Certificates
In Italy roughly thousand COVID certs issued to actual people were made available. I downloaded and processed couple thousand files in four different leaks just to find out most of the leaked certs were duplicates. I did combine and de-duplicate publicly available data. Here are the stats:
All leaked certs are still valid (disclaimer: at the time of writing this post). However, Italian government has only three sets of keys in place and replacing one would mean rendering 1/3 of all issued certificates as invalid. They have not chosen to do that. Most likely because the certs in circulation are harvested by somebody doing the checking with a malicious mobile app.
No surprises there. Most of them are for twice vaccinated people. Couple certs are found for three times vaccinated, test results and recovered ones. What's sad is the fact that such a leak exists. These QR-codes are for real human beings and their data should be handled with care. Not cool. 
What next
This unfortunate pandemic isn't going anywhere. More and more countries are expanding the use of COVID certs in daily use. For any cert checking to make sense, the cert must be paired with person's ID-card. Crypto math with the certs is rock solid, humans are the weak link here.
Since my August blog post about vacdec, the utility to take a peek into COVID-19 passport internals, I've got lots of feedback and questions. The topic, obviously, touches all of us. As this global pandemic won't end, I've been following different events, occurrences and incdients around digital COVID certificates.
TV and Print media bloopers
To anybody working with software, computers, data, encoding and transmitting data, especially in the era of GDPR, it's second nature to classify data. There is public data, confidential data, data containing GDPR-covered personal details, to classify by simple three criteria.
What's puzzling on how EU Digital COVID Certificate QR-codes where shown in media. Personally I contacted four different journalists in fashion of "Hey, you published all of your personal details in form of a QR-code. Please, retract and do not ever do that again." One journalist I talked in person and his comment was "I couldn't fathom this black-and-white blob should be kept secret!"
These leaks weren't by small and insignificant media. The codes I saw were portrayed from 9 o'clock evening TV news (cinematographer's cert), website of print media (journalist's own cert), Internet news video (producer's cert) and paper print of news (not sure whose cert that was).
As these bloopers aren't regularily seen anymore, it seems all of the media outlets have issued internal memos for not to display the QR-codes in a readable format anymore. I have facts of one media outlet's internal memo informing people not to publish real certs publicly. One positive change is most of them are using sample certs when they really want to portray a QR-code.
Brute-forcing ECDSA cryptography
In one instance, I bumped into a script-kiddies masterpiece. He was convinced of the possibility of brute-forcing the ECDSA-256 private key of one COVID Certificate issuer. I eyeballed the Python-code and indeed, it approached the problem by doing an nearly-forever loop of picking 256 random bits to match those bits against the public key. Nice! By carefully choosing the bits, that is how the thing works.
However, some suspicions rose. There was a comment in the discussion thread saying: "I don't think this code is working. My home computer has been crunching this for two weeks now and there doesn't seem to be any results."
Really! REALLY!!
These kids really have no clue.
Hints for brute-forcing ECDSA-crypto:
- If it was meant to be that easy, everybody was cracking the private key.
- Don't do random bits, instead go through 2^256 bits in sequence. By going random, there is no mechanism to check if that combination has been attempted already.
- Go parallel, split the sequence into chunks and do it simultaneously with multiple computers.
- Two to 256th power is: 115792089237316195423570985008687907853269984665640564039457584007913129639936. If for some reason you come to a conclusion of that particular number being on the larger scale, your conclusion is correct. IT IS!
- If some magic lottery bounces your way and you happend to find the correct sequence. For the love of god, do not tell anybody else of your findings. It takes you couple gazillion years to brute force the key, for the opposing party couple seconds to revoke it.
- Make sure to put in couple lottery tickets while at it. Chances are you'll become rich before finding the private key.
Forged Digital COVID Certificates
Somebody in Germany got their hands on the actual certificate issuing private key and did throw out couple fully valid and totally verifiable COVID certificates for different names. There were at least three different ones circulating that I know of. Here is one (notice the green color indicating validity):
This one was targeting Finnish customer pool and had first name of "Rokotepassieu" which translates as "Vaccination passport EU". Surname translated: "Contact me via Wickr" (for those who don't do Wickr, it's an AWS owned messaging app with highest standards on security).
According to Swedish government list of valid certificates, Germany has 57 sets of keys in use (disclaimer: at the time of writing this post). What German government had to do is revoke the public key which private key was misplaced. For thousands of people who had a valid COVID cert, they had to get theirs again with different signing key.
This kind of incident/leakage obviously caught attention from tons of officials. Making sellers' business dry up rather swiftly. Unfortunaly no details of this incident were disclosed. The general guess is for some underpaid person to be doing darknet moonlighting on the side with his access to this rare and protected resource.
Leaked Digital COVID Certificates
In Italy roughly thousand COVID certs issued to actual people were made available. I downloaded and processed couple thousand files in four different leaks just to find out most of the leaked certs were duplicates. I did combine and de-duplicate publicly available data. Here are the stats:
All leaked certs are still valid (disclaimer: at the time of writing this post). However, Italian government has only three sets of keys in place and replacing one would mean rendering 1/3 of all issued certificates as invalid. They have not chosen to do that. Most likely because the certs in circulation are harvested by somebody doing the checking with a malicious mobile app.
No surprises there. Most of them are for twice vaccinated people. Couple certs are found for three times vaccinated, test results and recovered ones. What's sad is the fact that such a leak exists. These QR-codes are for real human beings and their data should be handled with care. Not cool.
What next
This unfortunate pandemic isn't going anywhere. More and more countries are expanding the use of COVID certs in daily use. For any cert checking to make sense, the cert must be paired with person's ID-card. Crypto math with the certs is rock solid, humans are the weak link here.
Databricks CentOS 8 containers
Wednesday, November 17. 2021
MySQL Java JDBC connector TLSv1 deprecation in CentOS 8
Friday, November 12. 2021
Yeah, a mouthful. Running CentOS 8 Linux, in Java (JRE) a connection to MySQL / MariaDB there seems to be trouble. I think this is a transient issue and eventually it will resolve itself. Right now the issue is real.
Here is the long story.
I was tinkering with Databricks. The nodes for my bricks were on CentOS 8 and I was going to a MariaDB in AWS RDS. with MySQL Connector/J. As you've figured out, it didn't work! Following errors were in exception backtrace:
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
Weird.
Going to the database with a simple CLI-command of (test run on OpenSUSE):
$ mysql -h db-instance-here.rds.amazonaws.com -P 3306 \
-u USER-HERE -p \
--ssl-ca=/var/lib/ca-certificates/ca-bundle.pem \
--ssl-verify-server-cert
... works ok.
Note: This RDS-instance enforces encrypted connection (see AWS docs for details).
Note 2: Term used by AWS is SSL. However, SSL was deprecated decades ago and the protocol used is TLS.
Taking a look on the wire:
Two details popped out instantly: TLSv1 and TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA cipher. Both deprecated. Both deemed highly insecure and potentially leaking your private information.
Why would anybody using those? Don't MySQL/MariaDB/AWS -people remove insecure stuff from their software? What! Why!
Troubleshooting started. First I found SSLHandShakeException No Appropriate Protocol on Stackoverflow. It contains a hint about JVM security settings. Then MySQL documentation 6.3.2 Encrypted Connection TLS Protocols and Ciphers, where they explicitly state "As of MySQL 5.7.35, the TLSv1 and TLSv1.1 connection protocols are deprecated and support for them is subject to removal in a future MySQL version." Well, fair enough, but the bad stuff was still there in AWS RDS. I even found Changes in MySQL 5.7.35 (2021-07-20, General Availability) which clearly states TLSv1 and TLSv1.1 removal to be quite soon.
No amount of tinkering with jdk.tls.disabledAlgorithms in file /etc/java/*/security/java.security helped. I even created a simple Java-tester to make my debugging easier:
import java.sql.*;
// Code from: https://www.javatpoint.com/example-to-connect-to-the-mysql-database
// 1) Compile: javac mysql-connect-test.java
// 2) Run: CLASSPATH=.:./mysql-connector-java-8.0.27.jar java MysqlCon
class MysqlCon {
public static void main(String args[]) {
try {
Class.forName("com.mysql.cj.jdbc.Driver");
Connection con = DriverManager.getConnection("jdbc:mysql://db.amazonaws.com:3306/db", "user", "password");
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery("select * from emp");
while (rs.next())
System.out.println(rs.getInt(1) + " " + rs.getString(2) + " " + rs.getString(3));
con.close();
} catch (Exception e) {
System.out.println(e);
e.printStackTrace(System.out);
}
}
}
Hours passed by, but no avail. Then I found command update-crypto-policies. RedHat documentation Chapter 8. Security, 8.1. Changes in core cryptographic components, 8.1.5. TLS 1.0 and TLS 1.1 are deprecated contains mention of command:
update-crypto-policies --set LEGACY
As it does the trick, I followed up on it. In CentOS / RedHat / Fedora there is /etc/crypto-policies/back-ends/java.config. A symlink pointing to file containing:
jdk.tls.ephemeralDHKeySize=2048
jdk.certpath.disabledAlgorithms=MD2, MD5, DSA, RSA keySize < 2048
jdk.tls.disabledAlgorithms=DH keySize < 2048, TLSv1.1, TLSv1, SSLv3, SSLv2, DHE_DSS, RSA_EXPORT, DHE_DSS_EXPORT, DHE_RSA_EXPORT, DH_DSS_EXPORT, DH_RSA_EXPORT, DH_anon, ECDH_anon, DH_RSA, DH_DSS, ECDH, 3DES_EDE_CBC, DES_CBC, RC4_40, RC4_128, DES40_CBC, RC2, HmacMD5
jdk.tls.legacyAlgorithms=
That's the culprit! It turns out any changes in java.security -file won't have any effect as the policy is loaded later. Running the policy change and set it into legacy-mode has the desired effect. However, running ENTIRE system with such a bad security policy is bad. I only want to connect to RDS, why cannot I lower the security on that only? Well, that's not how Java works.
Entire troubleshooting session was way too much work. People! Get the hint already, no insecure protocols!
Yeah, a mouthful. Running CentOS 8 Linux, in Java (JRE) a connection to MySQL / MariaDB there seems to be trouble. I think this is a transient issue and eventually it will resolve itself. Right now the issue is real.
Here is the long story.
I was tinkering with Databricks. The nodes for my bricks were on CentOS 8 and I was going to a MariaDB in AWS RDS. with MySQL Connector/J. As you've figured out, it didn't work! Following errors were in exception backtrace:
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
Weird.
Going to the database with a simple CLI-command of (test run on OpenSUSE):
$ mysql -h db-instance-here.rds.amazonaws.com -P 3306 \
-u USER-HERE -p \
--ssl-ca=/var/lib/ca-certificates/ca-bundle.pem \
--ssl-verify-server-cert
... works ok.
Note: This RDS-instance enforces encrypted connection (see AWS docs for details).
Note 2: Term used by AWS is SSL. However, SSL was deprecated decades ago and the protocol used is TLS.
Taking a look on the wire:
Two details popped out instantly: TLSv1 and TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA cipher. Both deprecated. Both deemed highly insecure and potentially leaking your private information.
Why would anybody using those? Don't MySQL/MariaDB/AWS -people remove insecure stuff from their software? What! Why!
Troubleshooting started. First I found SSLHandShakeException No Appropriate Protocol on Stackoverflow. It contains a hint about JVM security settings. Then MySQL documentation 6.3.2 Encrypted Connection TLS Protocols and Ciphers, where they explicitly state "As of MySQL 5.7.35, the TLSv1 and TLSv1.1 connection protocols are deprecated and support for them is subject to removal in a future MySQL version." Well, fair enough, but the bad stuff was still there in AWS RDS. I even found Changes in MySQL 5.7.35 (2021-07-20, General Availability) which clearly states TLSv1 and TLSv1.1 removal to be quite soon.
No amount of tinkering with jdk.tls.disabledAlgorithms in file /etc/java/*/security/java.security helped. I even created a simple Java-tester to make my debugging easier:
import java.sql.*;
// Code from: https://www.javatpoint.com/example-to-connect-to-the-mysql-database
// 1) Compile: javac mysql-connect-test.java
// 2) Run: CLASSPATH=.:./mysql-connector-java-8.0.27.jar java MysqlCon
class MysqlCon {
public static void main(String args[]) {
try {
Class.forName("com.mysql.cj.jdbc.Driver");
Connection con = DriverManager.getConnection("jdbc:mysql://db.amazonaws.com:3306/db", "user", "password");
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery("select * from emp");
while (rs.next())
System.out.println(rs.getInt(1) + " " + rs.getString(2) + " " + rs.getString(3));
con.close();
} catch (Exception e) {
System.out.println(e);
e.printStackTrace(System.out);
}
}
}
Hours passed by, but no avail. Then I found command update-crypto-policies. RedHat documentation Chapter 8. Security, 8.1. Changes in core cryptographic components, 8.1.5. TLS 1.0 and TLS 1.1 are deprecated contains mention of command:
update-crypto-policies --set LEGACY
As it does the trick, I followed up on it. In CentOS / RedHat / Fedora there is /etc/crypto-policies/back-ends/java.config. A symlink pointing to file containing:
jdk.tls.ephemeralDHKeySize=2048
jdk.certpath.disabledAlgorithms=MD2, MD5, DSA, RSA keySize < 2048
jdk.tls.disabledAlgorithms=DH keySize < 2048, TLSv1.1, TLSv1, SSLv3, SSLv2, DHE_DSS, RSA_EXPORT, DHE_DSS_EXPORT, DHE_RSA_EXPORT, DH_DSS_EXPORT, DH_RSA_EXPORT, DH_anon, ECDH_anon, DH_RSA, DH_DSS, ECDH, 3DES_EDE_CBC, DES_CBC, RC4_40, RC4_128, DES40_CBC, RC2, HmacMD5
jdk.tls.legacyAlgorithms=
That's the culprit! It turns out any changes in java.security -file won't have any effect as the policy is loaded later. Running the policy change and set it into legacy-mode has the desired effect. However, running ENTIRE system with such a bad security policy is bad. I only want to connect to RDS, why cannot I lower the security on that only? Well, that's not how Java works.
Entire troubleshooting session was way too much work. People! Get the hint already, no insecure protocols!
macOS Monterey upgrade
Monday, November 1. 2021
macOS 12, that one I had been waiting. Reason in my case was WebAuthN. More about that is in my article about iOS 15.
The process is as you can expect. Simple.
Download is big-ish, over 12 gigabytes:
After the wait, an install will launch. At this point I'll typically quit to create the USB-stick. This way I'll avoid downloading the same thing into all of my Macs.
To create the installer, I'll erase an inserted stick with typical command of:
diskutil partitionDisk /dev/disk2 1 GPT jhfs+ "macOS Monterey" 0b
Then change into /Applications/Install macOS Monterey.app/Contents/Resources and run command:
./createinstallmedia \
--volume /Volumes/macOS\ Monterey/ \
--nointeraction
It will output the customary erasing, making bootable, copying and done as all other macOSes before this:
Erasing disk: 0%... 10%... 20%... 30%... 100%
Making disk bootable...
Copying to disk: 0%... 10%... 20%... 30%... 40%... 50%... 60%... 70%... 80%... 90%... 100%
Install media now available at "/Volumes/Install macOS Monterey"
Now stick is ready. Either boot from it, or re-run the Monterey installed from App Store.
When all the I's have been dotted and T's have been crossed, you'll be able to log into your newly upgraded macOS and verify the result:
At this point disappointment hit me. The feature I was looking for, WebAuthN or Syncing Platform Authenticator as Apple calls it wasn't available in Safari. To get it working, follow instructions in Apple Developer article Supporting Passkeys. First enable Developer-menu for your Safari (if you haven't already) and secondly, in it:
Tick the box on Enable Syncing Platform Authenticator. Done! Ready to go.
Now I went to https://webauthn.io/, registered and account with the Mac's Safari, logged in with WebAuthN to confirm it works on the Mac's Safari. Then I took my development iPhone with iOS 15.2 beta and with iOS Safari went to the same site and logged in using the same username. Not using a password! Nice. 
Maybe in near future WebAuthN will be enabled by default for all of us. Now unfortunate tinkering is required. Anyway, this is a really good demo how authentication should work, cross-platform, without using any of the insecure passwords.
macOS 12, that one I had been waiting. Reason in my case was WebAuthN. More about that is in my article about iOS 15.
The process is as you can expect. Simple.
Download is big-ish, over 12 gigabytes:
After the wait, an install will launch. At this point I'll typically quit to create the USB-stick. This way I'll avoid downloading the same thing into all of my Macs.
To create the installer, I'll erase an inserted stick with typical command of:
diskutil partitionDisk /dev/disk2 1 GPT jhfs+ "macOS Monterey" 0b
Then change into /Applications/Install macOS Monterey.app/Contents/Resources and run command:
./createinstallmedia \
--volume /Volumes/macOS\ Monterey/ \
--nointeraction
It will output the customary erasing, making bootable, copying and done as all other macOSes before this:
Erasing disk: 0%... 10%... 20%... 30%... 100%
Making disk bootable...
Copying to disk: 0%... 10%... 20%... 30%... 40%... 50%... 60%... 70%... 80%... 90%... 100%
Install media now available at "/Volumes/Install macOS Monterey"
Now stick is ready. Either boot from it, or re-run the Monterey installed from App Store.
When all the I's have been dotted and T's have been crossed, you'll be able to log into your newly upgraded macOS and verify the result:
At this point disappointment hit me. The feature I was looking for, WebAuthN or Syncing Platform Authenticator as Apple calls it wasn't available in Safari. To get it working, follow instructions in Apple Developer article Supporting Passkeys. First enable Developer-menu for your Safari (if you haven't already) and secondly, in it:
Tick the box on Enable Syncing Platform Authenticator. Done! Ready to go.
Now I went to https://webauthn.io/, registered and account with the Mac's Safari, logged in with WebAuthN to confirm it works on the Mac's Safari. Then I took my development iPhone with iOS 15.2 beta and with iOS Safari went to the same site and logged in using the same username. Not using a password! Nice.
Maybe in near future WebAuthN will be enabled by default for all of us. Now unfortunate tinkering is required. Anyway, this is a really good demo how authentication should work, cross-platform, without using any of the insecure passwords.
OpenSSH 8.8 dropped SHA-1 support
Monday, October 25. 2021
Why Windows 11 won't be a huge success
Tuesday, October 12. 2021
Lot of controversy on Microsoft's surprise release of Windows 11. I'm not talking about when they declared Windows 10 to be the "last Windows" and then releasing 11. Also changes in GUI have lots of discussion points. Neither the forced requirement of TPM2.0 which can be lowered to TPM1.2 with a registry change is not the deal-breaker. Is somebody whispering "Vista" back there?
What really makes all the dominoes fall is the lack of CPU-support.
In above graphs is visualization of three pieces of informatinon I scraped. First I took the list of Windows 11 supported CPUs from Windows Hardware Developer - Windows Processor Requirements. Then I went for https://ark.intel.com/ to collect Intel CPU data. Also did the same for https://www.amd.com/en/products/specifications/. As for some reason AMD releases only information from 2016 onwards, for visualization I cut the Intel to match the same.
Ultimate conclusion is, Microsoft won't support all Intel CPUs released after Q2 2017. With AMD stats are even worse, threshold is somewhere around Q2 2018 and not all CPUs are supported. Percentages for AMD are better, but also their volume is smaller.
If you want to do the same, I published my source code into https://github.com/HQJaTu/Windows-CPU-support-scraper. When run, it will produce a Google Spreadsheet like this:
... which can bevisualized further to produce above graphs.
One of my laptops is a Lenovo T570 from 2017. It ticks all the Windows 11 installer boxes, except CPU-support. Obviously, CPU-support situation is likely to change on progression of time, so I may need to keep running the scripts every quarter to see if there will be better support in the later releases of Windows 11.
Lot of controversy on Microsoft's surprise release of Windows 11. I'm not talking about when they declared Windows 10 to be the "last Windows" and then releasing 11. Also changes in GUI have lots of discussion points. Neither the forced requirement of TPM2.0 which can be lowered to TPM1.2 with a registry change is not the deal-breaker. Is somebody whispering "Vista" back there?
What really makes all the dominoes fall is the lack of CPU-support.
In above graphs is visualization of three pieces of informatinon I scraped. First I took the list of Windows 11 supported CPUs from Windows Hardware Developer - Windows Processor Requirements. Then I went for https://ark.intel.com/ to collect Intel CPU data. Also did the same for https://www.amd.com/en/products/specifications/. As for some reason AMD releases only information from 2016 onwards, for visualization I cut the Intel to match the same.
Ultimate conclusion is, Microsoft won't support all Intel CPUs released after Q2 2017. With AMD stats are even worse, threshold is somewhere around Q2 2018 and not all CPUs are supported. Percentages for AMD are better, but also their volume is smaller.
If you want to do the same, I published my source code into https://github.com/HQJaTu/Windows-CPU-support-scraper. When run, it will produce a Google Spreadsheet like this:
... which can bevisualized further to produce above graphs.
One of my laptops is a Lenovo T570 from 2017. It ticks all the Windows 11 installer boxes, except CPU-support. Obviously, CPU-support situation is likely to change on progression of time, so I may need to keep running the scripts every quarter to see if there will be better support in the later releases of Windows 11.
WebAuthN Practically - iOS 15
Monday, September 20. 2021
As Apple has recently released iOS 15, and iPadOS 15 and macOS 12 will be released quite soon. Why that is important is for Apple's native support for WebAuthN. In my WebAuthN introduction -post there is the release date for the spec: W3C Recommendation, 8 April 2021. Given the finalization of the standard, Apple was the first major player to step forward and start supporting proper passwordless authentication in it's operating systems. For more details, see The Verge article iOS 15 and macOS 12 take a small but significant step towards a password-less future.
For traditional approach with USB-cased Yubikey authenticator, see my previous post.
Registration
Step 1: Enter the username you'd like to register as.
Step 2: Go for Register
Step 3: Your browser will need a confirmation for proceeding with registration.
In Apple's ecosystem, the private key is stored into Apple's cloud (what!?). To allow access to your cloud-based secerts-storage, you must enter your device's PIN-code and before doing that, your permission to proceed is required.
Note: The option for "Use Security Key" is for using the Yubikey in Lightning-port. Both are supported. It is entirely possible to login using the same authenticator with a USB-C in my PC or Mac and Lightning with my iPhone or iPad.
Step 4: Enter your device PIN-code
Step 5: You're done! Now you have successfully registered.
Best part: No passwords! Private key is stored into Syncing Platform Authenticator. Btw. weird name that for WebAuthN in Apple-lingo. Ok, to be honest, WebAuthN is a mouthful too.
This was couple steps simpler than with Yubikey. Also there is the benefit (and danger) of cloud. Now your credential can be accessed from your other devices too.
Login
Step 1: Enter the username you'd like to log in as.
Step 2: Go for Login
Step 3: Your browser will need a confirmation for proceeding with login. A list of known keys and associated user names will be shown.
Step 4: Enter your device PIN-code
Step 5: You're done! Now you have successfully logged in.
Best part: No passwords!
That's it. Really.
Finally
I don't think there is much more to add into it.
In comparison to Yubikey, any of your Apple-devices are authenticators and can share the private key. Obviously, you'll need iOS 15 or macOS 12 for that support.
As Apple has recently released iOS 15, and iPadOS 15 and macOS 12 will be released quite soon. Why that is important is for Apple's native support for WebAuthN. In my WebAuthN introduction -post there is the release date for the spec: W3C Recommendation, 8 April 2021. Given the finalization of the standard, Apple was the first major player to step forward and start supporting proper passwordless authentication in it's operating systems. For more details, see The Verge article iOS 15 and macOS 12 take a small but significant step towards a password-less future.
For traditional approach with USB-cased Yubikey authenticator, see my previous post.
Registration
Step 1: Enter the username you'd like to register as.
Step 2: Go for Register
Step 3: Your browser will need a confirmation for proceeding with registration.
In Apple's ecosystem, the private key is stored into Apple's cloud (what!?). To allow access to your cloud-based secerts-storage, you must enter your device's PIN-code and before doing that, your permission to proceed is required.
Note: The option for "Use Security Key" is for using the Yubikey in Lightning-port. Both are supported. It is entirely possible to login using the same authenticator with a USB-C in my PC or Mac and Lightning with my iPhone or iPad.
Step 4: Enter your device PIN-code
Step 5: You're done! Now you have successfully registered.
Best part: No passwords! Private key is stored into Syncing Platform Authenticator. Btw. weird name that for WebAuthN in Apple-lingo. Ok, to be honest, WebAuthN is a mouthful too.
This was couple steps simpler than with Yubikey. Also there is the benefit (and danger) of cloud. Now your credential can be accessed from your other devices too.
Login
Step 1: Enter the username you'd like to log in as.
Step 2: Go for Login
Step 3: Your browser will need a confirmation for proceeding with login. A list of known keys and associated user names will be shown.
Step 4: Enter your device PIN-code
Step 5: You're done! Now you have successfully logged in.
Best part: No passwords!
That's it. Really.
Finally
I don't think there is much more to add into it.
In comparison to Yubikey, any of your Apple-devices are authenticators and can share the private key. Obviously, you'll need iOS 15 or macOS 12 for that support.
WebAuthN Practically - Yubikey
Sunday, September 19. 2021
Basics of WebAuthN have been covered in a previous post. Go see it first.
As established earlier, WebAuthN is about specific hardware, an authenticator device. Here are some that I use:
These USB-A / USB-C / Apple Lightning -connectibe Yubikey devices are manufactured by Yubico. More info about Yubikeys can be found from https://www.yubico.com/products/.
To take a WebAuthN authenticator for a test-drive is very easy. There is a demo site run by Yubico at https://demo.yubico.com/ containing WebAuthN site. However, as a personal preference I like Duo Security's demo site better. This Cisco Systems, Inc. subsidiary specializes on multi-factor authentication and are doing a great job running a WebAuthN demo site at https://webauthn.io/.
Registration
This illustrated guide is run using a Firefox in Windows 10. I've done this same thing with Chrome, Edge (the chromium one) and macOS Safari. It really doesn't differ that much from each other.
In every website, a one-time user registration needs to be done. This is how WebAuthN would handle the process.
Step 1: Enter the username you'd like to register as.
Step 2: Go for Register
Step 3: Your browser will need a confirmation for proceeding with registration.
The main reason for doing this is to make you, as the user, aware that this is not a login. Also the authenticator devices typically have limited space for authentication keys available. For example: Yubikeys have space for 25 keys in them. The bad thing about limited space is because of high level of security yielding low level of usability. You cannot list nor manage the keys stored. What you can do is erase all of them clean.
Step 4: Insert your authenticator into your computing device (PC / Mac / mobile).
If authenticator is already there, this step will not be displayed.
Step 5: Enter your authenticator PIN-code.
If you have not enabled the second factor, this step won't be displayed.
To state the obvious caveat here, anybody gaining access to your authenticator will be able to log in as you. You really should enable the PIN-code for increased security.
Step 6: Touch the authenticator.
The physical act of tringgering the registration is a vital part of WebAuthN. A computer, possibly run by a malicious cracker, won't be able to use your credentials without human interaction.
Step 7: You're done! Now you have successfully registered.
Best part: No passwords!
In this Duo Security test site, the sandbox will be raked on daily basis. Natually on a real non-demo site your information will be persisted much longer. Also note how your contact information like, E-mail address, mobile number or such wasn't asked. A real site would obviously query more of your personal details. Secondly, WebAuthN best practice is to have multiple authenticators associated with your user account. If you happen to misplace the device used initially for registration, having a backup(s) is advisable.
Next, let's see how this newly created user account works in a practical login -scenario.
Login
Step 1: Enter the username you'd like to log in as.
Step 2: Go for Login
Step 3: Insert your authenticator into your computing device (PC / Mac / mobile).
If authenticator is already there, this step will not be displayed.
Step 4: Enter your authenticator PIN-code.
If you have not enabled the second factor, this step won't be displayed.
Step 5: Touch the authenticator.
Again, human is needed here to confirm the act of authentication.
Step 6: You're done! Now you have successfully logged in.
Best part: No passwords!
Note how the public key can be made, well... public. It really doesn't make a difference if somebody else gets a handle of my public key.
Closer look into The Public Key
As established in the previous post, you can not access the private key. Even you, the owner of the authenticator device, can not access that information. Nobody can lift the private key, possibly without you knowing about it. Just don't lose the Yubikey.
Public-part of the key is known and can be viewed. The key generated by my Yubikey in PEM-format is as follows:
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAERbbifY+euxnszcMis99CsnH81Bhd
3EEG9B2Oh8VpgZPdFlF1OQ8FEbfuSxbbAK+l0mUOb7pJCODDUDqZ9lLrMw==
-----END PUBLIC KEY-----
Popping the ASN.1 cork with a openssl ec -pubin -noout -text -in webauthn-pem.key will result:
read EC key
Public-Key: (256 bit)
pub:
04:45:b6:e2:7d:8f:9e:bb:19:ec:cd:c3:22:b3:df:
42:b2:71:fc:d4:18:5d:dc:41:06:f4:1d:8e:87:c5:
69:81:93:dd:16:51:75:39:0f:05:11:b7:ee:4b:16:
db:00:af:a5:d2:65:0e:6f:ba:49:08:e0:c3:50:3a:
99:f6:52:eb:33
ASN1 OID: prime256v1
NIST CURVE: P-256
From that we learn, the key-pair generated is an ECDSA 256-bit. Known aliases for that are secp256r1, NIST P-256 and prime256v1. That weird naming means elliptic curves, aka. named curves.
For those into math, the actual arithmetic equation of secp256r1 -named curve can be viewed in an open-source book by Svetlin Nakov, PhD at https://cryptobook.nakov.com/. All the source code in this freely available book are at https://github.com/nakov/practical-cryptography-for-developers-book. The mathemathical theory how WebAuthN signs the messages is described in detail at https://cryptobook.nakov.com/digital-signatures/ecdsa-sign-verify-messages.
Back to those "pub"-bytes. Reading RFC5480 indicates out of those 65 bytes, the first one, valued 04, indicates this data being for an uncompressed key. With that information, we know rest of the bytes are the actual key values. What remains is a simple act of splitting the remaining 64 bytes into X and Y, resulting two 32-byte integers in hex:
X: 45b6e27d8f9ebb19eccdc322b3df42b271fcd4185ddc4106f41d8e87c5698193
Y: dd165175390f0511b7ee4b16db00afa5d2650e6fba4908e0c3503a99f652eb33
A simple conversion with bc will result in decimal:
X: 31532715897827710605755558209082448985317854901772299252353894644783958819219
Y: 100000572374103825791155746008338130915128983826116118509861921470022744730419
Yes, that's 77 and 78 decimal numbers in them. Feel free to go after the prime number with that public information!
Finally
The mantra is: No passwords.
With WebAuthN, you'll get hugely improved security with multiple authentication factors built into it. What we need is this to spread and go into popular use!
Basics of WebAuthN have been covered in a previous post. Go see it first.
As established earlier, WebAuthN is about specific hardware, an authenticator device. Here are some that I use:
These USB-A / USB-C / Apple Lightning -connectibe Yubikey devices are manufactured by Yubico. More info about Yubikeys can be found from https://www.yubico.com/products/.
To take a WebAuthN authenticator for a test-drive is very easy. There is a demo site run by Yubico at https://demo.yubico.com/ containing WebAuthN site. However, as a personal preference I like Duo Security's demo site better. This Cisco Systems, Inc. subsidiary specializes on multi-factor authentication and are doing a great job running a WebAuthN demo site at https://webauthn.io/.
Registration
This illustrated guide is run using a Firefox in Windows 10. I've done this same thing with Chrome, Edge (the chromium one) and macOS Safari. It really doesn't differ that much from each other.
In every website, a one-time user registration needs to be done. This is how WebAuthN would handle the process.
Step 1: Enter the username you'd like to register as.
Step 2: Go for Register
Step 3: Your browser will need a confirmation for proceeding with registration.
The main reason for doing this is to make you, as the user, aware that this is not a login. Also the authenticator devices typically have limited space for authentication keys available. For example: Yubikeys have space for 25 keys in them. The bad thing about limited space is because of high level of security yielding low level of usability. You cannot list nor manage the keys stored. What you can do is erase all of them clean.
Step 4: Insert your authenticator into your computing device (PC / Mac / mobile).
If authenticator is already there, this step will not be displayed.
Step 5: Enter your authenticator PIN-code.
If you have not enabled the second factor, this step won't be displayed.
To state the obvious caveat here, anybody gaining access to your authenticator will be able to log in as you. You really should enable the PIN-code for increased security.
Step 6: Touch the authenticator.
The physical act of tringgering the registration is a vital part of WebAuthN. A computer, possibly run by a malicious cracker, won't be able to use your credentials without human interaction.
Step 7: You're done! Now you have successfully registered.
Best part: No passwords!
In this Duo Security test site, the sandbox will be raked on daily basis. Natually on a real non-demo site your information will be persisted much longer. Also note how your contact information like, E-mail address, mobile number or such wasn't asked. A real site would obviously query more of your personal details. Secondly, WebAuthN best practice is to have multiple authenticators associated with your user account. If you happen to misplace the device used initially for registration, having a backup(s) is advisable.
Next, let's see how this newly created user account works in a practical login -scenario.
Login
Step 1: Enter the username you'd like to log in as.
Step 2: Go for Login
Step 3: Insert your authenticator into your computing device (PC / Mac / mobile).
If authenticator is already there, this step will not be displayed.
Step 4: Enter your authenticator PIN-code.
If you have not enabled the second factor, this step won't be displayed.
Step 5: Touch the authenticator.
Again, human is needed here to confirm the act of authentication.
Step 6: You're done! Now you have successfully logged in.
Best part: No passwords!
Note how the public key can be made, well... public. It really doesn't make a difference if somebody else gets a handle of my public key.
Closer look into The Public Key
As established in the previous post, you can not access the private key. Even you, the owner of the authenticator device, can not access that information. Nobody can lift the private key, possibly without you knowing about it. Just don't lose the Yubikey.
Public-part of the key is known and can be viewed. The key generated by my Yubikey in PEM-format is as follows:
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAERbbifY+euxnszcMis99CsnH81Bhd
3EEG9B2Oh8VpgZPdFlF1OQ8FEbfuSxbbAK+l0mUOb7pJCODDUDqZ9lLrMw==
-----END PUBLIC KEY-----
Popping the ASN.1 cork with a openssl ec -pubin -noout -text -in webauthn-pem.key will result:
read EC key
Public-Key: (256 bit)
pub:
04:45:b6:e2:7d:8f:9e:bb:19:ec:cd:c3:22:b3:df:
42:b2:71:fc:d4:18:5d:dc:41:06:f4:1d:8e:87:c5:
69:81:93:dd:16:51:75:39:0f:05:11:b7:ee:4b:16:
db:00:af:a5:d2:65:0e:6f:ba:49:08:e0:c3:50:3a:
99:f6:52:eb:33
ASN1 OID: prime256v1
NIST CURVE: P-256
From that we learn, the key-pair generated is an ECDSA 256-bit. Known aliases for that are secp256r1, NIST P-256 and prime256v1. That weird naming means elliptic curves, aka. named curves.
For those into math, the actual arithmetic equation of secp256r1 -named curve can be viewed in an open-source book by Svetlin Nakov, PhD at https://cryptobook.nakov.com/. All the source code in this freely available book are at https://github.com/nakov/practical-cryptography-for-developers-book. The mathemathical theory how WebAuthN signs the messages is described in detail at https://cryptobook.nakov.com/digital-signatures/ecdsa-sign-verify-messages.
Back to those "pub"-bytes. Reading RFC5480 indicates out of those 65 bytes, the first one, valued 04, indicates this data being for an uncompressed key. With that information, we know rest of the bytes are the actual key values. What remains is a simple act of splitting the remaining 64 bytes into X and Y, resulting two 32-byte integers in hex:
X: 45b6e27d8f9ebb19eccdc322b3df42b271fcd4185ddc4106f41d8e87c5698193
Y: dd165175390f0511b7ee4b16db00afa5d2650e6fba4908e0c3503a99f652eb33
A simple conversion with bc will result in decimal:
X: 31532715897827710605755558209082448985317854901772299252353894644783958819219
Y: 100000572374103825791155746008338130915128983826116118509861921470022744730419
Yes, that's 77 and 78 decimal numbers in them. Feel free to go after the prime number with that public information!
Finally
The mantra is: No passwords.
With WebAuthN, you'll get hugely improved security with multiple authentication factors built into it. What we need is this to spread and go into popular use!
WebAuthN - Passwordless registration and authentication of users
Monday, September 13. 2021
WebAuthN, most of you have never heard of it but can easily understand it has something to do with authenticating "N" in The Web. Well, it is a standard proposed by World Wide Web Consortium or 
. In spec available at https://www.w3.org/TR/webauthn/, they describe WebAuthN as follows:
Web Authentication:
An API for accessing Public Key Credentials
Level 2
W3C Recommendation, 8 April 2021
What motivates engineers into inventing an entirely new form of authenticating end users to web services is obvious. Passwords are generally a bad idea as humans are not very good using memorized secrets in multiple contexts. Your password is supposed to be a secret, but when it leaks from a website and is made public for The World to see your secret isn't a secret anymore. Using a different password for every possible website isn't practical without proper tooling (which many of you won't utilize for number of reasons). As all this isn't new and I've already covered it numerous posts, example Passwords - Part 3 of 2 - Follow up, this time I'm not going there.
Instead, let's take a brief primer on how authentication works. To lay some background for WebAuthN, I'm introducing three generations of authentication. Honestly it is weird how I cannot find anybody's previous work on how to define 1st gen or 2nd gen auth mechanisms. As usual, if you want to have something done properly, you'll have to do it by yourself. I'm defining the generations of authentication techniques here.
First Generation: Password - a memorized secret
This is your basic password-scenario, something most of us know and understand easily. In the picture below is something I drew to illustrate the flow of events during user authentication. On the left is you, the user. You want to login to a website you've registered before. As anybody having access to The Internet, anywhere in the globe, they can show up and claim to be you, the mechanism to identify you being YOU is with the secret you chose during registration process, aka. your password.
Process goes as follows: You open a web browser, go to a known address of the website, when authentication is requested, using a keyboard (touch screen or a real keyboard) enter your password credential to complete the claim of you being you. Finally as your password matches what the website has in its records, you are granted access and go happily forward.
The obvious weakness in this very common authentication mechanism has already been explained. What if somebody gets your password? They saw you typing it, or found the yellow post-it sticker next to your monitor, or ... many possibilities of obtaining your secret, including a guess. To mitigate this, there are MFA of 2-FA to introduce a Multi-Factor Authentication (or Two-Factor), but for the sake of introducing this generation, we skip that. It's an add-on and doesn't really alter this most commonly used process.
It's the simplicity that made this original one the most popular method of authentication. Nothing special really needed and this approach kinda works. It won't resist much of an attack, but it works.
Second Generation: Public Key Infrastructure (PKI)
One of the most prominent attempts to stop using passwords is PKI. As this mechanism isn't widely used in The Web, note the difference for the 1st gen diagram, this is not web, this is Secure Shell, or SSH.
In reality, you can use SSH with passwords also, but it is typical for not to. Actually, I always (no exceptions!) disable password authentication in my servers by introducing PasswordAuthentication no into sshd_config. Simply by not using passwords, my security improves a lot.
Second reality check: In HTTP/1.1 you can use certificate (aka. PKI) authentication. However, on currently used HTTP/2 that is not possible. Quote: "[RFC7540] prohibits renegotiation after any application data has been sent. This completely blocks reactive certificate authentication in HTTP/2". So, that's effectively gone in the form used previously. And no, to those shouting "my national ID-card has an ISO-7816 -chip in it and I can use the certificate in it to authenticate!". That tech isn't based on any of the web standards. All nations invented their own stack of custom software attaching a running process in your computer to your browser's Security Device API. As this isn't even commonly used, this changes nothing and I'll stand behind my claim: In web there really isn't PKI-auth in use.
SSH authentication process goes as follows: On your computer (or mobile device), you open your SSH-client to enter a known server. Authentication is integral part of a connection and cannot really be bypassed. On authentication, you will get the server's identity as a response and your client will verify the address+signature -pair to a previously stored value to make sure the server is in fact The Server you have accessed before. Next your private key will be needed. You have generated this during "registration" process (in SSH there really isn't registration, but declaring the keys to be used can be considered one). Best practice is to have your private keys encrypted, so to access the key I need to enter a password (pass phrase) for decryption. Note the difference for the previous gen. here: the password I type, nor my private key does not leave my computer, only a data packet signed with my private key will. Given PKI-math, the server can verify my signature using the public key on that end. As signature matches, server has verified the claim of me being me and allows access.
In general, this is very nice and safe approach. The introduction of server identity increases your security as a random party cannot claim to act as "The Server". Also the authentication mechanism used is a seriously thought/engineered and complex enough to withstand any brute-forcing or guesswork. Even if your private key would be encrypted with a super-easy passphrase, or not encrypted at all, it won't be easy for somebody else to mimic you. They really really really need to have your private key.
There is no real obvious weakness in this (besides this is not Web). What could be described as a "weakness" is the private key storage. What if it leaks? Your keys to "The Kingdom" are gone. Remember: the private keys should be encrypted to mitigate them being lost. However, your keys being lost/stolen is a lot less likely incident to happen in comparison of somebody guessing or harvesting your password from the server database. Even if you set your mind into generating a "poor" public/private key pair, in fact it is actually very secure. Much more secure than your default "123456" easily guessable password.
On the other hand, if your public key on the server side leaks, that's not a big deal. Servers do get hacked. Data does get stolen, but public is public. You can climb to a mountain and shout your public key for all to know (actually that's literally what GPG does). The math PKI uses in either RSA or ECDSA is considered really really very safe.
Third Generation: WebAuthN
Back to web with PKI. Building on top of first and second generations, there is an improvement: in WebAuthN the private key storage isn't a file in your computer. It's a FIDO2-compliant detachable hardware device. Note: in FIDO Alliance specs, the authenticator "device" can be a software running in your computer (or mobile phone, or whatever you're using as your personal computer). However, most typically deployed WebAuthN-solutions do use actual physical devices you can plug in and out of your computer. The authenticator devices don't have any batteries, are powered from the computer and are typically small enough to be stored in your pocket.
The authentication process goes as follows: You open a web browser, go to a known address of the website. On the website you choose to login and when authentication is requested by the server, it will send you its identification and a challenge as a response to your browser. The callenge will contain data signed by your public key. Obviously your public key was stored to the server during user registration process. As your browser will require help from the authenticator device, a connection will be made there. If your authenticator is protected by a PIN, a request to unlock is indicated by your browser. Using your keyboard, you punch in the PIN-code and your browser will pass on the challenge to the authenticator device. It will check the server's identification and see if it would have a private key in its storage to match that. If it does, it will signal your browser about the found private key. You choose to go forward with authentication on your browser and authenticator device will return the challenge with a signature added into it. Then your browser will deliver signed data over the Net, server on the other side will verify the signature and on successful match you're in! This entire process is a variation of 2nd gen PKI-authentication.
Drawback is obvious: you need to have the device at your hand to be able to log in. Most of regular users like you don't have a compatible FIDO2 authenticator at hand. If you do, on a failing scenario, what if you left the device at home? Or your dog ate it? Or you made an unfortuante mistake and dropped it into a toilet by accident. Well, good thing you thought of this and have configured a spare authenticator, right? Ok, you being a carefree fellow, you forgot where you placed it. At this point you are eventually locked out of the remote system, which is bad.
What's good is the fact that your private key cannot be stolen. Nobody, including you, don't know it nor can access it. All you can do with a device is to get a signed challenge (besides the obvious act of adding a new key during registration). Your security is standing on two factors, the private key stored into an unbreakable device and the fact that a PIN-code is needed on an attempt to access any of the data. That is pretty safe. Worst-case scenario is when your authenticator doesn't have PIN-code protection and somebody gets their hands on it. Then they can present themselves as you.
As WebAuthN is a new and good thing, the only reason it isn't widely used at this point is the need for a specific type of hardware device. These devices are commonly available and they aren't too expensive either. Single device can be used to authenticate into multiple sites, even. The avalability problem is simply because aren't too many manufacturers in existence yet.
Not to worry! This can be navigated around. Let's put on our future-goggles and look a bit ahead. What if "the device" needed here would be your mobile phone? Those things are commonly available and won't require anything else than operating system support and suitable software.
Practical WebAuthN
Enough theory! How does this work in real life?
Well, that's a topic for an another blog post. I'll investigate the USB-A/USB-C/Apple Lightning -solution in closer detail. Also the upcoming mobile device option will be discussed.
WebAuthN, most of you have never heard of it but can easily understand it has something to do with authenticating "N" in The Web. Well, it is a standard proposed by World Wide Web Consortium or . In spec available at https://www.w3.org/TR/webauthn/, they describe WebAuthN as follows:
Web Authentication:
An API for accessing Public Key Credentials
Level 2W3C Recommendation, 8 April 2021
What motivates engineers into inventing an entirely new form of authenticating end users to web services is obvious. Passwords are generally a bad idea as humans are not very good using memorized secrets in multiple contexts. Your password is supposed to be a secret, but when it leaks from a website and is made public for The World to see your secret isn't a secret anymore. Using a different password for every possible website isn't practical without proper tooling (which many of you won't utilize for number of reasons). As all this isn't new and I've already covered it numerous posts, example Passwords - Part 3 of 2 - Follow up, this time I'm not going there.
Instead, let's take a brief primer on how authentication works. To lay some background for WebAuthN, I'm introducing three generations of authentication. Honestly it is weird how I cannot find anybody's previous work on how to define 1st gen or 2nd gen auth mechanisms. As usual, if you want to have something done properly, you'll have to do it by yourself. I'm defining the generations of authentication techniques here.
First Generation: Password - a memorized secret
This is your basic password-scenario, something most of us know and understand easily. In the picture below is something I drew to illustrate the flow of events during user authentication. On the left is you, the user. You want to login to a website you've registered before. As anybody having access to The Internet, anywhere in the globe, they can show up and claim to be you, the mechanism to identify you being YOU is with the secret you chose during registration process, aka. your password.
Process goes as follows: You open a web browser, go to a known address of the website, when authentication is requested, using a keyboard (touch screen or a real keyboard) enter your password credential to complete the claim of you being you. Finally as your password matches what the website has in its records, you are granted access and go happily forward.
The obvious weakness in this very common authentication mechanism has already been explained. What if somebody gets your password? They saw you typing it, or found the yellow post-it sticker next to your monitor, or ... many possibilities of obtaining your secret, including a guess. To mitigate this, there are MFA of 2-FA to introduce a Multi-Factor Authentication (or Two-Factor), but for the sake of introducing this generation, we skip that. It's an add-on and doesn't really alter this most commonly used process.
It's the simplicity that made this original one the most popular method of authentication. Nothing special really needed and this approach kinda works. It won't resist much of an attack, but it works.
Second Generation: Public Key Infrastructure (PKI)
One of the most prominent attempts to stop using passwords is PKI. As this mechanism isn't widely used in The Web, note the difference for the 1st gen diagram, this is not web, this is Secure Shell, or SSH.
In reality, you can use SSH with passwords also, but it is typical for not to. Actually, I always (no exceptions!) disable password authentication in my servers by introducing PasswordAuthentication no into sshd_config. Simply by not using passwords, my security improves a lot.
Second reality check: In HTTP/1.1 you can use certificate (aka. PKI) authentication. However, on currently used HTTP/2 that is not possible. Quote: "[RFC7540] prohibits renegotiation after any application data has been sent. This completely blocks reactive certificate authentication in HTTP/2". So, that's effectively gone in the form used previously. And no, to those shouting "my national ID-card has an ISO-7816 -chip in it and I can use the certificate in it to authenticate!". That tech isn't based on any of the web standards. All nations invented their own stack of custom software attaching a running process in your computer to your browser's Security Device API. As this isn't even commonly used, this changes nothing and I'll stand behind my claim: In web there really isn't PKI-auth in use.
SSH authentication process goes as follows: On your computer (or mobile device), you open your SSH-client to enter a known server. Authentication is integral part of a connection and cannot really be bypassed. On authentication, you will get the server's identity as a response and your client will verify the address+signature -pair to a previously stored value to make sure the server is in fact The Server you have accessed before. Next your private key will be needed. You have generated this during "registration" process (in SSH there really isn't registration, but declaring the keys to be used can be considered one). Best practice is to have your private keys encrypted, so to access the key I need to enter a password (pass phrase) for decryption. Note the difference for the previous gen. here: the password I type, nor my private key does not leave my computer, only a data packet signed with my private key will. Given PKI-math, the server can verify my signature using the public key on that end. As signature matches, server has verified the claim of me being me and allows access.
In general, this is very nice and safe approach. The introduction of server identity increases your security as a random party cannot claim to act as "The Server". Also the authentication mechanism used is a seriously thought/engineered and complex enough to withstand any brute-forcing or guesswork. Even if your private key would be encrypted with a super-easy passphrase, or not encrypted at all, it won't be easy for somebody else to mimic you. They really really really need to have your private key.
There is no real obvious weakness in this (besides this is not Web). What could be described as a "weakness" is the private key storage. What if it leaks? Your keys to "The Kingdom" are gone. Remember: the private keys should be encrypted to mitigate them being lost. However, your keys being lost/stolen is a lot less likely incident to happen in comparison of somebody guessing or harvesting your password from the server database. Even if you set your mind into generating a "poor" public/private key pair, in fact it is actually very secure. Much more secure than your default "123456" easily guessable password.
On the other hand, if your public key on the server side leaks, that's not a big deal. Servers do get hacked. Data does get stolen, but public is public. You can climb to a mountain and shout your public key for all to know (actually that's literally what GPG does). The math PKI uses in either RSA or ECDSA is considered really really very safe.
Third Generation: WebAuthN
Back to web with PKI. Building on top of first and second generations, there is an improvement: in WebAuthN the private key storage isn't a file in your computer. It's a FIDO2-compliant detachable hardware device. Note: in FIDO Alliance specs, the authenticator "device" can be a software running in your computer (or mobile phone, or whatever you're using as your personal computer). However, most typically deployed WebAuthN-solutions do use actual physical devices you can plug in and out of your computer. The authenticator devices don't have any batteries, are powered from the computer and are typically small enough to be stored in your pocket.
The authentication process goes as follows: You open a web browser, go to a known address of the website. On the website you choose to login and when authentication is requested by the server, it will send you its identification and a challenge as a response to your browser. The callenge will contain data signed by your public key. Obviously your public key was stored to the server during user registration process. As your browser will require help from the authenticator device, a connection will be made there. If your authenticator is protected by a PIN, a request to unlock is indicated by your browser. Using your keyboard, you punch in the PIN-code and your browser will pass on the challenge to the authenticator device. It will check the server's identification and see if it would have a private key in its storage to match that. If it does, it will signal your browser about the found private key. You choose to go forward with authentication on your browser and authenticator device will return the challenge with a signature added into it. Then your browser will deliver signed data over the Net, server on the other side will verify the signature and on successful match you're in! This entire process is a variation of 2nd gen PKI-authentication.
Drawback is obvious: you need to have the device at your hand to be able to log in. Most of regular users like you don't have a compatible FIDO2 authenticator at hand. If you do, on a failing scenario, what if you left the device at home? Or your dog ate it? Or you made an unfortuante mistake and dropped it into a toilet by accident. Well, good thing you thought of this and have configured a spare authenticator, right? Ok, you being a carefree fellow, you forgot where you placed it. At this point you are eventually locked out of the remote system, which is bad.
What's good is the fact that your private key cannot be stolen. Nobody, including you, don't know it nor can access it. All you can do with a device is to get a signed challenge (besides the obvious act of adding a new key during registration). Your security is standing on two factors, the private key stored into an unbreakable device and the fact that a PIN-code is needed on an attempt to access any of the data. That is pretty safe. Worst-case scenario is when your authenticator doesn't have PIN-code protection and somebody gets their hands on it. Then they can present themselves as you.
As WebAuthN is a new and good thing, the only reason it isn't widely used at this point is the need for a specific type of hardware device. These devices are commonly available and they aren't too expensive either. Single device can be used to authenticate into multiple sites, even. The avalability problem is simply because aren't too many manufacturers in existence yet.
Not to worry! This can be navigated around. Let's put on our future-goggles and look a bit ahead. What if "the device" needed here would be your mobile phone? Those things are commonly available and won't require anything else than operating system support and suitable software.
Practical WebAuthN
Enough theory! How does this work in real life?
Well, that's a topic for an another blog post. I'll investigate the USB-A/USB-C/Apple Lightning -solution in closer detail. Also the upcoming mobile device option will be discussed.
macOS Big Sur 11.1 update fail
Monday, August 30. 2021
Big Sur was riddled with problems from getgo. It bricked some models, started installing when there was not enough disc space available and all kinds of weirness. Quite few of the minor versions had issues.
I was stuck with version 11.0.1 for over half a year. Of course my mac was hit by one of the problems, I tried navigating around the problem, contacted Apple Care (yay! finally got something ouf of that extra money) and they said a full install will be required. When doing the full install didn't work I lost motivation and just ignored any updates for months.
Going for the typical path, macOS of receving a notification about update or via App Store, they are pretty much the same thing, difference being if I actively seeked for updates or was reminded about it. Result:
Ok. Doesn't seem to work. I triied downloading on Wi-Fi, on Ethernet with couple of adapters, then I gave up and went for my favorite way of doing updates, USB-stick. Preparing the stick, booting from it, quite soon I was informed about "A software update is required to use this startup disk".
Clicking Update resulted an instant error:
Preparing something to do something didn't go as planned. Error message of "An error occurred preparing the software update" was emitted. Trying again wouldn't help. Booting into Internet recovery wasn't helpful either, error -2003F indicating failure to access resources over Internet. All avenues were explored, everything I knew was done. I totally lost motivation to attempt anything. All that time the Apple Care chat message "just do a clean reinstall" demotivated me more. I WAS TRYING TO DO A CLEAN REINSTALL!! It just didn't work.
I wasn't alone. Many many many users had exactly the same symptoms.
Months passed.
Macs are funny that way. Apple does "support" their old macOSes for security, but old hardware has a cap on what macOS will be the the last one for that particular hardware. At the same time you absolutely positively need to fall forward into latest macOS versions to have support for all nice software. My Mac wasn't capped by obsoleted hardware, it was capped by Apple's own QA failing and allowing a deploy on an OS version that wouldn't work properly. In this case, I needed to update Xcode, the must-have Mac / iOS developer tool. The version I had runing worked, but didn't support the new and shiny things I needed. I wasn't allowed to upgrade as my Big Sur version was too old.
You Maniacs! You blew it up! Ah, damn you! God damn you all to hell!
(For movie-ignorant readers, that quote is from -68 Planet of the Apes)
Need for a recent Xcode sparked my motivation. I went back to the grinding stone and trying to force my thing to update. I re-did everything I had already done before. No avail. Nothing works. Every single thing I attempted resulted in a miserable failure. To expand my understanding about the problem I read everything I could find about the subject and got hints what to try next. Apparently the problem was with the T2-chip.
For the umphteenth time I did a recovery boot and realized something I didn't recall anybody else mentioning:
Among numerous options to install or upgrade macOS I knew about, there is one additional approach in the recovery menu. And IT WORKED!
To possibly help out others still suffering from this, I left my mark into StackExchange https://apple.stackexchange.com/a/426520/251842.
Now I'm banging my head to a brick wall. Why didn't I realize that sooner!
Anyway, the Mac is updated, newer software is running. Until similar thing happens for the next time, that is.
Big Sur was riddled with problems from getgo. It bricked some models, started installing when there was not enough disc space available and all kinds of weirness. Quite few of the minor versions had issues.
I was stuck with version 11.0.1 for over half a year. Of course my mac was hit by one of the problems, I tried navigating around the problem, contacted Apple Care (yay! finally got something ouf of that extra money) and they said a full install will be required. When doing the full install didn't work I lost motivation and just ignored any updates for months.
Going for the typical path, macOS of receving a notification about update or via App Store, they are pretty much the same thing, difference being if I actively seeked for updates or was reminded about it. Result:
Ok. Doesn't seem to work. I triied downloading on Wi-Fi, on Ethernet with couple of adapters, then I gave up and went for my favorite way of doing updates, USB-stick. Preparing the stick, booting from it, quite soon I was informed about "A software update is required to use this startup disk".
Clicking Update resulted an instant error:
Preparing something to do something didn't go as planned. Error message of "An error occurred preparing the software update" was emitted. Trying again wouldn't help. Booting into Internet recovery wasn't helpful either, error -2003F indicating failure to access resources over Internet. All avenues were explored, everything I knew was done. I totally lost motivation to attempt anything. All that time the Apple Care chat message "just do a clean reinstall" demotivated me more. I WAS TRYING TO DO A CLEAN REINSTALL!! It just didn't work.
I wasn't alone. Many many many users had exactly the same symptoms.
Months passed.
Macs are funny that way. Apple does "support" their old macOSes for security, but old hardware has a cap on what macOS will be the the last one for that particular hardware. At the same time you absolutely positively need to fall forward into latest macOS versions to have support for all nice software. My Mac wasn't capped by obsoleted hardware, it was capped by Apple's own QA failing and allowing a deploy on an OS version that wouldn't work properly. In this case, I needed to update Xcode, the must-have Mac / iOS developer tool. The version I had runing worked, but didn't support the new and shiny things I needed. I wasn't allowed to upgrade as my Big Sur version was too old.
You Maniacs! You blew it up! Ah, damn you! God damn you all to hell!
(For movie-ignorant readers, that quote is from -68 Planet of the Apes)
Need for a recent Xcode sparked my motivation. I went back to the grinding stone and trying to force my thing to update. I re-did everything I had already done before. No avail. Nothing works. Every single thing I attempted resulted in a miserable failure. To expand my understanding about the problem I read everything I could find about the subject and got hints what to try next. Apparently the problem was with the T2-chip.
For the umphteenth time I did a recovery boot and realized something I didn't recall anybody else mentioning:
Among numerous options to install or upgrade macOS I knew about, there is one additional approach in the recovery menu. And IT WORKED!
To possibly help out others still suffering from this, I left my mark into StackExchange https://apple.stackexchange.com/a/426520/251842.
Now I'm banging my head to a brick wall. Why didn't I realize that sooner!
Anyway, the Mac is updated, newer software is running. Until similar thing happens for the next time, that is.
Nokia 5.3 got Android 11 upgrade
Monday, August 16. 2021
There it is:
This update was promised on 8th September 2020, roughly an year ago. See: https://twitter.com/nokiamobile/status/1303384763705819157
As promised, there it is. And for iPhone user like me, taking a screenshot and sending it into processing via Gmail is nearly impossible. These same Nokia-guys used to make phones where every trivial thing required 5-8 button presses. Even if every generation of Android takes out my user experience, I still appreciate the fact they maintain and provide updates to these no-mandatory-Facebook -phones.
Keep it up!
There it is:
This update was promised on 8th September 2020, roughly an year ago. See: https://twitter.com/nokiamobile/status/1303384763705819157
As promised, there it is. And for iPhone user like me, taking a screenshot and sending it into processing via Gmail is nearly impossible. These same Nokia-guys used to make phones where every trivial thing required 5-8 button presses. Even if every generation of Android takes out my user experience, I still appreciate the fact they maintain and provide updates to these no-mandatory-Facebook -phones.
Keep it up!
Wi-Fi 6 - Part 2 of 2: Practical wireless LAN with Linksys E8450
Sunday, August 15. 2021
There is a previous post in this series about wireless technology.
Wi-Fi 6 hardware is available, but uncommon. Since its introduction three years ago, finally it is gaining popularity. A practial example of sometimes-difficult-to-obtain part is an USB-dongle. Those have existed at least 15 years now. There simply is none with Wi-Fi 6 capability.
Additional twist is thrown at me, a person living in EU-reagion. For some weird (to me) reason, manufacturers aren't getting their radio transmitters licensed in EU. Only in US/UK. This makes Wi-Fi 6 appliance even less common here.
When I throw in my absolute non-negotiable requirement of running a reasonable firmware in my access point, I'll limit my options to almost nil. Almost! I found this in OpenWRT Table-of-Hardware: Linksys E8450 (aka. Belkin RT3200) It is an early build considered as beta, but hey! All of my requirements align there, so I went for it in Amazon UK:
Wi-Fi 6 Access Point: Belkin RT3200
Couple of days waiting for UPS delivery, and here goes:
This is exactly what I wanted and needed! A four-port gigabit switch for wired LAN, incoming Internet gigabit connector. 12 VDC / 2 A barrel connector for transformer. Given UK power plugs are from 1870s they're widely incompatible with EU-ones. Luckily manufacturers are aware of this and this box contains both UK and EU plugs in an easily interchangeable form. Thanks for that!
Notice how this is a Belkin "manufactured" unit. In reality it is a relabled Linksys RT3200. Even the OpenWRT-firmware is exactly same. Me personally, I don't care what the cardobard box says as long as my Wi-Fi is 6, is fast and is secure.
Illustrated OpenWRT Installation Guide
The thing with moving away from vendor firmware to OpenWRT is that it can be tricky. It's almost never easy, so this procedure is not for everyone.
To achieve this, there are a few steps needed. Actual documentation is at https://openwrt.org/toh/linksys/e8450, but be warned: amount of handholding there is low, for newbie there is not much details. To elaborate the process of installation, I'm walking trough what I did to get me OpenWRT running in the box.
Step 0: Preparation
You will need:
- Linksys/Belkin RT3200 access point
- Wallsocket to power the thing
- A computer with Ethernet port
- Any Windows / Mac / Linux will do, no software needs to be installed, all that is required is a working web browser
- Ethernet cable with RJ-45 connectors to access the access point's admin panel via LAN
- OpenWRT firmware from https://github.com/dangowrt/linksys-e8450-openwrt-installer
- Download files into a laptop you'll be doing your setup from
- Linksys-compatible firmware is at at:https://github.com/dangowrt/linksys-e8450-openwrt-installer/releases, get
openwrt-mediatek-mt7622-linksys_e8450-ubi-initramfs-recovery-installer.itb
- Also download optimized firmware
openwrt-mediatek-mt7622-linksys_e8450-ubi-squashfs-sysupgrade.itb
- Skills and rights to administer your workstation to have its Ethernet port a fixed IPv4-address from net 192.168.1.1/24
- Any other IPv4 address on that net will do, I used 192.168.1.10
- No DNS nor gateway will be needed for this temporary setup
Make sure not to connect the WAN / Internet into anything. The Big Net is scary and don't rush into that yet. You can do that later when all installing and setupping is done.
Mandatory caution:
If you just want to try OpenWrt and still plan to go back to the vendor firmware, use the non-UBI version of the firmware which can be flashed using the vendor's web interface.
Process described here is the UBI-version which does not allow falling back to vendor firmware.
Step 1: Un-box and replace Belkin firmware
After plugging the Access Point to a wall socket, flicking the I/O-switch on, attaching an Ethernet cable to one of the LAN-switch ports and other end directly to a laptop, going to http://192.168.1.1 with your browser will display you something like this:
What you need to do is try to exit the out-of-box-experience setup wizard:
For the "Ethernet cable is not connected" you need to click Exit. When you think of the error message bit harder, if you get the message, your Ethernet IS connected. Ok, ok. It is for the WAN Ethernet, not LAN.
Notice how setup "did not complete succesfully". That is fully intentional. Click "Do not set up". Doing that will land you on a login:
This is your unconfigured admin / admin -scenario. Log into your Linksys ... erhm. Belkin.
Select Configuration / Administration / Firmware Upgrade. Choose File. Out of the two binaries you downloaded while preparing, go for the ubi-initramfs-recovery-installer.itb. That OpenWRT firmware file isn't from manufacturer, but the file is packaged in a way which makes it compatible to allow easy installation:
On "Start Upgrade" there will be a warning. Click "Ok" and wait patiently for couple minutes.
Step 2: Upgrade your OpenWRT recovery into a real OpenWRT
When all the firmware flashing is done, your factory firmware is gone:
There is no password. Just "Login". An OpenWRT welcome screen will be shown:
Now that you're running OpenWRT, your next task is to go from recovery to real thing. I'm not sure if I'll ever want to go back, but as recommended by OpenWRT instructions, I did take backups of all four mtdblocks: bl2, fip, factory and ubi. This step is optinal:
When you're ready, go for the firmware upgrade. This time select openwrt-mediatek-mt7622-linksys_e8450-ubi-squashfs-sysupgrade.itb:
To repeat the UBI / non-UBI firmware: This is the UBI-version. It is recommended as it has better optimization for layout and management of SPI flash, but it does not allow fallbacking to vendor firmware.
I unchecked the "Keep settings and retain the current configuration" to make sure I got a fresh start with OpenWRT. On "Continue", yet another round of waiting will occur:
Step 3: Setup your wireless AP
Déjà vu!
You have seen this exact screen before. Login (there is no password yet):
Second time, same screen but with this time there is a proper firmware in the AP. Go set the admin account properly to get rid of the "There is no password set on this router" -nag. Among all settings, go to wireless configuration to verify both 2.4 and 5 GHz radios are off:
Go fix that. Select "Edit" for the 5 GHz radio and you'll be greeted by a regular wireless access point configuration dialog. It will include section about wireless security:
As I wanted to improve my WLAN security, I steer away from WPA2 and went for a WPA3-SAE security. Supporting both at the same time is possible, but securitywise it isn't wise. If your system allows wireless clients to associate with a weaker solution, they will.
Also for security, check KRACK attack countermeasures. For more details on KRACK, see: https://www.krackattacks.com/
When you've done, you should see radio enabled on a dialog like this:
Step 4: Done! Test.
That's it! Now you're running a proper firmware on our precious Wi-Fi 6 AP. But how fast it is?
As I said, I don't have many Wi-Fi 6 clients to test with. On my 1 gig fiber, iPad seems to be pretty fast. Also my Android phone speed is ... well ... acceptable.
For that speed test I didn't even go for the "one foot distance" which manufacturers love to do. As nobody uses their mobile devices right next to their AP, I tested this on a real life -scenario where both AP and I were located the way I would use Internet in my living room.
Final words
After three year wait Wi-Fi 6 is here! Improved security, improved speed, improved everything!
There is a previous post in this series about wireless technology.
Wi-Fi 6 hardware is available, but uncommon. Since its introduction three years ago, finally it is gaining popularity. A practial example of sometimes-difficult-to-obtain part is an USB-dongle. Those have existed at least 15 years now. There simply is none with Wi-Fi 6 capability.
Additional twist is thrown at me, a person living in EU-reagion. For some weird (to me) reason, manufacturers aren't getting their radio transmitters licensed in EU. Only in US/UK. This makes Wi-Fi 6 appliance even less common here.
When I throw in my absolute non-negotiable requirement of running a reasonable firmware in my access point, I'll limit my options to almost nil. Almost! I found this in OpenWRT Table-of-Hardware: Linksys E8450 (aka. Belkin RT3200) It is an early build considered as beta, but hey! All of my requirements align there, so I went for it in Amazon UK:
Wi-Fi 6 Access Point: Belkin RT3200
Couple of days waiting for UPS delivery, and here goes:
This is exactly what I wanted and needed! A four-port gigabit switch for wired LAN, incoming Internet gigabit connector. 12 VDC / 2 A barrel connector for transformer. Given UK power plugs are from 1870s they're widely incompatible with EU-ones. Luckily manufacturers are aware of this and this box contains both UK and EU plugs in an easily interchangeable form. Thanks for that!
Notice how this is a Belkin "manufactured" unit. In reality it is a relabled Linksys RT3200. Even the OpenWRT-firmware is exactly same. Me personally, I don't care what the cardobard box says as long as my Wi-Fi is 6, is fast and is secure.
Illustrated OpenWRT Installation Guide
The thing with moving away from vendor firmware to OpenWRT is that it can be tricky. It's almost never easy, so this procedure is not for everyone.
To achieve this, there are a few steps needed. Actual documentation is at https://openwrt.org/toh/linksys/e8450, but be warned: amount of handholding there is low, for newbie there is not much details. To elaborate the process of installation, I'm walking trough what I did to get me OpenWRT running in the box.
Step 0: Preparation
You will need:
- Linksys/Belkin RT3200 access point
- Wallsocket to power the thing
- A computer with Ethernet port
- Any Windows / Mac / Linux will do, no software needs to be installed, all that is required is a working web browser
- Ethernet cable with RJ-45 connectors to access the access point's admin panel via LAN
- OpenWRT firmware from https://github.com/dangowrt/linksys-e8450-openwrt-installer
- Download files into a laptop you'll be doing your setup from
- Linksys-compatible firmware is at at:https://github.com/dangowrt/linksys-e8450-openwrt-installer/releases, get
openwrt-mediatek-mt7622-linksys_e8450-ubi-initramfs-recovery-installer.itb - Also download optimized firmware
openwrt-mediatek-mt7622-linksys_e8450-ubi-squashfs-sysupgrade.itb
- Skills and rights to administer your workstation to have its Ethernet port a fixed IPv4-address from net 192.168.1.1/24
- Any other IPv4 address on that net will do, I used 192.168.1.10
- No DNS nor gateway will be needed for this temporary setup
Make sure not to connect the WAN / Internet into anything. The Big Net is scary and don't rush into that yet. You can do that later when all installing and setupping is done.
Mandatory caution:
If you just want to try OpenWrt and still plan to go back to the vendor firmware, use the non-UBI version of the firmware which can be flashed using the vendor's web interface.
Process described here is the UBI-version which does not allow falling back to vendor firmware.
Step 1: Un-box and replace Belkin firmware
After plugging the Access Point to a wall socket, flicking the I/O-switch on, attaching an Ethernet cable to one of the LAN-switch ports and other end directly to a laptop, going to http://192.168.1.1 with your browser will display you something like this:
What you need to do is try to exit the out-of-box-experience setup wizard:
For the "Ethernet cable is not connected" you need to click Exit. When you think of the error message bit harder, if you get the message, your Ethernet IS connected. Ok, ok. It is for the WAN Ethernet, not LAN.
Notice how setup "did not complete succesfully". That is fully intentional. Click "Do not set up". Doing that will land you on a login:
This is your unconfigured admin / admin -scenario. Log into your Linksys ... erhm. Belkin.
Select Configuration / Administration / Firmware Upgrade. Choose File. Out of the two binaries you downloaded while preparing, go for the ubi-initramfs-recovery-installer.itb. That OpenWRT firmware file isn't from manufacturer, but the file is packaged in a way which makes it compatible to allow easy installation:
On "Start Upgrade" there will be a warning. Click "Ok" and wait patiently for couple minutes.
Step 2: Upgrade your OpenWRT recovery into a real OpenWRT
When all the firmware flashing is done, your factory firmware is gone:
There is no password. Just "Login". An OpenWRT welcome screen will be shown:
Now that you're running OpenWRT, your next task is to go from recovery to real thing. I'm not sure if I'll ever want to go back, but as recommended by OpenWRT instructions, I did take backups of all four mtdblocks: bl2, fip, factory and ubi. This step is optinal:
When you're ready, go for the firmware upgrade. This time select openwrt-mediatek-mt7622-linksys_e8450-ubi-squashfs-sysupgrade.itb:
To repeat the UBI / non-UBI firmware: This is the UBI-version. It is recommended as it has better optimization for layout and management of SPI flash, but it does not allow fallbacking to vendor firmware.
I unchecked the "Keep settings and retain the current configuration" to make sure I got a fresh start with OpenWRT. On "Continue", yet another round of waiting will occur:
Step 3: Setup your wireless AP
Déjà vu!
You have seen this exact screen before. Login (there is no password yet):
Second time, same screen but with this time there is a proper firmware in the AP. Go set the admin account properly to get rid of the "There is no password set on this router" -nag. Among all settings, go to wireless configuration to verify both 2.4 and 5 GHz radios are off:
Go fix that. Select "Edit" for the 5 GHz radio and you'll be greeted by a regular wireless access point configuration dialog. It will include section about wireless security:
As I wanted to improve my WLAN security, I steer away from WPA2 and went for a WPA3-SAE security. Supporting both at the same time is possible, but securitywise it isn't wise. If your system allows wireless clients to associate with a weaker solution, they will.
Also for security, check KRACK attack countermeasures. For more details on KRACK, see: https://www.krackattacks.com/
When you've done, you should see radio enabled on a dialog like this:
Step 4: Done! Test.
That's it! Now you're running a proper firmware on our precious Wi-Fi 6 AP. But how fast it is?
As I said, I don't have many Wi-Fi 6 clients to test with. On my 1 gig fiber, iPad seems to be pretty fast. Also my Android phone speed is ... well ... acceptable.
For that speed test I didn't even go for the "one foot distance" which manufacturers love to do. As nobody uses their mobile devices right next to their AP, I tested this on a real life -scenario where both AP and I were located the way I would use Internet in my living room.
Final words
After three year wait Wi-Fi 6 is here! Improved security, improved speed, improved everything!
