Hacker's ramblings

There was bit of a duel (or a full-scale war) between Spamhaus and Cyberbunker. Spamhaus has published some information about their side here. All this dates back to 2011 when they first collided. The latest skirmish started with Spamhaus on the losing side, as allegedly Cyberbunker had some help with a botnet creating tons of DNS-requests to Spamnaus' servers. Cyberbunker naturally denies all this, but in reality they are simply coward liars! Who else would have the motivation to knock Spamhaus off the net. No need to think that any longer.

Since the enemy had some help, what happened next was Spamhaus joining forces with Cloudflare, a company specializing in mitigating the effects of a DDoS-attack. What happened at the end of March 2013 has been described as "The DDoS That Almost Broke the Internet" by Cloudflare blog.

The spam-blocking service Spamhaus is providing technically works on top of DNS. Anybody running a receiving mail-server can configure it to confirm the connecting client's IP-address with a simple DNS-query returning funny-but-pre-determined names as an answer to determine the "spamminess" level of connecting client. The judgement who is a spammer and who is not is made solely by the Spamhaus. That's what the dispute between them and Cyberbunker is all about.

As described by Cloudflare, technically Cyberbunker's (alleged) DDoS works by amplifying incoming 36 UDP-bytes containing a valid query for RIPE.net's zone into 100-fold. There are at least 30.000 open DNS-servers responding to recursive queries. All they have to do is spoof the original UDP-packet's sender's IP into Spamhaus and they have harnessed a huge Internet traffic amplification machine targeting a single IP-address.

Since I myself am running a couple of DNS-boxes, I wanted to re-verify my servers, that they cannot be used into such activity. I googled some and found The Measurement Factory's Open resolver test. That appears to be a piece of crap. You punch in an IP-address and get open/closed status as a response. You can enter any invented IP-address to get the closed-verdict. WTF?!

The second thing I found is much convincing: Open DNS Resolver Project. The problem with that one is, that they just browse The Net and try to find open DNS-servers. For example my boxes were not listed. Not as open, closed or existing. They don't publish information about properly configured DNS-servers. It still leaves the original question unanswered: Can my DNS-server be used for attacking innocent or not.

Here is my answer to the problem: http://opensource.hqcodeshop.com/DNStest/dnstest-cgi.pl

It caches the result of any query for 24 hours, and cannot be used for bullying somebody. That feature I simply stole copied from The Measurement Factory. Its fully written in Perl and even the source code is available for you to get.

Initial feedback after putting the thing on-line was to support FQDNs. The answer is NO. My thing won't do any unnecessary DNS-requests, if possible. But if you have any other suggestions, please drop a comment.

Everybody who has ever been to a store has seen those electronic gates which are meant to keep shoplifters from stealing stuff. Normal shoplifters use some kind of countermeasures and are actually not bothered by the gates, only honest people get to suffer from them. Typically the gate triggers the alarm by accident when shop personnel simply forgot to remove the tag, or sometimes a rolled wire of some sorts will resemble a coil so that the gate thinks that my recently bought extension cord and/or Ethernet cable is an anti-theft tag. The other not-so-typical scenario is that, there is a tag attached to thing that you bought, but the gate does NOT trigger the alarm. Well, this time that's what happened.

I'm at home and realize, that there is a tag attached. It looks ugly and annoying and should be removed. Since I've always wanted to know how do they remove them in the shop, I took my trustworthy Dremel and started cutting.

Here are the pics:

It looks that there are 4 lightly magnetic ball bearings inside a small cavity made out of plastic and steel. The steely part of the cave is also magnetic so that it attracts the ball bearings to stay on that side. Then there is the part they remove in the store, it is a metallic stud which really doesn't move a lot when pulled. If a lot of force is applied to the stud, the four ball bearings are tightening to the direction of the pull, so that is it impossible (or very very hard) to actually succeed in removing the stud from the tag. In the store (you see them next to the cash register) they have a powerful magnet which is applied to the plastic side of the tag. When the tag is placed on top of the magnet, it pulls the ball bearings down (with the help of gravity), making the stud move away. A removed stud slips back to the tag very easily without magnets or anything, the ball bearings just move out of the way.

The tag is fully covered with plastic and most of it is a coil for the gate. Normal tags don't have sharp edges or brownish dust from dremeling. This one does, since I literally cut it half. The white plastic part in the 2nd pic is typically covered with the black plastic. Also the stud in the pics 3 and 4 is bit longer, since it is not cut short with a power tool.

Next question typically is: How to remove them next time without cutting/breaking the tag? My answer is that I don't know. My tag is busted anyway, but next time I have one that is not busted, I'll try applying some sort of magnet and hitting the tag to the direction of the magned. Eventually, it boils down to the magnetic force, so a powerful one is recommended. I don't know if I have one that has enough pull in it, but I'll sure try. Another thing that comes into mind is to keep twisting the stud while pulling, it should make the ball bearings roll and stay loose enough.

Couple of weeks ago openSUSE-project released their latest desktop-Linux. That is the distro The Man himself, Linus Torvalds had a dispute about security policy of needing a root access to add a new wireless network. He actually said that openSUSE-people are morons. A year ago, they were very defensive and insisted that Mr. Man had it wrong. In reality: no other operating system requires demi-god permissions to do such a trivial task. In 12.3 the morons finally got it, connecting to a new wireless LAN does not require any special permissions.

My hardware for running a desktop-Linux is a very old Apple MacBook. The Mac OS X system info says, that this is a 1,1 hardware, making it pretty much one of the first ever Intel Macs there exists. It has two gigs of RAM and enough hard disk to run pretty much any modern disto. Being a Mac, it also has enough Intel chips in it to fulfill any requirements that modern distros have for 2D or 3D graphics, sound or display. It definitely lacks the I/O or CPU power that any not-6-years-old laptop might have, but it is very suitable for running a desktop-Linux. Mr. Torvalds prefers Apple Airs, but I didn't want to spend that much money on an used computer.

openSUSE install just keeps on improving. I always back up the old computer and do a fresh install, I sure haven't met a working operating system upgrade ever. During installation, all the settings are there if you need the, but the defaults are very good making the entire process flow smoothly. This time there was a glitch when the Atheros WLAN-chip was not auto-detected during install. I had to manually go configure network devices and add a wireless device. At that point the ath5k driver was detected and I got the box connected to The Net for the rest of the install. No other special things there.

After install the first thing I got was the pommed-package. It makes the Apple-keys work in Linux and is definitely needed. My keyboard layout is Finnish, so I also had to compile keyfuzz to get rid of those useless Apple-keys which are called Meta-keys in Linux. I need my alts, and do the following mappings:

 # Map Alt to Meta
458978 125
# Map Meta to Alt
458979 56
# Map Right Meta to Right Alt
458983 100

The final thing to do is to get the iSight-camera working. All it requires is the Apple-copyrighted firmware and it is ready to go. What I did, was to restore my previous file from a backup, but if you need to get one for yourself, there is ift-package or iSight Firmware Tools. With that you can extract the needed bits from Mac OS X device driver and place the resulting file into your Linux. There already is a Linux kernel-module isight_firmware waiting for the file to appear. As a result a brand new Video4Linux-device should appear and you can test it with MPlayer (that breaks couple of dozen copyrights and you need to get from The Net):

mplayer tv:// -tv driver=v4l2:width=320:height=240:device=/dev/video0 -fps 30

The 12.3 runs clearly much faster than 12.2. I have all the KDE4-desktop effects enabled and 12.2 really couldn't manage the 3D-graphics. 12.3 seems to be able to get more juice out of the Intel's 945 GPU. With all the modern software and latest Linux kernel the open-source -guys are finally getting there (with support from Novell, of course). This is actually a very usable desktop for a geek like me.

openSUSE 12.3 get's my seal-of-approval with a bonus thumbs up.

This is a really weird one. On 64-bit Windows, a regular USB bar code scanner is not detected as a HID-keyboard. Actually it falls into smthing really weird -category in the USB-devices. And needless to say, but the scanner effectively does not work. Windows simply states that "driver not found" and adds that "device may not function properly". I tried upgrading the driver from Device Manager, but no dice there.

Couple of users are complaining the same thing, but one actually has a solution. The idea is to first connect a real keyboard into USB-port and after that the barcode scanner. WTF?! It actually works! Windows gets fooled enough by the actual keyboard, that barcode scanner works even if the real keyboard is unplugged. It's just that an actual keyboard needs to be present during the driver detection.

I had couple of Zebex scanners and tried to make them work with my Windows 7, but all I got was frustration and no tangible results (beside the keyboard trick). I had a the-cheapest-there-is -model and a proper one, but there was no real difference in how Windows saw them. Based on the reports available in the Net, this is not a single manufacturer issue, its more like a Windows HID-keyboard issue. Then I was doing something else for a while, enough for the laptop power saver to kick in. When I got back to the computer and slapped it up from the sleep, then miraculously Windows detected the already plugged in barcode scanner as a HID-keyboard!! WTF?! #2

Ever since, both scanners have been functioning ok. Also, I'm pretty sure that now my laptop has been "tainted" and I cannot continue my tests with it anymore. I'd probably should re-install entire operating system just to confirm the results. But I'd rather not.

If anybody can explain what happened there, please drop a comment.

MySQL Enterprise Monitor is a really good tool to see what's going on in the DB. At least I'd like to give my DB-box all the possible resources, I'm running the Service Manager -part in another server.

Sometimes there is a need to upgrade servers. This time it moving other roles out of the way was piece-of-cake, except the Service Manager. In the entire Internet, there is no spot-on information about how to do it in detail. The only really relevant information I could find is B.5. Backing up MySQL Enterprise Service Manager in the MySQL documentation. In the doc they manage to describe how to back things up, but not really how to restore anything.

I did the operation in following steps:

1. Fresh installation of Service Manager in the new server
• I chose not to configure anything and ended the installation there
2. Backup of the data as described in the doc:
   mysqldump --single-transaction -uservice_manager -p -P13306 -h127.0.0.1 mem > mem.dump
   
3. Restore of data into the new server using command like:
   /opt/mysql/enterprise/monitor/mysql/bin/mysql -u service_manager -p -P13306 -h 127.0.0.1
   
4. In the DB-server the Enterprise Monitor Agent needs to be reconfigured to send information to a new Service Manager
1. Edit file /opt/mysql/enterprise/agent/etc/mysql-monitor-agent.ini
2. Confirm value of agent-mgmt-hostname
   
3. Confirm value of aggr-mem-baseurl
   
5. After these changes a login to the newly setup Service Manager showed the DB as fully functional
   

Hope this helps somebody.

That happened to me with March 2013 updates. The machine was stuck with "Operations are in progress" -message for 12 hours. At that point I deduced that it is unlikely for the machine to actually be doing anything.

The real question is: What to do?

1. Force the thing into reboot cycle?
2. Wait a while longer?

I chose 1. and was ready for the smelly thing to hit the fan. Nothing happened. Windows finalized the updates during boot, which is pretty much normal in the circumstances. After that I logged in and everything worked fine.

I bumped into couple of issues earlier. Article about missing Integration Services and Networking Status: degraded.

RedHat managed to package the Integration Service drivers into RHEL 6.4 which essentially is the base of CentOS 6.4. So, from now on the much required drivers are bundled in the installation source.

There are no major changes in the drivers, though. Network status is still degraded and a hint of upgrading the drivers is there. It seems to be a mystery to everybody how to do the upgrade.

The list of integration services is unchanged:

• Operating system shutdown
• Time synchronization
• Data Exchange
• Heartbeat
• Backup (volume snapshot)

This article in Microsoft's social network describes the changes. Looks like Dynamic Memory (ballooning) is the only new feature. That wasn't even in the RHEL 6.4 beta, but they pushed it into final release.

Current Linux implementation (Netatalk) of Apple Filing Protocol does support all the good stuff properly. Mainly:

• DHX2 (Diffie-Hellman Key Exchange 2) -based authentication: the old auth was plain-text, this is a huge improvement, all modern OS Xs have this as mandatory
  
• AFP Lock Stealing, + couple of other features: See Apple's requirements for Time Machine server
  

In my case, there is a HFS+ sparse file on an EXT4-partition. Setting this up with Time Machine is a breeze, however TM will "think" a while before mounting the sparse file and actually running the backup. There are a couple of user testimonials that occasionally (too often) TM will freeze and fail to do a backup. Also when the failure occurs, TM will fail to continue operation and a full backup media reset will be required.

The source of my information is QNAP wiki, which describes the process on old(ish) Mac OS X and Netatalk. I got my backup running based on that information. Also, if there are earlier reported failures to backup, I'll report back here.

The above setup sure is a bitch to get running. There is plenty of documentation and tools scattered around the Web, but it looks like the user funnel goes something like this:

• 1.000.000 users running Samba on their Linux
• 1.000 out of the above users are running OpenLDAP on their Linux(es)
• 1 out of the above users are running Samba with passdb backend = ldapsam

In the end, there is no definite document or tool to rely on. I spent about two weeks gathering information and trying out various approaches. I had to run my LDAP-queries unencrypted while sniffing the traffic with Wireshark to get an idea what kind of information is being requested.

In the end, just getting LDAP working on your server(s) is difficult enough for most people to get discouraged by the constant failures while attempting the setup. The advantages are there, being able to authenticate SSH-shell, secured web pages, WebDAV and any number of applications against exactly same user name / password -pair in a tested and secure manner is an execllent reward. Just having HTTP Basic Auth running against existing Linux userbase without LDAP is very difficult and mostly requires poking unnecessary holes to system security to get it running.

One of the really bad things is that not all documentations describes a simple step-by-step -process of making the setup secure. There is no need to allow all access to everybody, for example a simple:

olcAccess: {0}to attrs=userPassword,shadowLastChange
    by self write
    by anonymous auth
    by dn="cn=admin,dc=example,dc=org" write
    by * none

will make harvesting accidentally stored plain text -password much more difficult than the out-of-the-box solution does. All stored LDAP-passwords should be hashes, right? Not all tools default doing that.

When all the Samba-parts are in the LDAP, the above issue raises again. Using LanManager-passwords is out right stupid, they can be brute-forced with ease, but NTLM-passwords need to be protected with similar access-line:

olcAccess: {1}to attrs=sambaLMPassword,sambaNTPassword
    by dn.base="cn=admin,dc=example,dc=org" write
    by dn.base="cn=sambaservers,dc=example,dc=org" write
    by anonymous auth
    by self write
    by * none

After that, there is some sanity in system setup.

To get all the "Samba-parts" working into your LDAP, Microsoft Knowledgebase article Q243330 about Well-known security identifiers in Windows operating systems is a must-have reference. While debugging the LDAP-queries you will face something like this:

ldapsearch -x -b "dc=example,dc=org" \
"(&(&(objectclass=sambaGroupMapping)(sambaGroupType=4))
(|(|(|(|(|(|(|(|(|(sambaSIDList=S-1-1-0)(sambaSIDList=S-1-5-2))(sambaSIDList=S-1-5-11))
(sambaSIDList=S-1-22-2-1099))
(sambaSIDList=S-1-5-32-545))
(sambaSIDList=S-1-22-2-1101))
(sambaSIDList=S-1-22-2-1102))
(sambaSIDList=S-1-5-32-544)))))" cn gid

Initial impression will be WTF!? However, most of the SIDs are needed in your LDAP to make Windowses happy.

Some kind of setup wizard would be nice. It would save couple of weeks debugging / setup time.

Looks like running a blog has surpassed e-mail as the means of conveying spam. I wrote earlier about lot of automated comments, but the freemason idiots seemed to stop as they realized that their valuable information is not getting posted.

It does not mean, that I was left alone. Couple of other idiots started the same thing and I had to do something to stop their stupidity. So, I created a personal account at Akismet, there are plenty of information about them and most of the comments are about how using their service stops the spam flood completely. Luckily Serendipity supports Akismet's service out-of-the box and the setup was very simple.

Looks like, they're doing the same thing for blogs as SpamCop is doing for e-mail. And that is, essentially grinding spamming to halt. SpamCop have proven their value, it remains to be seen how effective Akismet actually is.

Gillian from Oracle informed me that my query is not valid SQL and the 5.5 version worked just because I was lucky.

The correct way of using aggregate function count() is something like this:

SELECT mlh.changedate, mlh_latest.counts, mlh.level
FROM memberlevelhistory mlh
INNER JOIN (
    SELECT member, MAX(changedate) as maxdate, COUNT(changedate) as counts
    FROM memberlevelhistory
    WHERE member = 5
    AND approved <> 'N'
) AS mlh_latest ON mlh.member = mlh_latest.member AND mlh.changedate = mlh_latest.maxdate
WHERE mlh.member = 5
AND mlh.approved <> 'N';

Now the result is equally correct on both tested versions.

MySQL 5.6.10 handles INNER JOIN / subquery -pair differently than 5.5.29. I found out this by accident when working code ceased to return proper results.

Example setup, a very simple table and couple of rows:

CREATE TABLE `memberlevelhistory` (
  `member` tinyint(3) unsigned NOT NULL,
  `changedate` date NOT NULL,
  `level` int(10) unsigned NOT NULL,
  `approved` char(1) NOT NULL,
  PRIMARY KEY (`changedate`,`member`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO `memberlevelhistory`
(`member`, `changedate`, `level`, `approved`)
VALUES
(5, '2009-08-01', 1, 'Y'),
(5, '2009-08-27', 2, 'Y'),
(5, '2009-10-01', 4, 'Y'),
(5, '2010-01-01', 5, 'Y'),
(5, '2010-02-01', 8, 'Y'),
(5, '2010-03-15', 9, 'Y'),
(5, '2011-02-01', 11, 'Y'),
(5, '2011-05-01', 12, 'Y'),
(5, '2012-02-01', 13, 'Y'),
(5, '2012-03-01', 14, 'Y'),
(5, '2012-04-01', 15, 'Y');

Description of columns:

• member: user ID
• changedate: when member lever was changed
• level: user level
• approved: level change approved by administration
  

The idea of the table is that most recent approved level is user's current level.

Example query to get user's current approved level with total number of approved user levels:

SELECT mlh.changedate, count(*), mlh.level
FROM `memberlevelhistory` mlh
INNER JOIN (
    SELECT member, changedate, level
    FROM `memberlevelhistory`
    WHERE member = 5
    AND approved <> 'N'
    ORDER BY `changedate` DESC
) AS `mlh2` ON mlh.member = mlh2.member AND mlh.changedate = mlh2.changedate
WHERE mlh.member = 5
AND mlh.approved <> 'N'

MySQL 5.5 result, current level as expected:

+------------+----------+-------+
| changedate | count(*) | level |
+------------+----------+-------+
| 2012-04-01 |       11 |    15 |
+------------+----------+-------+
1 row in set (0.00 sec)

MySQL 5.6 result, a surprise here:

+------------+----------+-------+
| changedate | count(*) | level |
+------------+----------+-------+
| 2009-08-01 |       11 |     1 |
+------------+----------+-------+
1 row in set (0.00 sec)

The query behaviour has changed. The subquery ORDER BY -clause has no effect. I did solve the problem of latest level with LIMIT 1 in the subquery, but it ruins the COUNT(*). I'm still working to replicate the 5.5 result in a single query, if a solution can be found, I'll blog about it.

I got bunch of automated comments to this blog. The comments were very generic about "how great this blog is" and "how fast the site loads", blah. blah. I typically check the moderation box for my blog entries, so they were just hoping to get automated publicity. In my case I just deleted the crap.

The idea of this spam-campaign was to distribute links to freemasonrysecrets.com

WTF?! Who cares about that?

SQLite has very little support for typical arithmetic functions. In the SQLite contrib-section there is an extension for that by Liam Healy. The description goes:

Provide mathematical and string extension functions for SQL queries using the loadable extensions mechanism.

• Math: acos, asin, atan, atn2, atan2, acosh, asinh, atanh, difference, degrees, radians, cos, sin, tan, cot, cosh, sinh, tanh, coth, exp, log, log10, power, sign, sqrt, square, ceil, floor, pi
• String: replicate, charindex, leftstr, rightstr, ltrim, rtrim, trim, replace, reverse, proper, padl, padr, padc, strfilter
• Aggregate: stdev, variance, mode, median, lower_quartile, upper_quartile

To ease the installation, I packaged into into a RPM for CentOS 6:

• sqlite-extension-functions-2010.01.06-1.x86_64.rpm
• sqlite-extension-functions-2010.01.06-1.src.rpm

The source-RPM will build quite easily on any RPM-disto. There are no weird dependecies or anything.

Example usage:

# sqlite3
SQLite version 3.6.20
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .load libsqlitefunctions.so
sqlite> select floor(1.9);
1
sqlite> select ceil(2.1);
3
sqlite> select reverse("reverse");
esrever
sqlite> .quit

Now developing apps with SQLite back-end is much easier.

I've been having Sonera (or TeliaSonera) Internet connection for ages. Occasionally I refer to my own IP with the DNS-name and now they chose to change them. It looks like they chose to change the subscriber identifier part of FQDN to indicate IPv4 address instead of some sort of internal identifier.

Typical Sonera broadband dynamic IP-address' reverse-DNS FQDN has format:

• Connection type: (fixed: dsl or cable)
• City identifier: always 3 characters
  
• Gateway identifier: (example: brasgw1)
• Subscriber identifier: hex-decimal -combo
  
• Dynamic broadband pool identifier: (fixed: dhcp.inet.fi)

A regexp would be:
^(dsl|cable)-([a-z]{3})([a-z0-9]+)-([0-9a-f]{6,}-\d{1,3})\.dhcp\.inet\.fi$

The old subscriber identifier had 8 hex digits, a dash and 1-3 decimal digits. For example: fe82eb00-56

The new subscriber identifier has 6 hex digits to represent the first 3 bytes of IPv4 address, a dash and 1-3 decimal digits for the last byte of IPv4 address. For example IPv4 address of 21.32.43.54 would be: 15202b-54

Wishful thinking: Are they finally preparing to offer IPv6?