Skip to content

Bash and Template Files

Last week I had to migrate my dns server and to do stuff properly I also added a couple of slave DNS servers just to make sure. Problem: ~15 domain names that need to be added to a fresh bind installation. The zone files didn’t change too much so that was fine, sed helped me out with the replacing of old IPs. But what about the bind configuration ? I had to add add the following declaration for all the 15 domains and on each of the slave dns server the “slave” equivalent:

zone "example.com" {
        type master;
        file "/etc/bind/zones/example.ro/zone.db";
};

I was in the mood form automation but I wanted something light and quick to setup. Obvious answer: bash.

I created a “template” file where and I replaced the domain name with a place holder:

zone \"$domain\" {
        type master;
        file \"/etc/bind/zones/$domain/zone.db\";
};

I then wrote a small bash loop that walks the array of domains and feeds them one by one to the template file. The end result was a nice config file with all of the domain names.

#!/bin/sh
for i in `find * -prune -type d`; do
domain=$i
eval "echo \"$(cat db.zones.tmpl)\""
done

The essence here is that the eval function forces bash to do parsing and variable replacement once more on the argument.

Yakuake

I’m going to start a series of blog posts about linux apps that I can’t live without. After a fresh system install these are the first installed.

The more and more I use Linux the more I find myself stuck into some console. While the default terminal app shipped with Gnome is just fine for small jobs it gets annoying when I accidentally close it and or when it’s not on the right desktop. Alt-tab-ing all the time is painful.

This is where the geek in me comes in. Remember the times of Quake and the “console”, when you pressed the ’tilda’ key (~) and a nice console dropped down from the top of the screen allowing you to type game commands ? Well that nifty little concept  made its way to the modern desktop!

Enter Yakuake or “A drop-down terminal emulator based on KDE Konsole technology” as the About menu says. I install it even if I use gnome and although it ads some tens of megs of libraries and breaks the “pure gnome install”. It’s so damn worth it! Pressing a magic key (default F12) instantly pops out the Yakuake window. Do your thing in the console and press the magic key again and the console disappears making way for your other apps. It’s simply brilliant. Yakuake has support for multiple terminal tabs so you can have multiple sessions opened. Using short cut keys you can quickly change the tabs. I consider that Yakuake improves my productivity just as much as the ‘screen’ utility, even tough it has nothing to do with ‘screen’ :)

Yakuake is not the only app that offers this type of functionality. Tilda is another ‘quake console clone’ and it’s based on GTK instead of QT.

Another alternative is ‘guake’, a younger app that aims at the same goal: quake style terminal emulator. It is also GTK based , hence the ‘g’, and it feels more snappy than tilda. With a bit of customizing it comes close to yakuake but I find it a bit unpolished.

Personally I find Yakuake faster and more friendly but it’s all about habits. Give them all a chance!

If you are using a Mac and want the same thing take a look at Visor ( Quake3 anyone? ). It is roughly the same thing.

The same thing goes for you Windoes Powershell lovers out there. You can have the amazing Powershell in a quake console: http://poshconsole.codeplex.com/

Nice pic goes here.

Hardware Keylogger Detection

Intro

Keyloggers are nifty things used to grab keyboard input for later analysis and use.

The most common type of keylogger is the software keylogger as it is just some program that records key presses. Running it usually requires administrator privileges and this limits the attack surface as one must first gain this type of privileges.

Hardware keyloggers on the other hand are just small devices that sit between the keyboard and PC and listen and record all the keystrokes.

This article refers to the KeyDaemon USB, a hardware keylogger produced by the Polish company KeeLog. Official website

The device is pretty small: on one end there’s an USB A plug (this goes to the PC) and on the other end there’s an USB A receptacle (here the keyboard is plugged). The plastic casing contains a microcontroller that listens in to the USB traffic. A 2GB storage module is also embedded in the small casing and is used to store the sniffed key presses.

The default mode is the “record mode” where the device silently records data. When a magic 3-key combination is pressed ( KBS is the default) the keylogger goes into “playback mode” and it makes visible the embedded flash storage. The keyboard is disabled and in place a flash storage device is detected by the operating system. Inside the flash disk there are two important files: log.txt which contains all the key presses captured by the keylogger and config.txt, which contains configuration settings for the device. By editing config.txt one can change the magic key combination (more on this later).

The keylogger is transparent and does not change the USB vendor ID or product ID. One can not tell that the device is plugged in just by looking at the USB device enumeration ( lsusb for example). This makes the detection of the device quite difficult at first sight.

Detection

The initial approach was to see if the current drain on the USB port was higher with the keylogger in place, but this turned out to be too impractical as it’s hard to do an accurate measurement. The OS support for this is also limited and I felt that this was a dead end.

Next I wanted to see how the keylogger handled unexpected flows of commands. The ioctl interface in linux permits changing the leds on the keyboard from user space. This nifty python script makes your keyboard flash the leds like a Las Vegas Casino billboard:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import fcntl
import os
import time
 
KDSETLED = 0x4B32
SCR_LED  = 0x01
NUM_LED  = 0x02
CAP_LED  = 0x04
 
console_fd = os.open('/dev/console', os.O_NOCTTY)
 
all_on = SCR_LED | NUM_LED | CAP_LED
all_off = 0
 
while 1:
	fcntl.ioctl(console_fd, KDSETLED, all_on)
	fcntl.ioctl(console_fd, KDSETLED, all_off)

Unfortunately the keylogger had no trouble in handling this type of data.

The next step was to see how the device handles traffic sent from the PC to the keyboard and not the other way around. One would expect that a properly designed keylogger would also take into consideration the direction of the traffic flow. As you will see this is not the case…

The USB keyboard uses the HID protocol to send and receive data to and from the PC. The basic data packet (or report) has 8 bytes. The first byte contains keycodes for the modifier keys (ALT, SHIFT, CTRL), the second byte is reserved for OEM and the remaining 6 bytes are used to send the keycodes for the pressed keys. So one can simultaneous press the modifier keys and 6 other keys.

Using libusb I wrote a very small application that sends the magic 3-key combination out to the keyboard. I didn’t think that this would work as I was expecting a propper design but to my surprise the device flash storage opened like a Christmas present from the first run. One interesting detail is that the keylogger needs to “warm up” since it was first plugged in. Sending the magic key combination immediately after plugging it in does not open the flash storage.

The next logical step was to further extend the app to brute force all the 3-key combinations. If no data can be written to the device then the flash storage has just popped up and a keylogger has been detected.

The device does not take into account the order in which the magic keys are pressed so KBS is also KSB or BKS. Also due to the fact that one can press a key only once a key combination of AAA for example would be invalid. This makes the number of key combinations very small and the brute force takes less than 1 second! While the code needs some polishing and some bugfixes (the USB handle is not cleanly released and this makes the keyboard unusable after execution – it needs to be replugged) it clearly proves that hardware keyloggers can be detected. This is a design flaw it will take a while until it will be fixed.

The PS2 variant of the keylogger has the same flaw and can be detected in the same manner : send all 3 key combinations out the ps2 port and the device will be detected at some point.

My proof-of-concept code uses a small library written by Paul J Stoffregen and Robin Coon of pjrc.com.

Given the fact that this device is so weak (less than 1s to detect) a script or service could be executed at boot time. Maybe companies that write security software could use this to implement a hardware keylogger detection function.

Other devices use a passphrase instead of a key combination. This is harder to brute force and as stated above it all depends on the way the keyloger snoops the traffic. If it does take into account the source of the traffic then the above method is useless and the keylogger is still invisible.

Tar file with the code : keelogdetect.tar
Source code available via SVN : http://code.google.com/p/cdumitru/source/browse/#svn/trunk/keelogdetect

Parsing XML data using bash and standard Unix tools

Parsing XML can be a tedious and unpleasant job if you insist on using just standard Unix tools like sed, awk, cut, grep and so on. One might say that it’s better to use python/perl/ruby/other language that ships with a full blown XML parser and use the standard Unix utilites for what they were meant for, plain old text files and not pesky XML. The problem with those nice programming languages is that they take away the one liners. You need to import stuff, have variables, flow control and so on.

A nice tool that makes one’s life easier when it comes to XML is XML2. It can convert a normal xml file to a more line oriented file format. The standard debian distribution has this neat tool in the repos so you are one apt-get away from using it.

 

One simple example. Take this XML file:


<xml>
<fruits>
<fruit name="apple" type="royal gala" quantity="2" price="1"/>
<fruit name="orange" type="tasty" quantity="4" price="1.5"/>
<fruit name="banana" type="green" quantity="3" price="1"/>
</fruits>
</xml>

We run xml2 against it:

cosu@roadwarrior:/tmp$ xml2 < fruits.xml
/xml/fruits/fruit/@name=apple
/xml/fruits/fruit/@type=royal gala
/xml/fruits/fruit/@quantity=2
/xml/fruits/fruit/@price=1
/xml/fruits/fruit
/xml/fruits/fruit/@name=orange
/xml/fruits/fruit/@type=tasty
/xml/fruits/fruit/@quantity=4
/xml/fruits/fruit/@price=1.5
/xml/fruits/fruit
/xml/fruits/fruit/@name=banana
/xml/fruits/fruit/@type=green
/xml/fruits/fruit/@quantity=3
/xml/fruits/fruit/@price=1

And now we extract all the fruit names:

cosu@roadwarrior:/tmp$ xml2 < fruits.xml |grep name |cut -d"=" -f2
apple
orange
banana

There you go! A fruit salad! Of course for more complicated stuff use other tools :)

 

Internet Exchange Points

The largest Romanian IXP is Interlan . Funny enough, Interlan is a response of the smaller ISPs  to the other big Romanian IXP, Ronix. Because 3 years ago joining Ronix was a complicated affair, a few small companies decided to take matters into their own hands. Currently,  Interlan has 3 times more traffic than Ronix.

Joining pdf files

Combining multiple pdfs into a single file can be handy for putting together one big final report or for submitting a single print job instead of multiple smaller ones. Joining pdfs in a Debian based Linux distribution can be easily done by using the pdfjoin utility. It is provided by the pdfjam package. One only needs to

sudo aptitude install pdfjam

Then all that needs to be done is cd-ing into the folder containing the large number of pdfs and running:

pdfjoin *.pdf –outfile out.pdf

There you go, instant pdf!

Choosing random entries from a group

In the past two weeks we had a lottery-type thing on RGC.ro (Romanian Guitarist Community). Proguitar, the official importer of Fender products in Romania, wanted to give-away a custom made Fender Stratocaster electric guitar. To register, the community users had to fill out a form and choose from a series of custom options for the guitar.

As organizers we had to pick out the lucky winner of the raffle. Usually this is done by someone who is impartial. Due to the fact that we had about 1600 entries and that we are geeks we wanted to do something that geeks would do. Therefore we ditched the “extract the name of the lucky winner from a bowl”. The geek version of this is described in RFC2777 – Publicly Verifiable Nomcom Random Selection

In short RFC2777 describes a simple publicly verifiable algorithm to pick out a set of entries from a group as random as possible. The keywords here are public – anyone can see how the entries are picked – and as random as possible. To have random values a thing called information entropy is needed. To get that initial random value full of juicy entropy we used, as suggested in the RFC, the results from three international lotteries. This initial random value was slightly modified for each “extracted” entry and then transformed into a MD5 hash. Due to the nature of a hash when slightly modifying the original the resulting hash differs heavily from the original hash.

Below you can find a naive python implementation that can be freely used for any purpose. Just make sure you fill in the entropySource with a good initial random value.

import md5                                                 

if __name__ == '__main__':

    entropySource = "9.24.30.32.36.40./18.25.35.43.46.47./1.3.4.8.23.31./"

    numberOfEntries = 1655
    numberOfWinners = 10  

    numbers = map( lambda x: x + 1, range( numberOfEntries ) )

    i = 0
    entries = numberOfEntries
    print "index \t hex value of MD5 \t div \t selected"
    while ( i < numberOfWinners ) :
        md5hash = md5.new()
        md5hash.update( chr( i ) + entropySource + chr( i ) )
        val = int( md5hash.hexdigest(), 16 )
        modulo = val % entries
        print str( i + 1 ) + "\t" + md5hash.hexdigest() + "\t" + str( entries ) + "\t" + str( numbers[modulo] )
        del numbers[modulo]
        i += 1
        entries -= 1

SNE Update 1

So over two months have passed since my last update. I was either  busy or not in the mood to update my blog. I will try to make up for lost posts somehow…

I’m now enrolled to the System and Network Engineering Master at University of Amsterdam. How I got here?

Continue reading ‘SNE Update 1’ »

Great success!

(that’s what Borat would say)

Today I’ve received wonderful news! I have been accepted to the System And Network Engineering Master at the University of Amsterdam! Starting from the end of August I’ll be relocating to Amsterdam for one year of full geek experience (hopefully!). I can not thank enough my girlfriend on being such a great support and motivator.Without her nothing could have happened.  Also my teachers (esp prof. Rughinis and prof. Tapus) at the Faculty of Automatic Control and Computer Science at University POLITEHNICA of Bucharest have been great mentors and supporters of my admission.

This could be a good time to add this blog to your RSS reader as starting from September I’ll be posting regularly on both geek related stuff and the lifestyle of an international student in the Netherlands.

Get your personal email account

Most people use free email services like yahoo, gmail or live. Unfortunately all the nice sounding email addresses are taken by now so new users have to come up with strange combinations like johndoe19__smth_smth@yahoo.com. That’s very hard to remember and it sounds very unprofessional.

Having an online presence is no longer such a big deal. With a few dollars a year you can get your own .com (or other top-level-domain) and another few dollars a month get you a hosting plan which provides you a couple megabytes for website storage and a number of email accounts. So with a small investment you can have a decent email like name.sourname@somedomain.com . That’s something that you could put on your personal business card. Few know that you can skip the email service offered by your webhost  and instead use a more reliable service.

Both Microsoft and Google offer domain email hosting as a free service. Microsoft calls this Windows Live Custom Domains ( https://domains.live.com/ ) while Google calls it’s service Google Apps ( http://www.google.com/apps/intl/en/group/index.html )

Using these services is quite simple. You just have to prove that you are indeed the owner of the domain and make some DNS modifications so that emails will be handled by Google or Microsoft. Modifying the DNS records is a process that can be made using the web interface set up by your hosting provider (the one that hosts your DNS records) or by directly edition your DNS configuration in case you manage the DNS yourself. Either way both Microsoft and Google give you directions on how and what to modify.
For the tech savvy readers there are 2 basic steps: add a CNAME record containing a random string to prove that you are the rightful owner and then modify the MX records with the one provided in the instructions. It’s not that complicated.

Why should you do this?
Well both Microsoft and Google provide a better service than a normal hosting company when it comes to reliability. Sure, you don’t sign a contract that mentions any SLA but statistically speaking both offer a kick-ass service. You don’t have to worry about backups, downtime, spam and so on. It just works. For small operations, say personal email or small companies like startups , this kind of service is ideal as it cuts costs and/or gives less headaches.
Using the administration page you can create, delete or reset any email account. If someone messes up his/hers password you can simply reset the account. 
By using either the Microsoft based service or the Google one you get access to other related services like Office Online or Google Docs because the created email accounts serve as Live IDs or Google Accounts. This opens a new world of online collaboration. I know a few startups that use these kind of services.

What are the downsides?
You don’t own your email (carefully read the EULA’s ) and some may not like this.
You are limited to 50 or 100 email accounts and when you hit that limit you have to upgrade to a paid service. Individuals and small companies will just ignore this.
The web mail interface will display ads just as gmail.com or live.com. Adblocker type software could make this a non-issue.
You get little to no tech support. This can be neglected by individuals or small companies considering the advantages.

Access to the email account is made either by browser or by email client. Google Apps email can be accessed by POP3, IMAP and webmail. Unfortunately Windows Live Custom Domains does not offer access using the IMAP or POP3 protocols. To use Outlook you need to install a small piece of software called Office Outlook connector. The advantage of this approach is that besides email you can synchronize your address book and calendars. The IMAP and POP3 protocols don’t allow that. For Thunderbird + live you need a plugin but you get only basic service : get/send emails, no calendar :( .

With 9$ a year you could get a .com domain. You just need a public DNS server to host your records and that’s it, you can sign up for free email hosting.

Regarding DNS hosting, this is really not an issue. http://freedns.afraid.org/ is a very good option. If you don’t like it you could always ask your geek friend to help you out.

It’s hard to tell which service is best. Right now I’m using both Live Custom Domains and Google Apps and I’m quite happy with either one. It all depends on what you want to achieve.

After a year or more of using Goggle Apps I’m thinking of decommissioning all of my postfix installs (yes postfix is better than qmail) and switching to one of the above options. Having a full blown email server (even if it’s just a virtual machine with just enough resources serving many domains by means of sql and virtual domains) seems more and more a waste of time and resources for small operations.

I have a gut feeling that more and more companies will outsource the email service. I’ve seen this happening on a large scale in a few Universities in Romania.  The Bucharest Academy of Economic Studies is using Google Apps to offer email accounts to all it’s students ( that’s more than 20.000 accounts!). Likewise there’s a small implementation of Live @EDU , a Microsoft programme that basically does the same thing, in the Faculty of Automatic Control and Computers at the POLITEHNICA University in Bucharest (that’s about 3000 accounts, give or take).