A nice tool that makes one’s life easier when it comes to XML is XML2. It can convert a normal xml file to a more line oriented file format. The standard debian distribution has this neat tool in the repos so you are one apt-get away from using it.

One simple example. Take this XML file:


<xml>
<fruits>
<fruit name="apple" type="royal gala" quantity="2" price="1"/>
<fruit name="orange" type="tasty" quantity="4" price="1.5"/>
<fruit name="banana" type="green" quantity="3" price="1"/>
</fruits>
</xml>


We run xml2 against it:

cosu@roadwarrior:/tmp$ xml2 < fruits.xml
/xml/fruits/fruit/@name=apple
/xml/fruits/fruit/@type=royal gala
/xml/fruits/fruit/@quantity=2
/xml/fruits/fruit/@price=1
/xml/fruits/fruit
/xml/fruits/fruit/@name=orange
/xml/fruits/fruit/@type=tasty
/xml/fruits/fruit/@quantity=4
/xml/fruits/fruit/@price=1.5
/xml/fruits/fruit
/xml/fruits/fruit/@name=banana
/xml/fruits/fruit/@type=green
/xml/fruits/fruit/@quantity=3
/xml/fruits/fruit/@price=1

And now we extract all the fruit names:

cosu@roadwarrior:/tmp$ xml2 < fruits.xml |grep name |cut -d"=" -f2
apple
orange
banana


There you go! A fruit salad! Of course for more complicated stuff use other tools

The largest Romanian IXP is Interlan . Funny enough, Interlan is a response of the smaller ISPs  to the other big Romanian IXP, Ronix. Because 3 years ago joining Ronix was a complicated affair, a few small companies decided to take matters into their own hands. Currently,  Interlan has 3 times more traffic than Ronix.

sudo aptitude install pdfjam

Then all that needs to be done is cd-ing into the folder containing the large number of pdfs and running:

pdfjoin *.pdf –outfile out.pdf

There you go, instant pdf!

As organizers we had to pick out the lucky winner of the raffle. Usually this is done by someone who is impartial. Due to the fact that we had about 1600 entries and that we are geeks we wanted to do something that geeks would do. Therefore we ditched the “extract the name of the lucky winner from a bowl”. The geek version of this is described in RFC2777 – Publicly Verifiable Nomcom Random Selection

In short RFC2777 describes a simple publicly verifiable algorithm to pick out a set of entries from a group as random as possible. The keywords here are public – anyone can see how the entries are picked – and as random as possible. To have random values a thing called information entropy is needed. To get that initial random value full of juicy entropy we used, as suggested in the RFC, the results from three international lotteries. This initial random value was slightly modified for each “extracted” entry and then transformed into a MD5 hash. Due to the nature of a hash when slightly modifying the original the resulting hash differs heavily from the original hash.

Below you can find a naive python implementation that can be freely used for any purpose. Just make sure you fill in the entropySource with a good initial random value.

import md5                                                 

if __name__ == '__main__':

    entropySource = "9.24.30.32.36.40./18.25.35.43.46.47./1.3.4.8.23.31./"

    numberOfEntries = 1655
    numberOfWinners = 10  

    numbers = map( lambda x: x + 1, range( numberOfEntries ) )

    i = 0
    entries = numberOfEntries
    print "index \t hex value of MD5 \t div \t selected"
    while ( i < numberOfWinners ) :
        md5hash = md5.new()
        md5hash.update( chr( i ) + entropySource + chr( i ) )
        val = int( md5hash.hexdigest(), 16 )
        modulo = val % entries
        print str( i + 1 ) + "\t" + md5hash.hexdigest() + "\t" + str( entries ) + "\t" + str( numbers[modulo] )
        del numbers[modulo]
        i += 1
        entries -= 1

I’m now enrolled to the System and Network Engineering Master at University of Amsterdam. How I got here?

It started in January last year when my lovely girlfriend showed me the the http://www.os3.nl web page. I have a strong background in networking and system administration and I was very excited after I read the master presentation. I decided that I should apply for the master!

To apply, a number of difficult steps had to be made. The first one was taking the TOEFL test. TOEFL (test of English as a foreign language) is a simple assessment of your English language knowledge. It does not test grammar or advanced vocabulary. The test has 4 sections : reading, listening, speaking and writing. Preparation materials can be found in certain sources and more than a few days of “training” are more than enough. The difficult part is the very long duration: 4 hours! When I left the testing I barely knew what my name was and where I should go next. After about 3 hours I almost lost motivation. My only wish was to get it finished because I was bored and pissed off.  The results are posted on the TOEFL web site after about 3 or 4 weeks. They need to be sent to the University in a timely manner separate from the application documents. Sending the results the first time is free. Additional submissions are not. The cost of the TOEFL course was about 170$. Besides the actual test, extra care must be taken when registering for the test. The number of seats is limited and reservation must be made in advance. I was very lucky to book in February and take the test at the beginning of March. Booking should be made with a few months in advance to be sure to get a seat.

Now that I was done with TOEFL, I sent my actual application form. There is a pre application form on the UvA website which must be filled in with personal data, unofficial transcripts and a letter of motivation. I believe that the preform is a method of screening the applicants. If you pass this phase, in a few weeks an official form is sent to you by email. This one must be printed and filled in. I had to obtain official transcripts from my University (PITA), two letters of recommendation ( I got them from two of my professors: Nicolae Tapus and Razvan Rughinis), a CV and another letter of motivation. I sent them and began to wait for the answer. It should have arrived at the end of May but it didn’t. After some more waiting I was informed that I had to take an intake test (see the OS3 web page – admission). The intake test consisted of 6 sections: TCP/ Networking, Discrete math(logic, set theory, graph theory),Unix, English writing (I had to make a brief on the TELNET RFC), English comprehension (some questions about the content of the RFC) and bash scripting. The first three were “pick the answer” kind of tests while the 4th and 5th involved typing text. The last one required creating working bash scripts. Nothing too fancy : use of the standard Unix tools (cut, grep,tr, sed, awk) and some knowledge on bash scripting (ifs, loops, etc). I took the test at the beginning of  August, after 2 months of waiting. The result came after about two weeks: I was accepted!

During the admission process, the communication with the SNE staff and the International Officer at UvA was excellent. I was very pleasantly surprised. My previous contact with University officials from Romania was not that positive …

I can’t give information on any scholarships as I applied too late. Application for a scholarship has to be made sooner. Check with the  International office!

A bit about the SNE OS3 master. There are only two masters programmes in Europe that have a system and network engineering track. SNE at UvA is one of them. The second one is in Oslo, Finland. This aspect makes this education to stand out from all the rest. The setting in which it takes place is also special. It lasts for one full academic year. There is an option to follow the master in two years as a part time student. More on that later. Back to the setting. In most universities students have to follow a mandatory “core” course pack which is directly connected to the master track. Additional courses can be picked form a very large pool depending on the student’s interest in various fields. This means that certain courses may overlap or some blocks/semesters can be busier than others. At SNE, there is no option in choosing the subjects. The list of subjects is decided beforehand for a very good reason: the SNE courses are available only to SNE students. Any other students are not allowed to participate. Also the schedule is very strict: 10-16 everyday (with a 30min – one hour lunch break ). During the morning, a lecture is given and in the afternoon students work on assignments related to the before given lecture.

The Dutch educational system organizes the academic year into blocks. A block has 8 or 4 weeks. At SNE, during one 8 week block, 2 courses are given, twice a week. The lectures take place during the first 7 weeks while exams are taken in the 8th. So two exams in October and two exams in December. In Romania I had 5  or 6 courses over a period of 14 weeks with 3 weeks of exams. The learning process is accelerated while the focus is kept on only two subjects. I still don’t know which approach is better.

The master thesis is split into two Research Projects. One is in January and another one in June. The projects take place at different companies and they can involve a number of subjects (security, large scale systems, networking, etc).

The SNE lab is open only to SNE students. The lab has two rooms, a lecture room and a server room. The lecture room has dual screen eco-friendly workstations for each student while the server room houses racks with experimentation servers for the students. Each student has his own 1U server and his own /27 public IP space. Lectures are given by the SNE staff but on several ocasions guest lectures from people with more expertise in a particular field are given.

The mix of students is still primarily Dutch with only 4 international students. I have another colleague from Romania (he studied in Timisoara at UPT), a pole and a chinese guy. The low number for internationals is because this was the first year when admission was open to foreigners.

The master can be finished in two years, as a part-time student. The part-time is a bit tricky as part-time means that you still have to be 3 full days at school. You get one course per block and also you can choose when to make the reaserch projects.

Untill now I had two courses: Essential Skills for Administrators(ESA) and Classical Internet Applications(CIA).

The ESA course was a crash course into stuff that normally should have been known from a previous education. It’s role is to set a common set of knowledge in regards of using an Unix environment, for all the students. The subjects treated are: versioning systems(cvs,svn,git), LaTeX (to create fancy documents), regex’s ,sed & awk (to process text files), xhtml/css/xml/xslt and three programming languages that can also be verywell used for scripting : Python, Perl and Ruby. The programming languages are presented from a basic OO level keeping language specifics to a minimum. The exam consists of a number of non-trivial questions and a practical part where the student must make use of LaTeX and xhtml.

The CIA course is more theoretical. It focuses on Unix and Internet history. The “classical” means that the course tackles protocols and standards that are at the core of the internet. It first starts with partitioning  by presenting various partitioning schemes (from DOS to GPT , Disklabels, Apple  ), CHS and LBA etc, goes to booting (ARP, RARP, BOOTP, DHCP), DNS , email, HTTP and lastly Directories. Without a doubt the most interesting part of this course was the DNSSEC chapter. DNSSEC is a very hot topic and therefore we spent 2 lectures on it of which one was a guest lecture from a SurfNET representative (Surfnet the equivalent of Roedu, the national education and research network).The exam consists of questions from the lectures.

The  teaching is of very high quality, yet the class atmosphere is not rigid. Dutch people have a way of making you comfortable while still being distant and polite. Working in such an exclusive environment means that contact with the other colleagues is more intense than in a normal master setting.

Certain aspects resemble my previous education at UPB but I feel that overall more effort is made by the professors.

For each course certain assignments are given. These assignments must be completed in order to be able to pass the course. They do not count to the grade in any way but not doing them in a sufficient manner means that you will fail the course. Evaluation of the assignments is made in a very straight forward way: RED/YELLOW/GREEN. If you get red you still have time to change/add stuff until the end of the block. Also there are no deadlines in completing the job except for the end of the block. The assignments involve a lot of hours of work as things have to work and they also need to be documented. Each student has to keep a log of his work. The log gets “graded” so doing a sloppy job in documenting even though everything works might still get you a red. A frustrating aspect is that there is no clear marking on how  red , yellow or green are given. On some assignments I spent a lot of time which translated in a “nice” or “excellent” comment but for getting just green with no comments only half of the effort was necessary. I lack good time management skills so I had to finish  some assignments in the exam week. This might get stressful and it also might affect the actual exams. Relaxed deadlines are a two edge sword…

My logs are publicly available here: https://www.os3.nl/2009-2010/students/cosmin_dumitru/start

It’s hard to give a strong opinion after two months. Right now things are getting more interesting as the following courses are far from basic. UvA has a partnership with Vrije University and so one of my mandatory courses is at Vrije. It’s called Distributed Systems and the lecturer is Marteen van Steen one of the authors of “Distributed Systems : Principles and Paradigms”. The second course I’m having this block is called Security of Systems and Networks. We will have about 10 guest lectures from various people on security and security related stuff.

The SNE functions like a group so the teachers seem more dedicated to the students. One example of this fact is the trip we’re going to have in December. All the students and professors will travel to Geneve to visit the LHC at CERN. Almost everything is payed by the SNE. On a normal master you wouldn’t have such an activity and even if that would be the case financial support is out of the question for such a large group. Group activities are organized from time to time: beer meetings, movie nights, special events, etc. The atmosphere in the group is very friendly and very open. There are some small quircks as computer people normally have social skills isuess but it’s nothing I’m not used to or that it can’t be overcome.

I think SNE is a very good opportunity as the education is not disconnected from the real world yet it doesn’t rely on proprietary technologies.It’s very good to have knowledge of the inner workings of Microsoft or Cisco but after some point the knowledge can’t be reused or easily adapted to some other context.

I will post some non-school related stuff at some point.

Sorry for the 2000 words posts!

Today I’ve received wonderful news! I have been accepted to the System And Network Engineering Master at the University of Amsterdam! Starting from the end of August I’ll be relocating to Amsterdam for one year of full geek experience (hopefully!). I can not thank enough my girlfriend on being such a great support and motivator.Without her nothing could have happened.  Also my teachers (esp prof. Rughinis and prof. Tapus) at the Faculty of Automatic Control and Computer Science at University POLITEHNICA of Bucharest have been great mentors and supporters of my admission.

This could be a good time to add this blog to your RSS reader as starting from September I’ll be posting regularly on both geek related stuff and the lifestyle of an international student in the Netherlands.

Having an online presence is no longer such a big deal. With a few dollars a year you can get your own .com (or other top-level-domain) and another few dollars a month get you a hosting plan which provides you a couple megabytes for website storage and a number of email accounts. So with a small investment you can have a decent email like name.sourname@somedomain.com . That’s something that you could put on your personal business card. Few know that you can skip the email service offered by your webhost  and instead use a more reliable service.

Both Microsoft and Google offer domain email hosting as a free service. Microsoft calls this Windows Live Custom Domains ( https://domains.live.com/ ) while Google calls it’s service Google Apps ( http://www.google.com/apps/intl/en/group/index.html )

Using these services is quite simple. You just have to prove that you are indeed the owner of the domain and make some DNS modifications so that emails will be handled by Google or Microsoft. Modifying the DNS records is a process that can be made using the web interface set up by your hosting provider (the one that hosts your DNS records) or by directly edition your DNS configuration in case you manage the DNS yourself. Either way both Microsoft and Google give you directions on how and what to modify.
For the tech savvy readers there are 2 basic steps: add a CNAME record containing a random string to prove that you are the rightful owner and then modify the MX records with the one provided in the instructions. It’s not that complicated.

Why should you do this?
Well both Microsoft and Google provide a better service than a normal hosting company when it comes to reliability. Sure, you don’t sign a contract that mentions any SLA but statistically speaking both offer a kick-ass service. You don’t have to worry about backups, downtime, spam and so on. It just works. For small operations, say personal email or small companies like startups , this kind of service is ideal as it cuts costs and/or gives less headaches.
Using the administration page you can create, delete or reset any email account. If someone messes up his/hers password you can simply reset the account. 
By using either the Microsoft based service or the Google one you get access to other related services like Office Online or Google Docs because the created email accounts serve as Live IDs or Google Accounts. This opens a new world of online collaboration. I know a few startups that use these kind of services.

What are the downsides?
You don’t own your email (carefully read the EULA’s ) and some may not like this.
You are limited to 50 or 100 email accounts and when you hit that limit you have to upgrade to a paid service. Individuals and small companies will just ignore this.
The web mail interface will display ads just as gmail.com or live.com. Adblocker type software could make this a non-issue.
You get little to no tech support. This can be neglected by individuals or small companies considering the advantages.

Access to the email account is made either by browser or by email client. Google Apps email can be accessed by POP3, IMAP and webmail. Unfortunately Windows Live Custom Domains does not offer access using the IMAP or POP3 protocols. To use Outlook you need to install a small piece of software called Office Outlook connector. The advantage of this approach is that besides email you can synchronize your address book and calendars. The IMAP and POP3 protocols don’t allow that. For Thunderbird + live you need a plugin but you get only basic service : get/send emails, no calendar .

With 9$ a year you could get a .com domain. You just need a public DNS server to host your records and that’s it, you can sign up for free email hosting.

Regarding DNS hosting, this is really not an issue. http://freedns.afraid.org/ is a very good option. If you don’t like it you could always ask your geek friend to help you out.

It’s hard to tell which service is best. Right now I’m using both Live Custom Domains and Google Apps and I’m quite happy with either one. It all depends on what you want to achieve.

After a year or more of using Goggle Apps I’m thinking of decommissioning all of my postfix installs (yes postfix is better than qmail) and switching to one of the above options. Having a full blown email server (even if it’s just a virtual machine with just enough resources serving many domains by means of sql and virtual domains) seems more and more a waste of time and resources for small operations.

I have a gut feeling that more and more companies will outsource the email service. I’ve seen this happening on a large scale in a few Universities in Romania.  The Bucharest Academy of Economic Studies is using Google Apps to offer email accounts to all it’s students ( that’s more than 20.000 accounts!). Likewise there’s a small implementation of Live @EDU , a Microsoft programme that basically does the same thing, in the Faculty of Automatic Control and Computers at the POLITEHNICA University in Bucharest (that’s about 3000 accounts, give or take). 

Besides the usual scripting basics like variables, functions, pipes and so on the later chapters show some usage of the scripting language for some more concrete problems like XML manipulation or user account management. Just give it a try, it’s free!

By default linuxes use the less command to display the man page requested by the user. The manpage is displayed as plain text and because of that it can be sometimes hard to find what you’re looking for. Keywords and special parameters are printed with a bold face to ease document navigation but sometimes this is not enough.

Navigation is done by using the up and down arrows , page up/ page down and the space key.
Searching through the document is done by typing the / character followed by the word or phrase to search for.

One useful hack is to color the manpage so that keywords parameters and so on are highlighted.

To do this you we have to set some environment variables:

export LESS_TERMCAP_mb=$'\E[01;31m' # begin blinking
export LESS_TERMCAP_md=$'\E[01;31m' # begin bold
export LESS_TERMCAP_me=$'\E[0m' # end mode
export LESS_TERMCAP_se=$'\E[0m' # end standout-mode
export LESS_TERMCAP_so=$'\E[01;44;33m' # begin standout-mode - info box
export LESS_TERMCAP_ue=$'\E[0m' # end underline
export LESS_TERMCAP_us=$'\E[01;32m' # begin underline


The strange ‘\E0 strings stand for color codes used by the bash shell. You can check out some info about that on the bash-prompt-howto

After you have customized your colors you can save the above commands in your .bashrc file (the one in your home folder) so that the variables are set every time you logon.

cosu@cosu-desktop:~/Desktop$ cat file | fold -s
my monitor resolution is soooooooooooooooooo small that more than 80 colums of
text give me a segfault.

-s stands for break at spaces. man fold for more options.