Background

A question came up on the Stack Exchange site Unix & Linux in which I wrote up a pretty good answer, that describes some of the mechanics of how a process deals with its user credentials, so I’m adding my writeup to the blog.

It really comes down to what makes up a process in Unix. A process can come into existence in one of 2 ways. Either via the fork() function or through one of the exec() functions in C.

fork()

fork() basically just makes a copy of the current process, but assigns it a new process ID (PID). It’s a child of the original process. You can see this relationship in the output of @ps@:

1 2 3 4 5 6 7 $ ps axjf PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 1 5255 1964 1964 ? -1 Sl 500 0:39 gnome-terminal 5255 5259 1964 1964 ? -1 S 500 0:00 \_ gnome-pty-helper 5255 18422 18422 18422 pts/1 18422 Ss+ 500 0:01 \_ bash 5255 30473 30473 30473 pts/4 30473 Ss+ 500 0:00 \_ bash 30473 782 782 30473 pts/4 30473 Sl 500 1:14 | \_ evince s.pdf

Here you can see that gnome-terminal is the parent process (PID = 5255) and that bash is it’s child (PID = 18422, PPID = 5255).

When a process forks from its parent, it “inherits” certain things, such as copies of all the file descriptors that the parent currently has for open files and the parent’s user and group IDs.

NOTE1: PPID = Parent Process ID.
NOTE2: The last 2 are what identify what file and group permissions this process will have when accessing the file system.

So if a process just inherits its user and group ID from its parent, then why isn’t everything just owned by root or a single user? This is where exec() comes in.

exec() Part #1

The exec() family of functions, specifically execve(), “replace” a current process image with a new process image. The terminology “process image” is really just a file, i.e. an executable on disk. So this is how a bash script can execute a program such as /usr/bin/time.

So what about the user ID and group ID? Well to understand that let’s first discuss the concept of “Persona”.

Persona

At any time, each process has an effective user ID, an effective group ID, and a set of supplementary group IDs. These IDs determine the privileges of the process. They are collectively called the [persona of the process]1, because they determine “who it is” for purposes of access control.

exec() Part #2

So in addition to being able to swap out the “process image”, exec() can also change the user & group IDs from the original “real” ones to “effective” ones.

An example

For this demonstration I’m going to show you what happens when we start out in a shell as our default UID/GID, and then spawn a child shell using one of my supplementary GIDs, making it the child shell’s effective GID.

To perform this I’m going to make use of the unix command newgrp. newgrp allows you to spawn a new shell passing it the supplementary group that I’d like to make my effective GID.

For starters:

1 2 $ id -a uid=500(saml) gid=501(saml) groups=501(saml),502(vboxusers),503(jupiter)

We can see that this shell is currently configured with my default UID/GID of saml & saml. Touching some files shows that this is the case as well:

1 2 3 4 5 6 $ touch afile1 $ touch afile2 $ ls -l total 0 -rw-rw-r-- 1 saml saml 0 May 21 23:47 afile1 -rw-rw-r-- 1 saml saml 0 May 21 23:47 afile2

Now we make our supplementary group jupiter the effective GID:

1 2 3 $ newgrp jupiter $ id -a uid=500(saml) gid=503(jupiter) groups=501(saml),502(vboxusers),503(jupiter)

Now if we touch some files:

1 2 3 4 5 6 7 8 $ touch afile3 $ touch afile4 $ ls -l total 0 -rw-rw-r-- 1 saml saml 0 May 21 23:47 afile1 -rw-rw-r-- 1 saml saml 0 May 21 23:47 afile2 -rw-r--r-- 1 saml jupiter 0 May 21 23:49 afile3 -rw-r--r-- 1 saml jupiter 0 May 21 23:49 afile4

We see that the shell’s effective GID is jupiter, so any interactions with the disk result in files being created with jupiter rather than my normal default group of saml.

References

• fork() man page
• exec() man page
• execve() man page
• credentials man page
• newgrp man page
• The GNU C Library
• The GNU C Library – 26.4 Creating a Process
• The GNU C Library – 26.5 Executing a File
• GID, current, primary, supplementary, effective and real group IDs?

Dear All, Very Happy to share a Links of One Another Distribution in Linux and It is Mageia Linux. Mageia is RPM Based Distribution. At Last I have Mentioned a Link to Download Mageia 3 Linux for 32-Bit ( i386,i686 ) and X86_64 ( 64-Bit ) Architecture. Major Features in Mageia 3 :- Updates to [...]

Backups are important, whether you are backing up your databases or your wedding pictures. The loss of data can ruin your day. While there is a huge list of backup software to choose from; some good, some not so good. One of the tools that I have used for years is rdiff-backup.

rdiff-backup is a rsync delta based backup tool that both stores a full mirror and incremental changes. It determines changes based on the rsync method of creating small delta files, which allows for rdiff-backup to restore files to any point in time (within the specified retention period).

In the examples below I will refer to two servers names, backup-server and server. The names are pretty self-explanatory but just in case, backup-server is the location where I permanently store files copied (backed up) from server.

Setting up rdiff-backup

Installing rdiff-backup is easy considering most Linux distributions include it into their default repositories. In this article I will be using Ubuntu for my example systems.

Note: For Red Hat you will need to enable the EPEL repository to install rdiff-backup via YUM.

Installing

In order for rdiff-backup to work both the source and destination will require the rdiff-backup package. You can install it via apt-get.

On backup-server:

root@backup-server# apt-get install rdiff-backup

On server:

root@server# apt-get install rdiff-backup Validate rdiff-backup versions match

One of the quirky things about rdiff-backup is that the tool does not support backwards capability with older versions. For this reason it is best to make sure that your rdiff-backup versions are the same on both servers.

On backup-server:

root@backup-server# rdiff-backup --version rdiff-backup 1.2.8

On server:

root@server# rdiff-backup --version rdiff-backup 1.2.8 Setting up SSH Keys

By default rdiff-backup uses SSH to communicate with remote systems to avoid typing a password every time rdiff-backup runs we will need to set-up SSH keys with passphrase-less authentication.

On backup-server:

root@backup-server# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub.

When asked leave the passphrase empty.

Once you have the SSH key generated you will need to copy the contents of /root/.ssh/id_rsa.pub to the remote servers for key-based authentication. For our configuration we will use a non-privileged user account (test), as this will let us implement rdiff-backup without giving the backup-server full access to the systems being backed up.

On backup-server:

root@backup-server:# scp /root/.ssh/id_rsa.pub test@server:/var/tmp/id_rsa.pub.temp

On server:

test@server:$ cat /var/tmp/id_rsa.pub.temp >> ~/.ssh/authorized_keys

You should now be able to SSH from backup-server to server without being asked for a password.

Running backup jobs

Now that backup-server is able to SSH to server without being asked a password and rdiff-backup is the same version on both systems we are able to perform the first backup.

The directory we will backup today is /var/tmp/backmeup and we will be backing it up to /var/tmp/backups/server.example.com/. I personally prefer to backup to a directory named after the originating server, that way there is no question as to where the files came from.

On backup-server:

root@backup-server:# mkdir -p /var/tmp/backups/server.example.com root@backup-server:# rdiff-backup test@server.example.com::/var/tmp/backmeup /var/tmp/backups/server.example.com/

rdiff-backup has now created a mirror of the /var/tmp/backmeup directory from server.example.com in /var/tmp/backups/server.example.com.

root@backup-server:# ls -la /var/tmp/backups/server.example.com/ total 52 drwxr-xr-x 3 root root 4096 May 19 13:07 . drwxr-xr-x 3 root root 4096 May 19 13:53 .. -rw-r--r-- 1 root root   25 May 19 13:07 10.file -rw-r--r-- 1 root root   24 May 19 13:07 1.file -rw-r--r-- 1 root root   24 May 19 13:07 2.file -rw-r--r-- 1 root root   24 May 19 13:07 3.file -rw-r--r-- 1 root root   24 May 19 13:07 4.file -rw-r--r-- 1 root root   24 May 19 13:07 5.file -rw-r--r-- 1 root root   24 May 19 13:07 6.file -rw-r--r-- 1 root root   24 May 19 13:07 7.file -rw-r--r-- 1 root root   24 May 19 13:07 8.file -rw-r--r-- 1 root root   24 May 19 13:07 9.file drwx------ 3 root root 4096 May 19 13:56 rdiff-backup-data

Now that we have backed up the original file we will run a second backup to capture changed data; this time a with a little more verbosity.

root@backup-server:# rdiff-backup -v5 test@server.example.com::/var/tmp/backmeup /var/tmp/backups/server.example.com/ Using rdiff-backup version 1.2.8 Executing ssh -C test@server.example.com rdiff-backup --server <truncated for length> Backup: must_escape_dos_devices = 0 Starting increment operation /var/tmp/backmeup to /var/tmp/backups/server.example.com Processing changed file . Incrementing mirror file /var/tmp/backups/server.example.com Processing changed file 1.file Incrementing mirror file /var/tmp/backups/server.example.com/1.file Processing changed file 10.file Incrementing mirror file /var/tmp/backups/server.example.com/10.file Processing changed file 2.file Incrementing mirror file /var/tmp/backups/server.example.com/2.file Processing changed file 3.file Incrementing mirror file /var/tmp/backups/server.example.com/3.file Processing changed file 4.file Incrementing mirror file /var/tmp/backups/server.example.com/4.file Processing changed file 5.file Incrementing mirror file /var/tmp/backups/server.example.com/5.file Processing changed file 6.file Incrementing mirror file /var/tmp/backups/server.example.com/6.file Processing changed file 7.file Incrementing mirror file /var/tmp/backups/server.example.com/7.file Processing changed file 8.file Incrementing mirror file /var/tmp/backups/server.example.com/8.file Processing changed file 9.file Incrementing mirror file /var/tmp/backups/server.example.com/9.file

As you can see -v5 tells us what files are being processed, this is handy to see what is being backed up or being restored.

Now if we only change files 1 – 3 and run rdiff-backup again rdiff-backup should only backup files that have changed leaving the others alone.

root@backup-server:# rdiff-backup -v5 test@server.example.com::/var/tmp/backmeup /var/tmp/backups/server.example.com/ Using rdiff-backup version 1.2.8 Executing ssh -C test@server.example.com rdiff-backup --server <truncated for length> Starting increment operation /var/tmp/backmeup to /var/tmp/backups/server.example.com Processing changed file . Incrementing mirror file /var/tmp/backups/server.example.com Processing changed file 1.file Incrementing mirror file /var/tmp/backups/server.example.com/1.file Processing changed file 2.file Incrementing mirror file /var/tmp/backups/server.example.com/2.file Processing changed file 3.file Incrementing mirror file /var/tmp/backups/server.example.com/3.file

If we look at the backup directory the number of files has not changed, however the contents and time stamps have.

root@backup-server:# ls -la /var/tmp/backups/server.example.com/ total 52 drwxr-xr-x 3 root root 4096 May 19 13:07 . drwxr-xr-x 3 root root 4096 May 19 13:53 .. -rw-r--r-- 1 root root   76 May 19 14:10 10.file -rw-r--r-- 1 root root   98 May 19 14:16 1.file -rw-r--r-- 1 root root   98 May 19 14:16 2.file -rw-r--r-- 1 root root   98 May 19 14:16 3.file -rw-r--r-- 1 root root   73 May 19 14:10 4.file -rw-r--r-- 1 root root   73 May 19 14:10 5.file -rw-r--r-- 1 root root   73 May 19 14:10 6.file -rw-r--r-- 1 root root   73 May 19 14:10 7.file -rw-r--r-- 1 root root   73 May 19 14:10 8.file -rw-r--r-- 1 root root   73 May 19 14:10 9.file drwx------ 3 root root 4096 May 19 14:16 rdiff-backup-data

rdiff-backup will keep the current mirror unchanged and any differences will be kept in diff files within the rdiff-backup-data directory. It is not advised to modify or interact with the mirror or diff files directly, it is better to use the rdiff-backup command itself.

Listing available backups

To see the available backups we can use rdiff-backup -l.

root@backup-server:# rdiff-backup -l /var/tmp/backups/server.example.com/ Found 5 increments:     increments.2013-05-19T13:56:57-07:00.dir   Sun May 19 13:56:57 2013     increments.2013-05-19T14:09:52-07:00.dir   Sun May 19 14:09:52 2013     increments.2013-05-19T14:11:29-07:00.dir   Sun May 19 14:11:29 2013     increments.2013-05-19T14:16:44-07:00.dir   Sun May 19 14:16:44 2013     increments.2013-05-19T14:29:38-07:00.dir   Sun May 19 14:29:38 2013 Current mirror: Sun May 19 14:30:20 2013

If a file has been deleted and rdiff-backup has ran since the file deletion you may not find the file in the directory, you can still however list the available backups for that file by specifying it as if it did exist.

root@backup-server:# rdiff-backup -l /var/tmp/backups/server.example.com/1.file Found 4 increments:     1.file.2013-05-19T13:56:57-07:00.diff.gz   Sun May 19 13:56:57 2013     1.file.2013-05-19T14:09:52-07:00.diff.gz   Sun May 19 14:09:52 2013     1.file.2013-05-19T14:11:29-07:00.diff.gz   Sun May 19 14:11:29 2013     1.file.2013-05-19T14:16:44-07:00.snapshot.gz   Sun May 19 14:16:44 2013 Current mirror: Sun May 19 14:30:20 2013 Restoring backed up files and directories

rdiff-backup has the ability to restore either individual files or entire directories, as long as rdiff-backup has the item within its incremental lists.

Restoring an individual file

When restoring an individual file with rdiff-backup you can either specify a time or the incremental file to restore from. For  the following example I will show using the incremental file.

root@backup-server:# cd server.example.com/rdiff-backup-data/increments/ root@backup-server:# rdiff-backup -v5 1.file.2013-05-19T14\:11\:29-07\:00.diff.gz test@server.example.com::/var/tmp/backmeup/1.file Restoring a directory

When restoring a directory however we will need to specify a specific time that we want to restore to.

root@backup-server:# rdiff-backup -v5 -r 1h server.example.com/ test@server.example.com::/var/tmp/backmeup

This command will restore the entire directory to where it was 1 hour ago or best it can depending on the backups available. rdiff-backup can support many time frames but I commonly find myself using the xDays format (e.g. 2D for 2 days).

Don’t use the force flag

While the above command will restore the whole directory it will only do so if the directory is empty. If the directory has files in it and you ask rdiff-backup to restore that directory than it will try to remove the existing files in order to match your backup. This action could result in data that has not been backed up being removed.

To protect against accidental deletion rdiff-backup requires the force flag to be used anytime a file is being overwritten or deleted.

root@backup-server:# rdiff-backup -v5 -r 1h server.example.com/ test@server.example.com::/var/tmp/backmeup Using rdiff-backup version 1.2.8 Executing ssh -C server.example.com rdiff-backup --server Fatal Error: Restore target /var/tmp/backmeup already exists, specify --force to overwrite.

I advise avoiding the use of the force flag whenever possible, if you truly do not want the contents of the directory than just remove them manually before restoring. I have seen many times where people used the force flag and accidentally overwrote a directory they did not mean (like /etc/ for example…).

Restoring to another location

When restoring with rdiff-backup you can restore files or directories to a location other than their originating source. This can be handy if you need to check the contents before completely restoring the file.

root@backup-server:# rdiff-backup -v5 -r 3h server.example.com/1.file test@server.example.com::/var/tmp/backmeup/1.file.restore Backup Retention

Backups are only as good as their retention period, without a retention period you will eventually run out of disk space or use far more disk space than you had originally planned. rdiff-backup has the ability to maintain a certain number of incremental copies. With rdiff-backup you can tell it to either keep a backup for a certain amount of time or for a certain number of backups.

On backup-server:

Time method

The time method uses the same time format as restore.

root@backup-server:# rdiff-backup --force --remove-older-than 4h /var/tmp/backups/server.example.com Number of backups method

To specify a number of backups use the number followed by a capital B.

root@backup-server:# rdiff-backup --force --remove-older-than 4B /var/tmp/backups/server.example.com

I used the force flag with the above commands as rdiff-backup requires force to be given if you are removing more than one incremental copy.

Providing more access with sudo

So far we have been backing up files and directories that the test user has access to; if we were to try and backup or restore a file that the test user does not have access to than the backup/restore will fail with a permission denied. To provide greater access you can either run rdiff-backup as the root user on the remote systems (which raises security concerns), or provide the test user with the ability to run rdiff-backup as the root user via sudo.

Example of permission denied error:

root@backup-server:# rdiff-backup -v5 test@server.example.com::/var/tmp/backmeup /var/tmp/backups/server.example.com Using rdiff-backup version 1.2.8 Executing ssh -C test@server.example.com rdiff-backup --server Exception '[Errno 13] Permission denied: '/var/tmp/backmeup'' raised of class '<type 'exceptions.OSError'>': Adding the rdiff-backup into /etc/sudoers

In order to allow the test user the ability to run rdiff-backup as root we need to add an entry into the /etc/sudoers file, which controls what commands users can run via sudo. To modify this file we will use the visudo command.

On server:

root@server:/var/tmp# visudo

Append:

## Give test user the ability to run rdiff-backup test    ALL = NOPASSWD: /usr/bin/rdiff-backup --server

As the test user you will now see rdiff-backup in the list of available sudo commands

test@server:~$ sudo -l User test may run the following commands on this host:     (root) NOPASSWD: /usr/bin/rdiff-backup --server

We are specifying NOPASSWD as by default sudo would normally ask the user for their password, which would not work very well with an automated backup script.

Running rdiff-backup with remote-schema

In order for rdiff-backup to use sudo we will need to change the command we have been using a bit; we will use the –remote-schema flag to tell rdiff-backup to run “sudo /usr/bin/rdiff-backup –server” on the remote system.

On backup-server:

Backup command

root@backup-server:# rdiff-backup -v5 --remote-schema 'ssh -C %s "sudo /usr/bin/rdiff-backup --server"' \ test@server.example.com::/var/tmp/backmeup /var/tmp/backups/server.example.com <truncated> Processing changed file 9.file Incrementing mirror file /var/tmp/backups/server.example.com/9.file

Restore command

root@backup-server:# rdiff-backup -v5 -r 3h --remote-schema 'ssh -C %s "sudo /usr/bin/rdiff-backup --server"' \ /var/tmp/backups/server.example.com/5.file test@server.example.com::/var/tmp/backmeup/5.file

By adding sudo we are allowing the test user to backup and restore any file on the system with rdiff-backup.

Adding restrict-read-only for even more security

While using rdiff-backup with sudo prevents people from using the SSH key to login as root to all of our remote systems. This solution by itself does not restrict someone from using rdiff-backups restore function from deploying compromised files.

For even more security we can use the –restrict-read-only flag to restrict rdiff-backup to only being able to read files and blocking all write requests. The down side of this setting is that it also prevents valid restore requests as well. If you are more worried about someone accessing your systems than having to edit the sudoers file every time you want to restore a file; than this is a good option.

Adding restrict-read-only to the sudoers entry

In order to add –restrict-read-only we need to add it to both the rdiff-backup command and the sudoers entry.

root@server# visudo

Modify to:

test    ALL = NOPASSWD: /usr/bin/rdiff-backup --server --restrict-read-only /

The / at the end is the path that you want rdiff-backup to be restricted to. This entry would give rdiff-backup the ability to backup all files on the system. If you are not backing up the entire system you can restrict this to a specific path as well to prevent rdiff-backup from reading other files on the system not within your path.

Running the backup command with restrict-read-only

Now that sudo allows us to run the full command we can add it to the remote-schema.

root@backup-server:# rdiff-backup -v5 --remote-schema 'ssh -C %s "sudo /usr/bin/rdiff-backup --server --restrict-read-only /"' \ test@server.example.com::/var/tmp/backmeup /var/tmp/backups/server.example.com Using rdiff-backup version 1.2.8 Executing ssh -C test@server.example.com "sudo /usr/bin/rdiff-backup --server"

If you modified the path in the sudoers file you would need to do the same with the rdiff-backup command above.

Automating with Cron

Automating rdiff-backup with cron is as simple as tossing the commands above into a script and adding it to the crontab. The below is meant only for example, I would advise anyone reading this to script in some more intelligence to handle failed backups and concurrent runs but if you needed something quick and dirty this would work.

On backup-server:

Creating the backup script root@backup-server# vi /root/backup-example.sh

Add:

#!/bin/bash ## Example rdiff-backup script - http://bencane.com ## This is not fancy, and you should really add error checking # Backup rdiff-backup -v5 --remote-schema 'ssh -C %s "sudo /usr/bin/rdiff-backup --server --restrict-read-only /"' \  test@server.example.com::/var/tmp/backmeup /var/tmp/backups/server.example.com # Clean Increments rdiff-backup --force --remove-older-than 4B /var/tmp/backups/server.example.com Adding to crontab

Once you have the script you can simply add the script into the crontab on the backup-server.

root@backup-server# crontab -e

Append:

# m h  dom mon dow   command 0 0 * * * /root/backup-example.sh > /dev/null 2>&1

The above crontab entry will run backup-example.sh every night at midnight. This will provide you with 4 days of incremental copies at all times.

Tags: backup package, backup server, backup tool, linux, rdiff-backup, red hat, sudo, ubuntu

Background

People that use Linux on a daily basis probably are completely oblivious to the actual mechanisms being used to store their passwords safely and securely on a given Linux system. Oh they might guess that their password is stored in the /etc/passwd file (they’d be wrong by the way) but most probably never even gave it a passing thought. So I thought I’d take the opportunity to shed some light on how Linux systems “stash” your precious password away.

Solution

So if your password isn’t actually stored in the /etc/passwd file then where does it get stored?

Answer: the /etc/shadow file.

This file is where all the keys to each user’s account are kept for safe keeping. Obviously only the root user can peer inside this file so all the commands we’ll be dealing with in this post, it should be assumed that you’ll need to either be root, or use sudo to run.

/etc/shadow

A typical /etc/shadow entry:

1 root:$6$bbmDJwcZHy5bgEDz$kFO.W/T7nUqcszZWl5RglxoDDAcDxevWpHVfN3v3f.Cx2ZeMcn5PX23VvnnkgtNWZf8hYtqsL0pPkZqyj50NY/:14362:0:33333:7:::

NOTE1: Don’t get too excited, the above isn’t really my entry, I made this one up.
NOTE2: Each field is separated by a colon (:) & we’re only concerned with the first two columns!

dissecting the hash

The key pieces to notice in that line of what looks like gibberish is the following:

• The first column, root is the user whom this entry belongs to from the /etc/passwd file.
• The second column, $6$..... is essentially the user’s hashed password.

Taking the second column apart further you should start to notice that’s it’s not complete gibberish after all.

For example:

• the first couple of characters, $6$, is a mark that tells the system what type of hashing was used to hash the password.
• The text between the next set of dollar signs, $bbmDJwcZHy5bgEDz$, is the actual salt that was used to hash your password.
• Everything else after, is your password + salt hashed using whatever hash function was specified at the beginning, $6$, in our example here.

Specifically if you look at the man page for the crypt command, man 3 crypt there is a section that discusses what the $6$ notation means:

So $5$salt$encrypted is an SHA-256 encoded password and $6$salt$encrypted is an SHA-512 encoded one.

NOTE: So in our case the password + salt is being hashed using the SHA-512 scheme.

design details

For reference purposes here’s the rest of that excerpt from the crypt man page:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 If salt is a character string starting with the characters "$id$" followed by a string terminated by "$":   $id$salt$encrypted   then instead of using the DES machine, id identifies the encryption method used and this then determines how the rest of the password string is interpreted. The following values of id are supported:   ID | Method ───────────────────────────────────────────────────────── 1 | MD5 2a | Blowfish (not in mainline glibc; added in some | Linux distributions) 5 | SHA-256 (since glibc 2.7) 6 | SHA-512 (since glibc 2.7)   So $5$salt$encrypted is an SHA-256 encoded password and $6$salt$encrypted is an SHA-512 encoded one.   "salt" stands for the up to 16 characters following "$id$" in the salt. The encrypted part of the password string is the actual computed password. The size of this string is fixed:   MD5 | 22 characters SHA-256 | 43 characters SHA-512 | 86 characters   The characters in "salt" and "encrypted" are drawn from the set [a–zA–Z0–9./]. In the MD5 and SHA implementations the entire key is significant (instead of only the first 8 bytes in DES). Now what?

So by now you’re probably saying to yourself. OK, big deal, my password is hashed with some salt and stored in /etc/shadow. What else?

generating the hash manually using mkpasswd

For starters you can generate the $6$... string yourself manually using the mkpasswd command:

1 2 $ mkpasswd -m sha-512 password saltsalt $6$saltsalt$qFmFH.bQmmtXzyBY0s9v7Oicd2z4XSIecDzlB5KiA2/jctKu9YterLp8wwnSq.qc.eoxqOmSuNp2xS0ktL3nh/

In the above command we’re specifying that we want to use the SHA-512 hash, our password is the string password and our salt string is saltsalt. As before we can see in our resulting string the following components:

• $6$ – which hash function was used
• saltsalt – the string “saltsalt” was used
• qFmFH.bQmmtXzyBY0s9v7Oicd2z4XSIecDzlB5KiA2/jctKu9YterLp8wwnSq.qc.eoxqOmSuNp2xS0ktL3nh/ – password + salt hashed using SHA-512

generating the hash manually using Python

I came across the following nice Python one-liner that effectively does the same thing as the mkpasswd command discussed above.

1 2 3 $ python -c "import crypt, getpass, pwd; \ print crypt.crypt('password', '\$6\$saltsalt\$')" $6$saltsalt$qFmFH.bQmmtXzyBY0s9v7Oicd2z4XSIecDzlB5KiA2/jctKu9YterLp8wwnSq.qc.eoxqOmSuNp2xS0ktL3nh/ generating the hash manually using Perl 1 2 $ perl -e 'print crypt("password","\$6\$saltsalt\$") . "\n"' $6$saltsalt$qFmFH.bQmmtXzyBY0s9v7Oicd2z4XSIecDzlB5KiA2/jctKu9YterLp8wwnSq.qc.eoxqOmSuNp2xS0ktL3nh/ authconfig

Before I wrap up I thought I’d mention one final tool authconfig that’s included on Red Hat distros such as Fedora, CentOS, and RHEL. This tool allows you to change what hash algorithm is being used on a particular system. The command to change a system to use SHA-512 would be as follows:

1 authconfig –passalgo sha512 –update

See the man page for authconfig for more details.

conclusions

And with that you are now a little more in the know as to how Linux systems take your password and store them in the /etc/shadow file.

References links

• crypt man page
• Thread: generate hashed password for /etc/shadow – ubuntuforums.org
• How to produce test hashes for various formats
• Shadow Fu
• Using ruby to generate SHA512 crypt-style hashes formatted for /etc/shadow?
• Calculate SHA Hash of Linux Password in Shadow File

When a variety of ISPs and services filter out most of it, you only get what slips through the net. I've noticed something that made me appreciate the scale of spam. My blog is tiny. I get perhaps an average of 100 views to any post that I make. I rarely get comments, although I think that's normal for most blogs now, even those much more established and more popular than mine.

I use Drupal for my blog, with the Mollom module to deal with attempted spam in an intelligent way. Every time I check for comments held in moderation, it's often at 0, or the odd genuine comment. Every so often a spam comment gets through to there, but it's mostly empty.

I took a look at the events logs, in particular the Mollom logs for the last 3 days. I think each page shows about 50 entries, I've not counted them. Over the last week or so I've been experimenting with clearing the logs, and checking later to see how much it filled up and how fast. I was stunned.

In one 24hr period, it can often fill up 4 pages of log entries. That's 200 failed spam attempts in one single 24hr period. In many of these cases it's a Gmail account blasting through a batch of maybe 10 attempts in a minute, then again an hour later.

Apart from the odd comment, none of this is getting through. My real amazement was in just how much of this I was getting, and mainly from the angle of "this blog is a nothing blog, from a random Joe on the internet". I can't imagine the amount a household name site would get. It also gave me a new appreciation of just how much spam is filtered out before we even see it.

To all the organisations around the world who help keep our spam to a minimum, I humbly thank you.

Tags: Spam

I came across this question on the Stack Exchange site Unix & Linux. The question interested me so I answered it but thought I’d cross post it on my blog as well, given I took a pretty significant amount of time to put together a test case and write-up of how the solution ultimately worked.

Problem

I’m using rsync to copy some files from a share to another.

Recursively, I need to:

- delete files at the destination that are deleted in the origin
- Only sync php and js files
- exclude de rest of file types
- Don’t delete .svn/ directory in the destination

If I use this:

rsync -zavC --delete --include='*.php' --include='*.js' --exclude="*" /media/datacod/Test/ /home/lucas/Desktop/rsync/

Then rsync is not recursive because exclude=”*” excludes all files but also folders

If I add --include="*/" then the .svn/ directory gets deleted (it also gets included)

How can I solve this mind blasting dilemma?

Solution

The solution I ultimately came up with made use of a little known feature, at least to me, called filters. Filters allow you to play games with the includes/excludes by protecting portions based on regular expressions. Read on, I’ll discuss them further down.

1 2 rsync -avzC --filter='-rs_*/.svn*' --include="*/" --include='*.js' --include='*.php' \ --exclude="*" --delete dir1/ dir2/ test data

To help determine if my solution was going to work or not I created some sample data so that I could test it out. For starters I wrote a script that would generate the data. Here’s that script, setup_svn_sample.bash:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 #!/bin/bash   # setup .svn dirs mkdir -p dir{1,2}/dir{1,2,3,4}/.svn   # fake data under .svn mkdir -p dir1/dir{1,2,3,4}/.svn/origdir mkdir -p dir2/dir{1,2,3,4}/.svn/keepdir   # files to not sync touch dir1/dir{1,2,3,4}/file{1,2}   # files to sync touch dir1/dir{1,2,3,4}/file1.js touch dir1/dir{1,2,3,4}/file1.php

Running the above script produces the following directories (dir1 & dir2):

source dir

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 $ tree -a dir1 dir1 |-- dir1 | |-- file1 | |-- file1.js | |-- file1.php | |-- file2 | `-- .svn | `-- origdir |-- dir2 | |-- file1 | |-- file1.js | |-- file1.php | |-- file2 | `-- .svn | `-- origdir |-- dir3 | |-- file1 | |-- file1.js | |-- file1.php | |-- file2 | `-- .svn | `-- origdir `-- dir4 |-- file1 |-- file1.js |-- file1.php |-- file2 `-- .svn `-- origdir

destination dir

1 2 3 4 5 6 7 8 9 10 11 12 13 14 $ tree -a dir2 dir2 |-- dir1 | `-- .svn | `-- keepdir |-- dir2 | `-- .svn | `-- keepdir |-- dir3 | `-- .svn | `-- keepdir `-- dir4 `-- .svn `-- keepdir

Running the above rsync command which includes the --filter below we can see that it’s only syncing the files that match the --include patterns:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 rsync -avzC --filter='-rs_*/.svn*' --include="*/" --include='*.js' --include='*.php' \ --exclude="*" --delete dir1/ dir2/ sending incremental file list dir1/file1.js dir1/file1.php dir2/file1.js dir2/file1.php dir3/file1.js dir3/file1.php dir4/file1.js dir4/file1.php   sent 480 bytes received 168 bytes 1296.00 bytes/sec total size is 0 speedup is 0.00

Resulting dir2 afterwards:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 $ tree -a dir2 dir2 |-- dir1 | |-- file1.js | |-- file1.php | `-- .svn | `-- keepdir |-- dir2 | |-- file1.js | |-- file1.php | `-- .svn | `-- keepdir |-- dir3 | |-- file1.js | |-- file1.php | `-- .svn | `-- keepdir `-- dir4 |-- file1.js |-- file1.php `-- .svn `-- keepdir Why does it work?

The key piece to this script is to make use of the filters capability of rsync. Filters allow you to remove files from the matched set at various points in the command. So in our case we’re filtering any files that match the pattern */.svn*. The modifiers -rs_ tell the filter that we want to filter on both the source side as well as the target side.

excerpt from the FILTER NOTES section of rsync’s man page

- An s is used to indicate that the rule applies to the sending side. When a rule affects the sending side, it prevents files from being
transferred. The default is for a rule to affect both sides unless --delete-excluded was specified, in which case default rules become sender-side only. See also the hide (H) and show (S) rules, which are an alternate way to specify sending-side includes/excludes.

- An r is used to indicate that the rule applies to the receiving side. When a rule affects the receiving side, it prevents files from being deleted. See the s modifier for more info. See also the protect (P) and risk ® rules, which are an alternate way to specify receiver-side includes/excludes.

See man rsync for more details.

Tips for figuring this out (hint using --dry-run)

While describing how to do this I thought I’d mention the --dry-run switch to rsync. It’ extremely useful in seeing what will happen without having the rsync actually take place.

For Example

Using the following command will do a test run and show us the decision logic behind rsync:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 rsync --dry-run -avvzC --filter='-rs_*/.svn*' --include="*/" \ --include='*.js' --include='*.php' --exclude="*" --delete dir1/ dir2/ sending incremental file list [sender] showing directory dir3 because of pattern */ [sender] showing directory dir2 because of pattern */ [sender] showing directory dir4 because of pattern */ [sender] showing directory dir1 because of pattern */ [sender] hiding file dir1/file1 because of pattern * [sender] showing file dir1/file1.js because of pattern *.js [sender] hiding file dir1/file2 because of pattern * [sender] showing file dir1/file1.php because of pattern *.php [sender] hiding directory dir1/.svn because of pattern */.svn* [sender] hiding file dir2/file1 because of pattern * [sender] showing file dir2/file1.js because of pattern *.js [sender] hiding file dir2/file2 because of pattern * [sender] showing file dir2/file1.php because of pattern *.php [sender] hiding directory dir2/.svn because of pattern */.svn* [sender] hiding file dir3/file1 because of pattern * [sender] showing file dir3/file1.js because of pattern *.js [sender] hiding file dir3/file2 because of pattern * [sender] showing file dir3/file1.php because of pattern *.php [sender] hiding directory dir3/.svn because of pattern */.svn* [sender] hiding file dir4/file1 because of pattern * [sender] showing file dir4/file1.js because of pattern *.js [sender] hiding file dir4/file2 because of pattern * [sender] showing file dir4/file1.php because of pattern *.php [sender] hiding directory dir4/.svn because of pattern */.svn* delta-transmission disabled for local transfer or --whole-file [generator] risking directory dir3 because of pattern */ [generator] risking directory dir2 because of pattern */ [generator] risking directory dir4 because of pattern */ [generator] risking directory dir1 because of pattern */ [generator] protecting directory dir1/.svn because of pattern */.svn* dir1/file1.js dir1/file1.php [generator] protecting directory dir2/.svn because of pattern */.svn* dir2/file1.js dir2/file1.php [generator] protecting directory dir3/.svn because of pattern */.svn* dir3/file1.js dir3/file1.php [generator] protecting directory dir4/.svn because of pattern */.svn* dir4/file1.js dir4/file1.php total: matches=0 hash_hits=0 false_alarms=0 data=0   sent 231 bytes received 55 bytes 572.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN)

In the above output you can see that the ./svn directories are being protected by our filter rule. Valuable insight for debugging the rsync.

References

- Delete extraneous files from dest dir via rsync?
- Above scripts in a tarball

Readers who viewed this page, also viewed:

• Formatting text using Textile
• Powered by Where did they go from here?

You go to open a WMV video file in Nautilus on a new GNOME install, GNOME Video opens, and you are immediately greeted with: "Advanced Streaming Format (ASF) demuxer".  A message that is terribly helpfully, but at least it offers to "Search" for a solution. The auto-find-the-solution button works sometimes, but not always.  An easier solution is to preemptively install the required packages. 
zypper ar http://ftp.gwdg.de/pub/linux/packman/suse/openSUSE_12.3 packman
zypper in gstreamer-plugins-bad-orig-addon gstreamer-plugins-libav
gstreamer-plugins-ugly-orig-addon w32codec-all
Add the Packman repository and install the required packages.The package install may prompt you to allow a vendor change (from the 'official' openSUSE repository to the Packman repository).  This vendor change is desired so it should be allowed.  Once the packages are installed GNOME Video player should play WMV files which use ASF multiplexing without further complaint.
Aside:  If you don't want to be pestered about package vendor changes in the future you can edit  /etc/zypp/zypp.conf and set "solver.allowVendorChange = true".  But don't do that unless you know what that means.

The Syrian uprising has ballooned into a catasrophe on many levels after several years of fighting and it shows no signs of abating. Let's explore some of the options available to us:

- attempt to negotiate a ceasefire. It's clear that this is unlikely to hold though. It also feels as though a lot of previous attempts have been disengenuous or have been used to stall, seeking better terms, etc... Believe that only if there is greater force applied will be hold (more on this later).
http://www.foxnews.com/world/2013/05/14/syria-wants-details-about-us-russian-initiative-before-deciding-whether-to/
http://www.nytimes.com/2013/05/15/world/middleeast/syria-developments.html?_r=0
http://english.alarabiya.net/en/views/news/middle-east/2013/05/15/Is-it-a-peace-or-war-plan-for-Syria-.html
http://www.un.org/News/dh/infocus/Syria/FinalCommuniqueActionGroupforSyria.pdf
- increased support for the rebellion. It's not entirely clear just what exactly we're supporting here (if concerned about longer term spread of weapons from conflict develop/consider stronger but limited lifetime weapons). It was previously a peaceful uprising but it has since turned into violence with the problem excerbated by foreign combantants and groups who share links with terrorist groups and have other interests besides that of the Syrian people. Violations and various atrocities (from both sides) need to be dealt with as well.
http://www.theaustralian.com.au/news/breaking-news/syria-oppn-condemns-heart-eating-video/story-fn3dxix6-1226642510509http://worldnews.nbcnews.com/_news/2013/05/14/18244907-sheer-savagery-syrian-rebel-rips-out-soldiers-heart-human-rights-watch-says?lite
http://www.dailystar.com.lb/News/Middle-East/2013/May-15/217157-syria-rebels-vow-to-punish-those-committing-atrocities.ashx http://www.guardian.co.uk/world/2013/apr/28/syrian-nerve-gas-claims-eyewitness
http://original.antiwar.com/srichman/2013/05/14/no-intervention-in-syria/
- direct and full intervention/invasion. We've seen Iraq/Afghanistan weren't clear cut and this one is probably going to be just as difficult if more so (how could we possibly make it any worse than it currently is?). It's also becoming clear that surrounding countries are already getting dragged in with regards to both the humantarian problem as well as the conflict itself with many of them being used as launchpads or support for military action in Syria itself. Invasion should be considered an option but only if all other options have been exhausted and have been proven to be unworthwhile.http://www.usnews.com/opinion/blogs/world-report/2013/05/15/turkey-hopes-to-convince-us-to-act-in-syria
- de-militarise the conflict. This means that no more (ANY) weapons whether are to be supplied to either side whether that means re-supply, fulfilling existing contracts, etc...
http://world.time.com/2013/05/14/putin-netanyahu-meet-to-discuss-syria/
Hopefully, this will also make both sides more amenable to genuine peace talks (clearly, will not work if one side continues to arm though).
- direct but limited intervention. One option that I've been considering is destroying all air-fields/military bases/large clusters of heavy weapons/artillery/munitions and so on, shutting down all borders inbound to Syria (not easy). This will result in a stalemate situation (especially if the neither side are continued to be supplied with weapons).
http://www.washingtonpost.com/blogs/worldviews/wp/2013/05/13/six-ways-assad-has-turned-the-tide-in-syria/
http://www.npr.org/2013/04/30/179855633/c-j-chivers-on-the-ground-in-syria
Hopefully, this will also make both sides more amenable to genuine peace talks (clearly, will not work if one side continues to arm though). Another option that has been widely considered is targeted, direct action against regime leadership. There will of course be repercussions should this avenue be pursued...
- a pure peace keeping intervention? Long range strikes (as outlined in previous point) combined with an international, armed peace keeping ground force (rules of engagement mean that they their primary job will be to defend non combatants, themselves, and finally to maintain peace)? Peace keeping force must have clear agenda and provide prior warning. If there is any untoward activity they have a go ahead to use force to stop it whether that pertains to rebel or regime activity. It can not be stressed enough that this peace keeping force is not about joining in the conflict. It is about stopping it and getting back to normality as quickly as possible. Obvious problem is whether or not the fighting will simply start up again the minute the peace keeping force leaves?
- let them continue to fight it out until it's conclusion. Cynical but it also means that one side is likely to be a more complete victor which may result in a more stable long term situation.
- offer the current regime safe passage out. Unlikely to be accepted given some of the messages that have been sent out.
- don't bother trying to implement a ceasefire prior to creating a transition plan or running an election? If both sides can just maintain peace on their side of the conflict (clear lines of demarcation and buffer zones so that we can minimise break outs of fighting) while elections (obvious problems here especially vote those relating to 'tampering') are running perhaps we can figure out just exactly what the Syrian people actually want (this will also mean that we can disavow everyone of all possible doubt over what the desire of the actual Syrian population is). Who's in charge of running election? A combination of existing regime/rebels/neighbours with international observers? How can you when so many people are displaced (people in refugee camps in particular)? Require identification for them to participate while existing people can simply show up at polling booths. How much will displaced people skew the results of any potential election. Obvious questions are, whether they want existing regime or rebels to succeed? What should be the timeline going forward? How is normal life going to be restored? etc...
- break up of the country should be considered if it means a cessation of hostilities in spite of warnings.
http://www.presstv.ir/detail/2013/05/14/303424/iran-warns-against-syria-disintegration/
- half baked measures so far have proven unlikely to turn the tide. If there is intervention (in any form whether diplomatic, military, etc...) there must be far greater force behind it to simply get it over and done with so that everyone can get on with their lives.
- don't go into talks with any pre-conditions. Push hard but give peace a genuine chance. Not sure how some people can be so optimistic that UN June 12 plan has a genuine chance given the fact that the conflict has continued unabated and esclated for several years (I've said before and I'll say it again defense, intelligence, and defense should work together and only be pressing harder will be able to force a cessation of hostilities.).
http://english.alarabiya.net/en/perspective/analysis/2013/05/14/Arabs-Turkey-see-no-role-for-Assad-in-future-Syria-.html
http://www.un.org/News/dh/infocus/Syria/FinalCommuniqueActionGroupforSyria.pdf
- provide flares and other camouflage options because it's clear that most of the weapons involved are fairly simple/non-guidance based. Likelihood that they will resort to carpet/cluster bombing even though they are already using makeshift weapons?

Key questions/issues:
- can you honestly say that Assad is fit and do the Syrian people want him to lead Syria?
- if there is intervention and there is a power vacuum is this worse than what would occur if we didn't intervene?
- the style/size of the intervention. Direct, continued covert, etc...
- even if we aren't directly involved what are the indirect impacts of continued conflict in Syria?
- will any leadership be better/worse than the previous one?
- what other moves are other stakeholders likely to make should further direct/indirect action occur?
- even if there is a transition is it going to be representative and will it hold?
http://au.news.yahoo.com/world/a/-/world/17146065/france-sees-snags-in-plans-for-syria-peace-talks/
http://www.naharnet.com/stories/en/82916-france-warns-syria-conference-will-be-very-difficult
http://www.globalresearch.ca/obama-cameron-hold-syria-war-summit-in-washington-more-weapons-for-al-qaeda/5334993
- if there is intervention does the International community support or lead?
- are current peace talk offers genuine?
- limited public support/appetite for intervention.
- the longer the fight goes on the more desperate people have become. Concern is that either solution breaks down because new leadership may be just as bad or worse than previously or else it breaks down simply because they aren't strong enough to deal with the issues that continue to stem from this conflict.
- is this a situation that needs to be 'managed' because it can't be fixed completely in future without long term commitment?
- something which needs to be kept in mind is that many International bodies need reform or are simply losing their relevance. I think that the after several recent incidents the United Nations is beginning to fall into this category as well. In which case, I think the question we should all be asking ourselves is whether some  the power plays that are occurring are really worth it. At some point this isn't a question of interests, it's a question of humanity. It's a question of being able to distinguish between right and wrong, between human and primitive animal. If the United Nations doesn't give us the ability to do what is required, what is right in order to end this situation then the International community must surely see fit to either change the existing frameworks stopping us from doing so or find a way of working around them.
http://fullcomment.nationalpost.com/2013/05/13/jonathan-kay-forget-red-lines-for-assad-its-time-to-start-saving-innocent-syrian-civilians/

http://www.un.org/News/dh/infocus/Syria/FinalCommuniqueActionGroupforSyria.pdf
http://abcnews.go.com/US/wireStory/assembly-expected-approve-syria-resolution-19181298
http://blogs.reuters.com/great-debate/2013/05/14/learning-the-wrong-lessons-from-israels-intervention-in-syria/
http://www.irishtimes.com/news/world/middle-east/no-fly-zone-is-best-of-bad-options-for-syria-1.1393250
http://www.news.com.au/world-news/australian-aid-may-be-propping-up-syrian-regime/story-fndir2ev-1226642141799
http://www.japantimes.co.jp/news/2013/05/15/world/syria-forum-prompts-guarded-optimism/
http://www.guardian.co.uk/commentisfree/2013/may/13/syria-post-superpower-era-obama-indecision
http://www.washingtonpost.com/world/assad-forces-gaining-ground-in-syria/2013/05/11/79147c34-b99c-11e2-b568-6917f6ac6d9d_story.html
http://news.xinhuanet.com/english/world/2013-05/14/c_132379592.htm
http://www.nytimes.com/2013/04/30/opinion/ill-considered-advice-on-syria.html?_r=0
http://www.washingtonpost.com/world/national-security/iraq-history-at-bush-center-shows-need-for-caution-on-syria/2013/04/29/ea124816-ae80-11e2-98ef-d1072ed3cc27_story.html
http://www.pbs.org/newshour/bb/world/jan-june13/syria2_04-29.html
http://www.wired.com/dangerroom/2013/05/syria-weapons-2/
http://www.globalresearch.ca/obama-cameron-hold-syria-war-summit-in-washington-more-weapons-for-al-qaeda/5334993
http://www.nytimes.com/2013/05/15/world/middleeast/syria-developments.html?_r=0
http://www.bangkokpost.com/news/world/349997/putin-netanyahu-set-for-talks-on-syria
http://www.thehindu.com/opinion/op-ed/russia-plays-the-missile-card/article4712306.ece

Linux Mint

Yes, you heard it right! Get your backups up to date and your gear ready for the next release of the best Linux distribution available. Grab it here:

Dear All, It is not possible that on every Single System there is mail server available. There is no mail server configured on your System and You want to send e-mail using SMTP Authentication then you can send mail by ssmtp utility. You can use the Same Method for RHEL / CentOS / Fedora / [...]

Background

UEFI looks to be a major pain in the @$$, but like it or hate it everyone in the Linux community will need to learn to navigate it. Here’s a list of useful UEFI resources that I’ve come across as I’ve started to get smarter about how to deal with this beast.

Solution

Wikipedia

• Unified Extensible Firmware Interface

Ubuntu Docs

• UEFI
• UEFIBooting – Booting Linux natively on a UEFI system (without BIOS CSM) using GRUB2

AskUbuntu

• Install Ubuntu next to Windows 8
• Installing Ubuntu on a Pre-Installed UEFI Supported Windows 8 system

Rodsbooks.com

• Managing EFI Boot Loaders for Linux
• Managing EFI Boot Loaders for Linux: Using GRUB 2
• Managing EFI Boot Loaders for Linux: Dealing with Secure Boot

Misc.

• Running RHEL 6 on EFI BIOS-based system

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

Normally on this blog I tend to write about more complicated tasks or fancy Linux tricks and completely overlook some of the most basic tasks that a SysAdmin needs to know. Today I have decided that I will make my blog a little more comprehensive and add some posts with some of the basics.

Along with this I will be starting a new category, called Sysadmin Basics and I will try to post an additional article each week that covers some of the more basic concepts and commands used by Linux and Unix Sysadmins.

Remove Directories with the rmdir command

The rmdir command is used to delete and remove empty directories. I bolded empty as it is important to note that rmdir will only remove a directory if there are no files within that directory. If you want to remove a directory and all files within that directory, skip down to the rm section of this article.

Remove a single empty directory # rmdir somedir/ Remove multiple empty directories (in a single tree) # rmdir -p somedir/a/b/c/d/e/f/whoa

While rmdir will not remove directories with files in it; rmdir will recursively remove a directory tree that has no files. In the example somedir only has directory a within it, and the a directory only has b which only has c and so on.

Remove multiple empty directories

The above command will also fail if there are multiple directories in one single directory, to handle that scenario you can list the directories individually and include the –ignore-fail-on-non-empty flag.

# rmdir --ignore-fail-on-non-empty -p somedir/a/b/c/ somedir/a2/b2/

Without the –ignore-fail-on-non-empty flag the command will still print that somedir is not empty even though it removes somedir. This is due to the fact that both command line arguments ask rmdir to remove somedir and rmdir cannot remove that directory until the last step.

Removing Files and Directories with the rm Command

While the rmdir command is solely for directories the rm command can remove both files and directories. With the right combination of flags rm will also remove entire directories, files and all.

Remove a file # rm a-file rm: remove regular empty file `a-file'? y

On it’s own rm will not prompt a user before removing a file; to keep systems safe from accidental file removals some distributions of Linux will ship with an alias for rm with the default .bashrc file. This alias gives the interactive (-i) flag for rm, this tells rm to prompt the user before removing files and directories.

# alias alias rm='rm -i' Remove a file without being prompted

While you can simply unalias the rm alias, a simplier and generally used method to remove files without being prompted is to add the force (-f) flag to the rm command. It is advisable that you only add the force (-f) flag if you really know what you are removing.

# rm -f b-file Remove a file without being prompted and with verbosity

If you don’t want to be prompted for each file removable but also want to keep an eye on rm in case the command starts removing unexpected files, you can simply add the verbose (-v) flag.

# rm -fv c-file removed `c-file' Remove multiple files

There are many ways to remove multiple files, one method is to simply list each file you want to remove.

# rm -f a-file b-file Removing multiple files with a wildcard

The bash command line supports wildcards and regex statements. A simplier way to remove all files that end in the word file is to simply state *file. I suggest being cautious with wildcards as it is entirely possible to remove a file without meaning to.

# rm -f *file Remove files using a regex

Another common method of deleting files is to use regex statement, the below would remove anything that looks like files-0 through files-9 but would not remove files-a or files-list.

# rm -f files-[0-9] Remove a directory and all of it’s contents with rm

If you want to simply remove an entire directory and all of the contents within, including both files and directories the easiest method is to add the recursive (-R) flag to rm. If you are in any way unsure of what you are doing than drop the force (-f) and replace it with verbose (-v) or interactive (-i).

# rm -Rf somedir/ Tags: bash, empty directories, linux, rm command, rmdir command, sysadmin, unix

Adding static routes in Linux can be troublesome, but also absolutely necessary depending on your network configuration. I call static routes troublesome because they can often be the cause of long troubleshooting sessions wondering why one server can’t connect to another.

This is especially true when dealing with teams that may not fully understand or know the remote servers IP configuration.

The Default Route

Linux, like any other OS has a routing table that determines what is the next hop for every packet.

Print the routing table contents

There are numerous commands that show the routing table but today we will use the ip command as this command will be replacing the route command in future releases.

# ip route show 10.1.6.0/26 dev eth0 proto kernel scope link src 10.1.6.21 10.1.7.0/24 dev eth1 proto kernel scope link src 10.1.7.41 default via 10.1.6.1 dev eth0

As you can see in the example routing table there are numerous routes however 1 route shows as the default route. This routing table tells the system that if the IP that is being communicated to does not fall into any of the other routes than send the packets to the default route defined as 10.1.6.1. The default route basically acts as a catchall for any packet that isn’t being told what to do in the above routes.

Our Example System

In today’s article I will be referencing an example network configuration in order to show how static routes are added, why to add them and some basic troubleshooting.

Example Interface Configuration

eth0:

# cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 BOOTPROTO=static IPADDR=10.1.6.21 NETMASK=255.255.255.192 ONBOOT=yes

eth1:

# cat /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 BOOTPROTO=static IPADDR=10.1.7.41 NETMASK=255.255.255.0 ONBOOT=yes Example Default Route Configuration # cat /etc/sysconfig/network NETWORKING=yes HOSTNAME=testing.example.com GATEWAY=10.1.6.1

The GATEWAY configuration in /etc/sysconfig/network tells the system that 10.1.6.1 is the default route. This configuration could also be added to /etc/sysconfig/network-scripts/ifcfg-eth0 file; However if multiple ifcfg-<interface> files have a GATEWAY this may provide unexpected results as there can only be one default route.

Example Why we need a static route

For our example network configuration we have two interfaces; eth0 (10.1.6.21) for the internet, and eth1 (10.1.7.41) for the internal network. If we were to hook up to a backup server such as 10.1.5.202 we would want the connectivity to go through eth1 the internal network, rather than eth0 which is the internet network.

Since 10.1.5.202 is not in the same subnet at eth1 (10.1.7.0/24) the routing table does not automatically route the packet through eth1 and would then hit the “catchall” default route out eth0. To force all of our packets destined for 10.1.5.202 out eth1 we will need to set up a static route.

Adding a Static Route Adding the route to the current routing table

Adding the static route is a fairly simple task however before we start we must first know the gateway for the internal network; for our example the gateway is 10.1.7.1.

Adding a single IP # ip route add 10.1.5.202/32 via 10.1.7.1 dev eth1

The above command adds a route that tells the system to send all packets for 10.1.5.202 and only that IP to 10.1.7.1 from device eth1.

Adding a subnet of IP’s

In order to add a whole subnet than you will need to change the CIDR on the end of the IP. In this case I want to add anything in the 10.1.5.0 – 10.1.5.255 IP range. To do that I can specify the netmask of 255.255.255.0 in CIDR format (/24) at the end of the IP itself.

If a CIDR (or netmask) is not specified the route will default to a /32 (single ip) route.

# ip route add 10.1.5.0/24 via 10.1.7.1 dev eth1

The difference between these two routes is that the second will route anything between 10.1.5.0 and 10.1.5.255 out eth1 with 1 route command. This is useful if you need to communicate with multiple servers in a network and don’t want to manage lengthy routing tables.

Adding the route even after a network restart

While the commands above added the static route they are only in the routing table until either the server or network service is restarted. In order to add the route permanently the route can be added to the route-<interface> file.

# vi /etc/sysconfig/network-scripts/route-eth1

Append:

10.1.5.0/24 via 10.1.7.1 dev eth1

If the above configuration file does not already exist than simply create it and put only the route itself in the file (# comments are ok). When the interface is restarted next the system will add any valid route in the route-eth1 file to the routing table.

I highly suggest that when possible anytime you add a route to the route-<interface> files that the interface itself is restarted to validate whether the route is actually in place correctly or not. I have been on many late night calls where a static route was not added correctly to the configuration files and was removed on the next reboot, which is also long after everyone has forgotten that a static route was required.

Troubleshooting a Static Route Check if the route is in the routing table

Before performing any deep down troubleshooting steps the easiest and first step should be to check if the routing table actually has the route you expect it to have.

# ip route show 10.1.5.0/24 via 10.1.7.1 dev eth1 10.1.6.0/26 dev eth0 proto kernel scope link src 10.1.6.21 10.1.7.0/24 dev eth1 proto kernel scope link src 10.1.7.41 default via 10.1.6.1 dev eth0 Use tcpdump to see tcp/ip communication

The easiest way that I have found to find out whether a static route is working correctly or not is to use tcpdump to look at the network communication. In our example above we were attempting to communicate to 10.1.5.202 through device eth1.

# tcpdump -qnnvvv -i eth1 host 10.1.5.202 tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 16:50:35.880941 IP (tos 0x10, ttl 64, id 59563, offset 0, flags [DF], proto: TCP (6), length: 60) 10.1.7.41.41403 > 10.1.5.202.22: tcp 0 16:50:35.881266 IP (tos 0x0, ttl 59, id 0, offset 0, flags [DF], proto: TCP (6), length: 60) 10.1.5.202.22 > 10.1.7.41.41403: tcp 0

The above tcpdump command will only listen on eth1 and output only results that to or from 10.1.5.202.

TCP connections require communication from both the source and the destination, to validate a static route you can simply initiate a tcp connection (telnet to port 22 in this case) from the server with the static route to the destination server. In the output above you can see communication from 10.1.7.41 to 10.1.5.202 from the eth1 interface, this line alone shows that the static route is working correctly.

If the static route was incorrect or missing the tcpdump output would look similar to the following.

# tcpdump -qnnvvv -i eth1 host 10.1.5.202 tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 16:50:35.881266 IP (tos 0x0, ttl 59, id 0, offset 0, flags [DF], proto: TCP (6), length: 60) 10.1.5.202.22500 > 10.1.7.41.22: tcp 0

In the above, only the target server is communicating over eth1.

Tags: default route, ip command, ip route, linux, linux distributions, network, network communication, network configuration, red hat, route, routing table, static routes, troubleshooting, WAN

Hello everyone, I just made a video and uploaded on youtube. hopefully it will make it little easier to understand what conary actually can do and how it can help you if you break your system someday.

 

This is my first video and more to come, also the quality will improve with time. So will my voice. As this was on the go…..

 

Hopefully it’s still useful for some users, so please add a comment here or youtube to let me know it’s useful. Will keep my going.

 

youtube link: Foresight Linux information

Also I edited it a bit at the beginning, so it won’t show the terminal when it’s done at youtube. But might show now, as it’s newly added.

The post Foresight Linux information video on youtube appeared first on Foresight Linux.

This is a continuation of my previous post:
http://dtbnguyen.blogspot.com/2013/05/going-green.html If you haven't already figured out already I'm a big fan of using the environment to aide us. To this end I've been thinking of some changes to current technology (obviously, some ideas are extremely feasible but others are probably best left for the distant future) to allow us to live a more sustainable future:
- been thinking about better use of aerodynamics in cars. Provided the car is light enough and there is sufficient speed we may be able to make use of dynamic aerodynamic technologies. For instance, in Formula 1 teams were recently playing around with 'Drag Reduction Systems' which basically involved a opening/closing a flap on the back of the rear wing. Let's take this a step further. We have a channel at the front which runs through the middle of the car which when adjusted can send air over the back or under the back end of the back of the car (whose shape could possibly be dynamically altered at will?). In affect we have what amounts to variable aerodynamics and therefore theoretical weight). Ultimately, at higher speeds the effective weight of the car will become lighter which means increased fuel efficiency, reduced wear on car parts (remember in my re-design we would be throwing away a lot of parts as well so the effect is compounded), more than likely a car which requires less servicing, etc... Of course, a lot of tweaking may be require to ensure 'safe' levels of upforce though at all times though I suspect this may be computer controlled... Moreover, if the design already has aerodynamics in mind we may be able to retro-fit wings and turbines for air flight at some stage.
https://en.wikipedia.org/wiki/Ground_effect_vehicle
http://en.wikipedia.org/wiki/List_of_ground_effect_vehicles
Curious to know efficiency levels of flying cars? Are we ready to fly en-masse as yet?
http://www.smh.com.au/digital-life/cartech/coming-soon-your-personal-flying-car-20130509-2j8w2.html
http://en.wikipedia.org/wiki/Fuel_economy_in_aircraft
http://en.wikipedia.org/wiki/Energy_efficiency_in_transportation
- been thinking about ECU modifications further. At traffic lights, we shut down our engines so that only one cylinder is used (a button somewhere or perhaps even automated)(we shut down cylinders as opposed to entire engine to reduce wear on batteries which is a problem with some existing systems). We should also have the ability to deal up/down cylinders, change timing at will, etc... The options are to provided would be similar to what is provided to Formula 1 currently but the options provided obviously be determined by manufacturers. Mandate transmission/engine modes performance, fuel efficiency/mix, etc... like they do in higher end cars?
http://en.wikipedia.org/wiki/Start-stop_system
http://en.wikipedia.org/wiki/Idle_reduction
- thinking more about reducing effective weight further without requiring extra power. One way is by altering our roads so that they are slightly magnetised. At the bottom of our cars (possibly the tyres/wheels (strategically placed to increase contact area)?) are also magnets/superconductors. They will repel one another and may possibly increase fuel efficiency.
http://en.wikipedia.org/wiki/Asphalt
http://en.wikipedia.org/wiki/Asphalt_concrete
Such technology could possibly be investigate on existing rail based vehicles?
- reduction of hydraulics/mechanical techology and increased use of electronics and fly-by-wire technology in cars. For instance, instead of relying on power steering, differentials, rack and pinion steering mechanism we use electronics and other techniques. While power distribution would still be centralised we would have what basically amounts to two mini-gearboxes at the front wheels (or on each side) of the car. Changing the gearing in these gearboxes allows each tyre to turn at a different rate which ultimately means that not only can get rid of a lot of extra mechanical parts (and weight) but we can get better turning circles as well. Obviously, a lot of testing is required to get this right/safe though.
- make seats (and other non-essential components) entirely removable (apart from drivers seat?). Significant reduction in weight, allows people the flexibility of increasing cargo/seating capacity of car at will as well, and obviously means fuel savings as well. Consider same technique in aircraft as well? I've been on a lot of aircraft where there was only a small number of people aboard. Curious to know what the fuel savings would be? Should be stop particular flights from running if there isn't enough demand?
http://news.nationalgeographic.com/news/energy/2013/04/130423-reshaping-flight-for-fuel-efficiency/
- thinking about the large/small car debate further. Let's say we go with one seat cars from now on but they also had the ability to be able to communicate with one another (similar to technology used in high end military systems such as the JSF). In effect, they could 'connect/communicate electronically' and take you to the same place at the same time (I'm thinking driverless as well as driver based world) anywhere/anytime. Moreover, we could dedicate a particular car to taking cargo in 'driverless mode' and if we so desired we could physically connect/detach them at will as well like in Lego/Transformers (I prefer to call it 'Transformatech').
http://www.forbes.com/sites/pikeresearch/2013/05/08/are-e-bicycle-sales-reducing-car-sales-in-europe/
- need to think about insulation further. As I previously indicate in other post films are one way of achieving it with windows but I think we should think further about using insulating materials in cars (impact of fuel consumption on cars?).
- thinking about means of further reducing effective volume of engines, power and fuel consumption but believe that cylinder reduction as outlined above may be enough?
- thinking about introducing something akin to 'suspended seating/shock absorption' in seats. If you've ever been involved in any form of accident (even low speed ones) you'll realise the level of shock that goes through the car. This would hopefully would reduce the number of whiplash incidents.
- obviously to make some of the technology above work we'll need to make our cars out of lighter materials. Cost is the big issue though...
- rig our traffic lights so that we don't have to start/stop so often (acceleration/decceleration are the biggest contributors to fuel consumption)
http://www.nytimes.com/2013/04/02/us/to-fight-gridlock-los-angeles-synchronizes-every-red-light.html?pagewanted=all&_r=0 - consider dynamic speed limits/signage as well which will mean that traffic flows as quickly/safely (increased average speed in low traffic areas will mean you arrive at destination quicker, which means less time road and less chance of accident?) as is possible?
- need to examine better methods of manufacture of technology.
http://www.extremetech.com/extreme/122231-solar-panels-made-with-ion-cannon-are-cheap-enough-to-challenge-fossil-fuels
One alternative means of dealing with the waste of silicon production I've been thinking about is by liquifying it and then spraying it on to a non-stick material (similar technique that they use in cooking). The resulting silicon is then 'peeled off'.
- further research into lubrication required. Longer term, I think we need to think about less mechanical/physical contact technologies though (superconductors, wireless, electromagnetic fields, etc...). If there is no contact there's no lubrication required...
http://www.siemens.com/innovation/en/news/2011/new-project-generator-with-superconductors.htm
- been thinking of this thermodynamic control issue. I've been thinking about something akin to a 'heat bubble'. Basically, we have a gaseous atmosphere around is more amenable to heating/cooling... Of course, finding a substance which is capable of doing this and is non-toxic/explosive? Possibilities with regards to cooking?
http://en.wikipedia.org/wiki/Heating_pad
http://en.wikipedia.org/wiki/Specific_heat_capacity
Another method is actually embedding heating pad style channels in our clothing (charged by putting them in the oven/microwave) so that we can basically have heat on tap.
- based on what I've read most cars use a form of heat pump to deal with heating. What if we start to use some of the heat from the engine itself though? We control the level of heat by changing level of seperation between car body and engine? and mix of ambient air and engine heat? Something else that probably needs to be looked at is air extraction. Based on what I've read a combination of extraction and new infusion of air is far better mechanism of ensuring new air circulation and hence termpature control.
https://en.wikipedia.org/wiki/Electric_heating
https://en.wikipedia.org/wiki/Nichrome
http://www.guardian.co.uk/technology/2006/dec/14/energy.insideit
https://en.wikipedia.org/wiki/Heat_pump
Would also like to see stronger use of chemically based heating technologies. If we use conventional electrical means in order to heat up something similar to a heat pack, then use chemical based heat energy from the actual heat pack itself thereafter, we can can further reduce further 'active energy consumption. The benefit is that based on what I've read 'Heat Pack' style technologies are quite re-useable and may actually be easier to maintain in the long term. Possibly consider 'magnetic cooling' or other sources of more efficient cooling?
http://en.wikipedia.org/wiki/Magnetic_refrigeration
https://en.wikipedia.org/wiki/Heat_pump
- thinking about introducing materials into wallpaper, carpet underlay, ceramic tyles (only where appropriate), paints and dyes so that they they provide a layer of insulation as well?
- use the ground in a better fashion as well. Based on what I've seen the temperature of the ground/sea is generally cooler than that of the air. If we had thermally conductive pipes that actually built into architecture and could be lowered raised at will (they could even be left in there permanently as a means of regulating overall temperature depending on ground temperature fluctuations with insulation being used when higher temperatures are required) we may be able to gain some form of passive cooling (heat tends towards cooler areas)?
- would like to see another design philosophy addition to our technology. We shouldn't think about energy being generated at centralised points only. We should try to de-centralise it as well. In previous post outlined using environment as a means of gaining energy. Believe that we should take it a step further. Whenever possible, we should take the position of not only mitigating energy requirements but becoming 'energy neutral' as well. Namely, the object in question should be able to have it's own source of power generation and storage. For instance, there have recently been stride into transparent solar panels. It's clear that efficiency is a problem but given time we may get to the point where our windows, walls, and even lights may be able to power our buildings.
http://www.extremetech.com/extreme/149163-mit-startup-makes-transparent-solar-cells-that-will-allow-your-smartphone-to-power-itself
http://www.thenational.ae/thenationalconversation/industry-insights/energy/dubai-looks-to-rooftop-solar-power-revolution
In keeping with this theme of self power generation is something I called 'Tendril Technology'. For a while now, scientists have been experimenting with nanotechnology and tiny motors. What if we do the reverse? If you've ever seen a wind tunnel you'll see that they sometimes use strips of paper in specific locations to determine direction of flowing gas/liquids. Let's take it a step further. Rarely are the aerodynamics of any object perfect. If we can design tiny electrical generators that stick to particular circuits/areas of a vehicle (like tendrils) than we can make use of the movement of the car itself to generate free energy as the car moves. Of course, this will possibly come at the cost of drag and I'm unsure of the economics of the concept itself? Possibilties are of course endless if such technology is viable/works... We could generate energy ourselves by virtue of our own personal movement. In the meantime am thinking about kinetic based generators being used on cattle and other animals. If insted of just a cow bell, a power generator was also installed I'm curious how much 'free energy' we could create (similar technology to road/pavement based power technology as outlined in previous post)?
http://en.wikipedia.org/wiki/Electric_generator
- been thinking of other 'free energy' possibilities. One is based on sound. A speaker is driven through electrical current but it can also act as a microphone and generate current as well. How much electricity could you generate using such technology at airports, shopping centres, concerts, etc?
- more intelligent use of solar panels. One design that I saw for a solar farm involved a large number of materials directed towards a group/central point of solar anels. Take this further. If we use parabolic mirrors or have a series of moving mirrors that re-direct sunlight based on the position of the sun we can gain even further gains in efficiency?
- have been considering shield concept further. One option is to use space junk (old satellites, space stations, etc...)(we may need to develop a 'Scooper Bot' to herd junk into the correct area before we can commence building obviously) to build part of the structure for the final shields (one at both poles in geosynchronous orbit. Else low Earth orbit may be fine as well depending on the impact of the shield, size required, etc?). Others things to think about is whether we should make it static or do we allow it to have varying levels of power (like a blind that you can open/shut to varying degrees)? Whether we have multiple shields in regional areas to allow for better climate control around the world? If this is the case would we be able to finally control the weather? Would these sheilds provide any protection against solar flares? Could we also connect satellites/space stations to them? Would they have any impact upon our existing satellite networks and other operations? We'll likely require improved methods of joining/construction in outer space (we need to do this at some point down the line anyhow).
http://www.irishexaminer.com/breakingnews/world/spacewalk-bid-to-fix-iss-leak-594052.html
- even if we can't mandate use of green technology but we may be able to guarantee orders. Government forms huge proportion of GDP in a lot of developed countries and if we were able to guarantee orders of truly revolutionary green technologies that may increase the incentive for companies to make a bet on green technologies.
- better access/marketing of recycling facitilies require? Should we mandate levels of recycled product in appropriate areas?
http://en.wikipedia.org/wiki/Oil_recycling
Possibly better develop better means of providing means of transferrance of recylable goods? Clear that we need some local options though we'll debate this issue in another post.
http://www.plasticsnews.com/article/20130424/NEWS/130429956/new-york-city-adds-rigid-plastics-to-recycling-program#
http://qz.com/82640/china-doesnt-want-your-trash-anymore-and-that-could-spell-big-trouble-for-american-cities/
- have been thinking about methods of automatically sorting landfill (it's clear that at some point we should be able to use robots). One uses pre-defined shape/colour coding of packaging. Another involves using facial recognition and then sorting based on a database of goods commonly found in supermarkets and shopping centres...
- it's clear that we still have significant problems with regards to reaching other planets.
http://www.news.com.au/technology/sci-tech/nasa-says-mars-travel-a-priority-for-united-states/story-fn5fsgyc-1226636397899
Perhaps we should consider setting up a moon base in the meantime? Experiments with life support, energy/food production, and mining in particular?
- should we start to have environmental efficiency levels on buildings as well?
- based on what I've seen a lot of energy storage technologies I've come across are thermally/environmentally dependent to extract maximum performance. Do we design them with insulation and active thermal management systems as well (already present in Lithium based technologies to some extent)?
- don't think we're dealing with Chinese dumping situation particularly well. Clear that many countries have issues with the Chinese though so it's obvious that they have a case. Problem with tariffs is that they will drive overall prices up. Consider taking it to the WTO? If tariff is preferred, make it smaller/take it and then use this to subsidise (partly or completely) local industry? It will hopefully drive prices downwards overall (dumpers now have to drive prices down further to compete or run the risk of being un-competitive (they may be willing to do so in the short/medium term)) and make adoption of such options much more easier/sensible down the line? http://www.dw.de/eu-imposes-anti-dumping-duties-on-chinese-solar-panels/a-16798471
http://www.smh.com.au/business/carbon-economy/chinese-kneecapping-solarpanel-rivals-20130509-2j97b.html
http://www.smh.com.au/business/carbon-economy/eu-to-slap-hefty-tariffs-on-chinese-solar-pv-20130509-2j8tc.html
http://www.smh.com.au/business/carbon-economy/germany-seeks-amicable-end-to-china-solar-tiff-20130514-2jiww.html
http://www.globalpost.com/dispatch/news/thomson-reuters/130513/eu-agrees-china-solar-panel-duties-boldest-move-yet
http://www.forbes.com/sites/christophercoats/2013/05/13/eu-finally-moves-on-chinese-solar-threats/

- some countries are clearly protectionist with regards to clean energy generation technologies in Europe or else don't want/have the resources to invest in such infrastructure. Consider opening up grids/lines between countries and provide excess energy across borders? Consider setting up joint ventures between countries? Thereafter, use equity shares to perhaps sell off to re-invest? Sovereignty/percentage share still remains the same ideally as was originally.
http://au.news.yahoo.com/world/a/-/world/17085680/analysis-central-europe-power-markets-to-stay-niche-as-banks-exit/
- as state previously more alternative fuel research required...
use thermal conditions/enzymes/catalytsts as a means of controlling electrical discharge rate?

http://blogs.wsj.com/chinarealtime/2013/05/09/chinas-vision-for-a-new-urbanization/
https://www.gov.uk/government/news/european-ministers-set-out-timetable-for-eu-ets-reform
http://www.theaustralian.com.au/opinion/columnists/winds-of-change-blowing-in-china/story-e6frg76f-1226637881187
http://www.theaustralian.com.au/national-affairs/sustainability-a-new-way-of-life/story-fn59niix-1226637910585
http://www.slate.com/articles/technology/future_tense/2013/05/china_cap_and_trade_carbon_tax_the_country_may_lead_the_global_climate_change.html
http://canadafreepress.com/index.php/article/55047
http://thinkprogress.org/climate/2013/03/29/1791811/bombshell-imf-study-united-sates-is-worlds-number-one-fossil-fuel-subsidizer/?mobile=nc
http://online.wsj.com/article/SB10001424127887323744604578470841012284404.html
http://online.wsj.com/article/SB10001424127887323528404578452483656067190.html?mod=trending_now_1
http://www.bbc.co.uk/news/science-environment-22491491
http://www.theage.com.au/business/carbon-economy/heavy-debt-weighs-on-chinas-renewable-energy-sector-20130514-2jiws.html
http://www.guardian.co.uk/environment/2013/may/08/ed-davey-coalition-climate-change-sceptics
http://blog.chron.com/sciguy/2013/05/how-much-has-the-climate-change-controversy-poisoned-the-well-of-environmentalism/
http://www.wired.com/wiredscience/2013/05/the-science-laureate-of-the-united-states/
http://www.boeing.com/stories/videos/vid_11_tailored_arrivals.html?Quartz-Tailored-Arrivals
http://www.extremetech.com/computing/105343-graphene-improves-lithium-ion-battery-capacity-and-recharge-rate-by-10x
http://www.extremetech.com/extreme/106539-stanford-creates-everlasting-nanoparticle-battery-electrode-free-water-based-electrolyte
http://www.extremetech.com/extreme/142962-princetons-nanomesh-nearly-triples-solar-cell-efficiency
http://www.nzherald.co.nz/business/news/article.cfm?c_id=3&objectid=10882569
http://www.afr.com/p/technology/intel_super_chip_haswell_no_tablet_08kxaMcIN4ray5YvXoRk0O
http://en.wikipedia.org/wiki/Carbon_capture_and_storage
http://www.gizmag.com/research-carbon-dioxide-methanol/11483/
http://en.wikipedia.org/wiki/Paper_battery
http://phys.org/news5805.html
http://www.theage.com.au/business/saviour-needed-for-australias-auto-industry-20130412-2hqwy.html
http://www.smh.com.au/comment/uni-is-all-about-teaching-not-research-papers-20130506-2j3dr.html
http://www.smh.com.au/opinion/political-news/liberal-party-discontent-grows-20130509-2jau3.html
http://www.smh.com.au/opinion/political-news/carbon-price-working-coal-slumps-clean-energy-soars-20130509-2jals.html
http://www.wired.com/autopia/2013/05/al_drivebywire/
http://www.wired.com/autopia/2013/05/lamborghini-egoist/
http://smh.drive.com.au/motor-news/lamborghini-singleseater-20130513-2jh4a.html
http://smh.drive.com.au/motor-news/vf-commodore-less-fuel-less-power-20130510-2jbht.html

Background

From time to time it’s useful if you can turn up the debugging messages that come from Bash, when working out either interactive or shell script problems. Here are 2 methods that can help in getting down to the details.

Solution

There are essentially 2 methods.

Method #1: -x method

When writing a shell script you’ll sometimes want to turn on line by line debugging. There’s basically 2 ways to to this.

Before we get started, suppose we have this sample script, myscript.bash:

1 2 3 4 #!/bin/bash   echo "hi" echo "bye"

First you can run your script like so:

1 2 3 4 5 % bash -x myscript.bash + echo hi hi + echo bye bye

As an alternative you can add the following line, set -x to the top of our shell script to enable debugging as well:

1 2 3 4 5 #!/bin/bash   set -x echo "hi" echo "bye" 1 2 3 4 5 % ./myscript.bash + echo hi hi + echo bye bye Method #2: env SHELLOPTS=xtrace …

This approach sets the env. variable SHELLOPTS=xtrace which has the same effect as using bash -x.

For example:

1 2 3 4 5 % env SHELLOPTS=xtrace ./myscript.bash + echo hi hi + echo bye bye

You can also use this technique to debug your bash environment (think .bashrc and .bash_profile) like so:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 % env SHELLOPTS=xtrace bash ... ... +++++ line='complete -f -X '\''!*.@(zip|[ejw]ar|exe|pk3|wsz|zargo|xpi|sxw|o[tx]t|od[fgpst]|epub|apk)'\'' unzip zipinfo' +++++ line=' unzip zipinfo' +++++ list=("${list[@]}" $line) +++++ read line +++++ '[' 'complete -f -X '\''*.Z'\'' compress znew' '!=' 'complete -f -X '\''*.Z'\'' compress znew' ']' +++++ line='complete -f -X '\''*.Z'\'' compress znew' +++++ line='complete -f -X '\''*.Z'\'' compress znew' +++++ line=' compress znew' +++++ list=("${list[@]}" $line) +++++ read line +++++ '[' ' zcmp, zdiff, z*grep, zless, zmore intentionally not here, see Debian: #455510' '!=' '# zcmp, zdiff, z*grep, zless, zmore intentionally not here, see Debian: #455510' ']' ... ...

Here you can see every command getting executed from the system and user’s .bashrc and .bash_profile as bash starts up.

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

Readers who viewed this page, also viewed:

• Intro to my Blog’s [one-liner] Code Block Style Guides
• Powered by Where did they go from here?

Quiz on chapter 1 of book Linux Device Drivers (LDD3) body {margin: 10px;} Linux Device Drivers (LDD3)

1. A device driver provides

Policy
Mechanism
Policy and Mechanism
Neither of them


2. A piece of code that can be added to kernel at runtime is called as

Model
Program
Module
Kernel Code


3. /dev/consosle is a

Character Device
Block Device
Network Device
USB Device


4. To link an object code dynamically to the running kernel we can use the command

insert
modins
inmod
insmod


5. Who is allowed to load modules into the kernel ?

Superuser
All users
Kernel
No body


6. The linux is versioned under which license ?

GNU GPL 1
No license
GNU GPL 2
NON GNU GPL


7. Which of the following is not a class of devi ce

Character
String
Block
Network


8. Third party softwares should be treated with care to take care of the

Security of kernel
Look of the kernel
Language of the kernel
Length of the code


9. Is it possible to compile the linux kernel with not support for modules ?

Yes
No
Depends on the Compiler
Depends on the architecture


10. Linux supports only i386 architecture true or false ?

True
False
Depends on the compiler
It supports only one other architecture.

Hello, This article will show you how to change password of LDAP Users. In This Article, I have demonstrated that How to change your Password for those users which are in 389-ds or Redhat-ds. This is very simple you just need to change few parameters and You will able to change password of Ldap users [...]

This command lets you turn the build in OS X firewall on and off, on both for specific services or essential services. It works with OS X 10.5, 10.6, 10.7 and 10.8.

So the big news over the past week or so on the FOSS social networking front has been that Identica is moving the back end to Pumpio, and the Identica accounts will be moved over. This has ultimately led me to move away from Identica.

The thing for me, is that it's not just about the social network itself, but how you interact with it. As I see it, there's two genres. There's the short text burst network, like Twitter and Identica, and there's the kitchen sink social network like Google+, Facebook, Diaspora and now Pumpio. There's nothing wrong with any of this. Choice is a good thing. I kinda liked the short burst version but totally understand why a larger block is a good thing.

The thing that defines it for me, is that I like to use a client. I don't want to have to open a web browser to read replies or post something. To me, that turns it into just another website. With Twitter and Identica, there are many clients for all platforms. When Identica switches over to Pumpio, the vastly different, and apparently improved APIs will break Identica support in those clients.

Who knows how long it'll take for these clients to get Pumpio support, or if they will. I guess some will, while others won't. Some may be more advanced in that path than others. In the meantime, those of us who use clients will be forced to use the web browser while we're dangling for an update to our client of choice. This has led me to explore a different approach, and one that doesn't include Identica or Pumpio.

I use Facebook, but as little as possible. It's a "for family" thing. I was already using Twitter via a Twitter bridge. I've now moved that up the chain to be a first layer, not a second. I used to check my RSS feeds, then post to Identica, now I've started cutting Identica out of that process and going straight to Twitter. I've had a GMail account for ages, I'd never even considered enabling Google+ on it. I have now, although I'm still figuring out circles, communities, hangouts etc.

But wait, I hear you say, "you said you didn't want to use a web browser for this stuff, if you're using it for those, why not Pumpio too?"

I'm not using it as a website. I'm not using any of those as a website. I'm using browser addons and plugins to have notifiers, quick access to posts, replies, direct links etc. All three have good mobile app support too, so there's consistency across platforms. Having said that, I won't be letting Facebook anywhere near there.

It's not even as if there's just one app or plugin. There's loads of them for Chrome, Firefox, Android. The app developers have a wide variety of ideas on what they like, which means you can try them and find a way you like to work.

If none of those existed, and you had to interact with Twitter, Google+ and to a lesser extent Facebook like ordinary websites, where you bookmark them, and have to visit specific pages to check latest posts or replies, then I wouldn't use any of them. Right now, that's what Pumpio is. I know this is about priorities and resources, where Evan is concentrating on getting the site right, and encouraging developers to use the APIs to create apps and plugins. I hope this pays off. As that starts to happen, I'll continue to explore coming back to Pumpio.

"What about federation?" I hear you ask. I did try installing Statusnet a couple of times. The first time I installed it wrong. The second it went fine. I saw way too many people in my very limited timeline who had federated and had no end of issues with updates breaking, or having to clear and restart daemons or whatever that there's no way I'd want the hassle of maintaining my own instance. All I did was install on a localhost and play with it. I never announced it, nor joined up with others etc.

I believe Evan has learned a lot from how federating wasn't as easy as he'd hoped, when he built Pumpio. I believe it has a lot of new improvements in this area. I hope so. I like the concept of federating, but it's only of interest to me, when it gets to a similar level of maintenance stability and maturity as WordPress or Drupal. That just takes time.

I wish Evan every success with Pumpio, and maybe down the line I'll be using it. In the meantime I've decided to ease back from Identica. You can follow me on Twitter and Google+. It'll take some time to find and add people on both. It's not something I do often, and when I did in the past, I concentrated on Identica and mostly ignored Twitter.

Tags: TwitterIdenticaPumpioGoogle+Facebook