luni, 16 noiembrie 2009

Bash URI parser using SED

Warning! This version is now obsolete!
Check out the new and improved version (using only Bash built-ins) here!



Here is a command-line (bash) script that uses [cci lang="bash"]sed[/cci] to split the segments of an URI into usable variables. It also validates the given URI since malformed strings produce the text "ERROR" which can be handled accordingly:
[cc lang="bash" nowrap="true"]
# Assembling a sample URI (including an injection attack)
uri_1='http://user:pass@www.example.com:19741/dir1/dir2/file.php'
uri_2='?param=some_value&array[0]=123&param2=\`cat /etc/passwd\`'
uri_3='#bottom-left'
uri="$uri_1$uri_2$uri_3"

# Parse URI
op=`echo "$uri" | sed -nrf "uri.sed"`

# Handle invalid URI
[[ $op == 'ERROR' ]] && { echo "Invalid URI!"; exit 1; }

# Execute assignments
eval "$op"

# ...work with URI components...
[/cc]Notice the [cci_bash]"uri.sed"[/cci_bash] file given to [cci_bash]sed[/cci_bash]?

vineri, 6 noiembrie 2009

Recursive file/directory change-detection

Version 2


Another, much faster method would be to use [cci_bash]ls -lR[/cci_bash] to browse over the filesystem. On a newly installed Debian virtual machine (on Xen) hashing the entire filesystem (the root directory) took approximately 1.7 seconds. So, here it is: [cc lang="bash"]ls -lR "$D" | sha1sum | sed 's/[ -]//g' [/cc]

This method is sensitive to file name, size and modification size; usually that would be enough but if you need more control use...

Version 1


Detect when the contents of a file or directory ($D) changes: [cc lang="bash"]find "$D" | while read f; do stat -t "$f"; done | sha1sum | sed 's/[ -]//g' [/cc] This yields a hash of the current state of the file or directory which is extremely sensitive to even the most subtle changes (even a simple touch to any file/directory somewhere inside "$D" changes the generated hash).

Detect number of CPUs on a machine

UPDATE: Steven pointed out (very nicely) that there's no need for [cci_bash]cat[/cci_bash] in this picture, [cci_bash]grep[/cci_bash] would do just fine on its own. So, thanks Steven!

Detect how many CPU cores are present on the running machine:

[cc lang="bash"]
grep -c processor /proc/cpuinfo
[/cc]

This can be very useful when writing multi-threaded programs to properly match the number of threads with the number of CPU cores.