These days I'm doing some data munching and I've been introduced by a kind colleague to a couple of simple things which made me think how it is possible that I ever lived without them: screen and xargs. If you do use linux and are not aware of these: know that your life can be better.
(Hoping this is not obvious for everyone. And if you have similarly miraculous tools, please let me know in the comments.)
Subscribe to:
Post Comments (Atom)
2 comments:
i like tmux better than screen (can't tell why, though)
If you like xargs, then you will probably love GNU parallel. It works by parallelizing the commands that you would pass to xargs.
This means that it allows you to use all the cores of your machine to get the job done. Not only that, but it also shows you some estimate of how much time is left for the task to be completed.
A silly example comparing xargs and parallel:
find . -type f -print0 | xargs -0 gzip -9
find . -type f -print0 | parallel -0 --eta gzip -9
In fact, it goes further than that: you can use other idle computers on your network to distribute the job.
Disclaimer: I'm the Debian maintainer of GNU parallel.
Post a Comment