Bash: Working with files
From Joe's Boredom
Remove duplicate lines in file
<source lang="bash">
- Will sort file out-put
sort -u file.in > file.out
- Works same as above, sort file out-put
sort file.in | uniq > file.out
- Will remove duplicates and keep file in same order
awk ' !x[$0]++' file.in > file.out</source>
Remove blank lines in file
<source lang="bash"> sed '/^$/d' input.txt > output.txt
- or
grep -v '^$' input.txt > output.txt
- Loop dir, remove blank lines from all files
files="/home/me/data/*.txt" for i in $files do
sed '/^$/d' $i > $i.out mv $i.out $i
done </source>
Read file line by line
<source lang="bash"> while read line do echo $line; done < "file.lst" </source>
Count lines in file
<source lang="bash">countTotal=$(awk 'END { print NR }' file.name); echo "$countTotal"; </source>
Check if file exists
<source lang="bash"> if [ -s "$file_name" ]; then
echo 'yes';
fi </source>
Check if dir exists
<source lang="bash"> if [ -d "$folder_path" ]; then
echo 'yes';
fi </source>
