meta data for this page
  •  

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
general:computerenvironment:shell_basic_commands [2019/12/01 20:32] – rubengeneral:computerenvironment:shell_basic_commands [2020/09/07 18:20] (current) – [If / Else] ingo
Line 19: Line 19:
 Remember to call the file only in the first command given: Remember to call the file only in the first command given:
 cat ejemplo.txt | grep "xxxxx" cat ejemplo.txt | grep "xxxxx"
 +
 +----
 +Tip: You can create this file with:
 +<code>
 +nano ejemplo.txt
 +</code>
 +paste the contents of the ejemplo file into the nano editor and close it using STRG/CMD + X
 +
 +Alternatively you can download the file by clicking on its name and do the exercises in a linux environment on your local computer (for example using mobaXterm on windows, or in the command line on macOS).
 +----
 +
  
 ===== Reading files + counting lines ===== ===== Reading files + counting lines =====
Line 29: Line 40:
   * ''cat ejemplo.txt | sort | uniq'' (sorts the content of the file, removes successive identical lines, and displays the output)   * ''cat ejemplo.txt | sort | uniq'' (sorts the content of the file, removes successive identical lines, and displays the output)
   * ''cat ejemplo.txt | sort | uniq -c'' (counts the number of unique lines in the file)   * ''cat ejemplo.txt | sort | uniq -c'' (counts the number of unique lines in the file)
 +
 +
 +===== Wildcards =====
 +The commands will apply to all the files in the current directory that match the pattern.
 +"*" is any number of any character. For example, the following command will concatenate all the files that have the ending ".txt": 
 +<code>
 +cat *.txt
 +</code>
 +"?" works as any single character:
 +<code>
 +cat ejemplo.t?t
 +</code>
 +
 +----
 +
 +===== Tab completion =====
 +
 +You can auto-complete paths by pressing the tab key once or twice anytime while writing a path. If the auto-completion does not work, there might be something wrong with your path..
 +
 +You can check out [[https://youtu.be/igwD7cL6QOk|this video]] to see how it works.
 +----
  
 ===== Pattern matching using grep ===== ===== Pattern matching using grep =====
Line 45: Line 77:
  
 ---- ----
 +
 ===== Regular expressions ===== ===== Regular expressions =====
   * ''grep "^x" ejemplo.txt'' (returns all lines starting with an 'x')   * ''grep "^x" ejemplo.txt'' (returns all lines starting with an 'x')
Line 129: Line 162:
 The output of the first part of the command is not getting passed as the input to the second part of the command, since the file is being read again.  The output of the first part of the command is not getting passed as the input to the second part of the command, since the file is being read again. 
 </hidden> </hidden>
-<hidden Moving files>+<hidden Saving file with the input name>
 <code> <code>
 cat ejemplo.txt > copy.txt cat ejemplo.txt > copy.txt
Line 137: Line 170:
 Input file is also the output. The contents of input file will be brutally deleted. Input file is also the output. The contents of input file will be brutally deleted.
 </hidden> </hidden>
 +<hidden Moving files into non-existing directories>
 +<code>
 +cat ejemplo.txt > copy.txt
 +mv copy.txt folder
 +cd folder
 +cat folder
 +</code>
 +If a directory does not exist, 'mv' renames the file instead</hidden>
 <hidden Grep and quotations> <hidden Grep and quotations>
 +Trying to find all ">" symbols in the following file:
 +<file txt important_sequence.fasta>
 +>Important_sequence1
 +AAATTCTCACCCCTCAGAAA
 +>Important_sequence2
 +ACCTCAGAAAAATTCTCACC
 +</file>
 <code> <code>
-echo -e $">Important_sequence\nAAATTCTCACCCCTCAGAAA" > important_sequence.fasta 
-cat important_sequence.fasta 
 grep > important_sequence.fasta grep > important_sequence.fasta
 </code> </code>
Line 153: Line 199:
 <hidden File names> <hidden File names>
 Some naming schemes can make your life as a programmer a living hell. Some naming schemes can make your life as a programmer a living hell.
-<file txt output\ file\ final\ final\ definitive\ 6.fasta>+<file txt output file final final definitive 6.fasta>
 aaa aaa
 bbb bbb
Line 159: Line 205:
 </file> </file>
 <code> <code>
 +rename -e 's/_/ /g' output_file_final_final_definitive_6.fasta
 cat output\ file\ final\ final\ definitive\ 6.fasta cat output\ file\ final\ final\ definitive\ 6.fasta
 </code> </code>
 </hidden> </hidden>
 +
 ---- ----
  
-====== Part II ======+====== Slightly more advanced bash commands ======
  
 ===== For-loop ===== ===== For-loop =====
Line 179: Line 227:
 **For-loop** **For-loop**
  
-  * for <fc #4682b4>i</fc> in <fc #4682b4>*plo.txt</fc>; do <fc #800080>echo $i</fc>; done+<ff serif><fs large>for <fc #4682b4>i</fc> in <fc #4682b4>*plo.txt</fc>; do <fc #800080>echo $i</fc>; done</fs></ff>
  
   * A <fc #4682b4>list of files</fc> that you want to process; namely, the files ejemplo.txt and copy_ejemplo.txt   * A <fc #4682b4>list of files</fc> that you want to process; namely, the files ejemplo.txt and copy_ejemplo.txt
Line 191: Line 239:
 <code>for i in *plo.txt; do cat $i | wc -l | sed -e "s/^/$i\t/"; done</code> <code>for i in *plo.txt; do cat $i | wc -l | sed -e "s/^/$i\t/"; done</code>
  
 +<hidden Exercise>
 +Find the lines that contain "a", "b", and "c" in ejemplo.txt, and save them into their respective files (a.txt, b.txt, c.txt). 
 +This means, each file should look like this:
 +<file txt a.txt>
 +aaaaa xxxxx
 +aaaaa bbbbb
 +axaxa bxbxb
 +</file>
 +Hint: let me start the command for you:
 +<code>
 +for i in a b c; do ...
 +</code>
 +</hidden>
  
  
 +----
 +
 +===== AWK =====
 +This command is also considered a programming language on its own. It is particularly useful when needing to process the elements of a table. The basic syntax is the following:
 +
 +<ff serif><fs large>awk <fc #008000>-F "\t"</fc> '{print <fc #4682b4>$2</fc> <fc #fa8072>"\t"</fc> $1}'</fs></ff>
 +  * Indicate the <fc #008000>delimiter of the table</fc>. Here it is specified to tabs, but the default is space.
 +  * <fc #4682b4>$ indicates the number of column</fc> that you wish to return
 +  * <fc #fa8072>Text added to the table</fc> must be written inside quotation marks; in this case the text is the addition of a tab character.
 +  * Columns can also be processed with mathematical operations. For instance, print $1 + $2.
 +
 +
 +<code>
 +grep "aaaaa" ejemplo.txt
 +grep "aaaaa" ejemplo.txt | awk -F "\t" '{print $2 "\t" $1}'
 +</code>
 +
 +<ff serif><fs large>grep "aaaaa" ejemplo.txt | awk -F "\t" '<fc #800080>$2 == "xxxxx"</fc>
 + {print <fc #ff0000>$0</fc>}'</fs></ff>
 +
 +  * You can indicate to only print the rows where the <fc #800080>column meets a certain condition</fc>. In this case, column 2 must be equal to "xxxxx".
 +    * Equal ==
 +    * Not equal !=
 +    * Greater than >
 +  * <fc #ff0000>$0</fc> prints all columns of the table
 +
 +----
 +
 +===== If / Else =====
 +
 +<code>
 +cat ejemplo.txt | wc -l
 +if [[ $(cat ejemplo.txt | wc -l) -ge 9 ]] ; then echo "File has nine or more lines"; else echo "File has less than nine lines"; fi
 +</code>
 +To compare numbers:
 +  * -eq = is equal to
 +  * -ne = is not equal to
 +  * -ge = is greater than or equal to
 +  * -le = is lesser than or equal to
 +  * -gt = is greater than
 +  * -lt = is lesser than