Filters


  1. To help you get familiar with "filters", and have a feeling about their powers, here are some extra exercises for you.
  2. Type "wget https://ms16.ipv6.club.tw/~solomon/acm0612.csv" to download the file "acm0612.csv" to your current working directory.
  3. Use an editor (vim or nano) to inspect the file.  You can see that the file has 29 lines (the first line is the header).  Each line has 8 fields.
  4. When the first time I saw the 6th field "是否有意願參加", I have a question "Will anyone submit this form with this field saying NO?"  So let's use Linux filters to verify this.
  5. Exercise 1: Use "cut" to extract the 6th field.  This helps you to inspect the values of the 6th field.
  6. Exercise 2: Because the previous results have 29 lines which are beyond the screen height, you are unable to see all of them.  Although you can use your mouse to scroll up and down to see the whole output, for a professional like me who doesn't use a mouse much, it is a little convenient.  Therefore, I decide to pipe the result to the command "uniq".  (You may run "man uniq" to learn the synopsis of "uniq". And you are encouraged to try "uniq -c" to compare its difference from "uniq".)
  7. Exercise 3: We want to exclude those who didn't show up.  (The rain was really very heavy on June 12.)  And we only need the information about "Student ID", "Name", "ACM Membership" for the following exercise.  Therefore, please use "grep" to get those lines whose 8th field is TRUE, then extract fields 2, 4, 5.  Save the output in a file "participant.txt".
  8. Exercise 4: We want to know, among these participants, who already have an account on ACM1.  To do so, we need to prepare a file which lists all students who already have an account on ACM1.  We know account information is stored in /etc/passwd.  (For U2 students who are working on ms16, you may retrieve a copy of /etc/passwd of ACM1 at https://ms16.ipv6.club.tw/~solomon/passwd).  However, in that file we see several accounts used by system services like "mail", "news", "uucp", so we want to extract only account names which look like "s11x321xxx".  Now, you may want to use "cut" to extract the 1st field, and pipe to "grep" to filter out those accounts which begin with "s11".  Now before we store these account names into a file "account.txt", there is one minor task we must do.  We notice that in "participant.txt", the "Student ID" does not have a leading "s", so we want to remove the leading "s" for data we obtain from /etc/passwd so that the Student ID will be consistent in two files.  This can be easily done by piping the accounts to "cut".  In addition to cutting fields, "cut" also allows you to cut some "columns" by the "-c" option.  This is quite useful for our current task, because we want to cut the text from column 2 to column 9.  So by running "cut | grep | cut" and redirect the output to "account.txt", you got a file with all students who already have accounts on ACM1.
  9. Exercise 5: With the two files "participant.txt" and "account.txt", it is easy to support our decision making.  "wc -l participant.txt" will tell us that 23 students attended the Pizza Party and Webpage Training on 6/12.
  10. Exercise 6: If we want to know which students in "participant.txt" already have accounts on ACM1, we can use "grep -f PATTERN_FILE" to print students who are listed in "account.txt", which is the PATTERN_FILE.  If you further pipe the result to "wc -l", you can quickly see there are 19 students.
  11. Exercise 7: Adding "-v" to the "grep" command in the previous task will show you the 4 students who do not have accounts on ACM1 yet.  They are not ACM members, either.  So, we may pass this information to ACM Membership Chair (洪靖容) so that she can further contact these students and invite them to join ACM to learn more skills with us in the future.