Posted on by

Lab 8: Pipes

Here is your Lab 8.

Lab 8 is a graded exercise. You have two weeks to complete. Submit your solution to the IVLE workbin before Sunday, 13 November 2011, 10:00pm.

Note that this lab is to be completed on SunFire, to reduce the dependencies on Linux machines in the OS lab.

36 thoughts on “Lab 8: Pipes”

  1. Hello Prof Ooi,

    May I clarify my understanding of Lab 8. Consider the following bush command:

    “ps -eaf | grep (matric no) | wc -l”

    In the context of Lab 8, does that mean:
    (1) I do ps -eaf, which gives me 100 results
    (2) Then based on the 100 results, I do grep (matric no), which gives me 5 results
    (3) Then based on the 5 results, I do wc -l, which gives me an output of value 5?

    I asked this because I realized that the actual bash does ‘real-time update’, i.e. the result wc -l will include the grep process which the above example did not.

    My question, then, is: Which is the one that we are supposed to do? If it is the latter, may you advise how to do real-time update?

    • Whether or not the “grep” process is included is beyond our control. Due to race condition, the “grep” process may or may not be running when “ps” is executing.

      So, you need not worry about “real-time” update. Just fork() and exec(), and let the OS scheduler takes care of the rest.

  3. Dear Prof. Ooi,

    When I tried to compile my bush in sunfire, I get the following error:
    readline/readline.h: No such file or directory

    I’ve tried linking the readline library in gcc and also linking curses.h and other proposed solutions in google, but it still gives the same error message.

    Any suggestions on how to resolve this?

    Thank you for your help.

    • Ivan, how did you compile?

      You need to tell gcc where to look for header files and library files with the -I and -L options. These options have been setup for you inside the Makefile.

      • I tried something like this:
        gcc -lreadline bush2.c
        gcc -lreadline -lncurses bush2.c (I put include curses.h)

        and permutations of it (of course with gcc in front)

        • On Sunfire, the libraries and header files are stored under /opt/sfw/library and /opt/sfw/include respectively. So, you need to say

          gcc -I/opt/sfw/include -L/opt/sfw/library bush2.c parse.o -lreadline -lncurses

          You can run “make”, which will print out the command used to compile while it is compiling.

  5. I have the same problem

    bush2.c:8:31: fatal error: readline/readline.h: No such file or directory

    I’m using sunfire

  6. Dear Prof,

    if I do something like “ls | wc”, my bush prints the correct result but exits right after that (both on sunfire and in os lab)

    this is due to exit(0) if command == NULL @ L20, L21

    I wonder if this is acceptable?

          • for example, “ls | wc” gives “0 0 0”
            I guess the result of the first command didn’t go into the second command correctly; is it due to the difference between sunfire and linux?
            I only difference I found is dup2() doesn’t close old file in sunfire, so I added a close() after each dup2(), but the result is still wrong

  7. can’t seem to login my OS lab anymore.. any idea what’s the password for common OSlab_user ? Or how to retrieve old password? I want to try testing my program to see if it works on OSlab rather than sunfire

    • Please look for Mr. Chan TF at the Technical Helpdesk (if he is not in the OS lab already) for password issues.

  8. okay nvm, I kinda fixed it on sunfire lol….
    must I use fd[2*MAX_PIPE]?

    can i change it to fd[MAX_PIPE][2]?

  9. Hello Professor,
    I’m just wondering, the parent process has to close unused pipes, but how does the program ensure that the child process has already finished writing/reading from the pipe before the parent process has to close them? Thanks!

    • It does not have to. A file descriptor is associated with a process. So the parent can close the write end of a file descriptor (pointing a file) and the child can still write to the file descriptor (pointing to the same file).

  10. may i ask when i do a pipe(test)

    when i close(test[1]) on the child side, does the parent side test[1] get closed as well?

    or is it after I fork, the pipe actually has 4 ends? (2 on child, 2 on parent)

    • @Ming Kit,

      A file descriptor is something associated with a process — you can view it as a process’s pointer to a “file”.

      When you close a file descriptor in one process, other file descriptors (in other processes) pointing to the same “file” remains open.

      Yes, you can view it as “the pipe has 4 ends after you fork()”

  11. A bug you might face in this lab is that a process might hang waiting to read from the pipe. This could happen if you do not close unused ends of the pipe.

    Why must unused ends of a pipe be properly closed for the pipe to work properly?

    There are three things I need to explain.

    1.
    Typically a command would keep reading until the EOF character is read. Try this with commands such as cat or wc that reads from the stdin in your shell without any argument. The command would read until you enter the EOF character (Control-D).

    2.
    When the write-end of a pipe is closed, an EOF character is sent to the read end.
    When all write-ends of a pipe is closed, attempting to read from the pipe would return an EOF.

    3.
    Now, let’s see why we need to close each unused end of a pipe. Let’s consider the following example.

    We have two processes P and C. P sets up a pipe with two file descriptors Rp and Wp (R for read end, W for write end) and fork the child process C. Let’s call the file descriptors at the child process of the same pipe Rc and Wc.

    Consider the example where P wants to send something to C using the pipe. During lecture, I said we need to close the unused Rp and Wc (since P writes to Wp and C reads from Rc) for the pipe to work properly.

    Now suppose C forgot to close Wc. After P writes to Wp and closes Wp, C would continue to try to read from Rc. This is because the OS sees that another process leaves Wc open, and could still write to it. C therefore would not read EOF that it is waiting for, and would keep waiting!

  12. Hi Prof,

    Thanks you for the detailed explanation of closing of writing end of pipes. However I would like to ask about closing of the reading ends. I’ve tried both closing and not closing the reading ends of pipes and it make no difference to the result which I think is reasonable as all the reading ends will just be there waiting for the EOF. So this makes me wonder is it necessary to close reading ends and will there be any effect if they are not closed.

    • @Ping Yang,

      This is a very good question. The same situation would apply to the writing end as well.

      What would happen if you write to a pipe where all the reading ends are closed? A SIGPIPE signal would be sent to the calling process, and write returns an EPIPE error. So that the calling process can stop writing (e.g., when you do “yes | head”).

      If one process leaves an unused reading end open, then the OS would expect that someone would still be interested in reading and thus would allow write to keep writing, until the buffer is full and write( ) blocks. The writing process would hang.

  13. I use make in sunfire and it has the following message,
    gcc -g -Wall -I/opt/sfw/include -L/opt/sfw/library -o bush parse.o bush2.c -lreadline -lncurses

  14. Hi Prof,

    Can i assume that the maximum process allowed for the pipping is 8?

    eg.
    ls | grep a | grep a | grep a | grep a | grep a | grep a | grep a | grep a

    Thank you.

    Your Sincerely,
    Loke Yan Hao

    • Sorry. I read through the code, and found out that there is actually a part whereby the program will ensure that the command does take in more than 8 pipes.

      Thanks =)

  16. Hi Prof,
    I tried 3 different approaches on Linux which all give the correct answer, but none of them works on Sunfire. (for example, “ls | wc” always give “0 0 0”) I already replaced the waitpid() with the new code you gave us

    Could you plz take a look at my code tomorrow during the lecture break and give me some advice? (I can print out and bring to class)

    • Yes, I can take a look. But besides the code, I would like you to draw pictures of pipes by tracing through the code (just like the ones I drew in lecture) and show what each fd is point to, how they are duplicated, which one is closed etc.

      Tomorrow, you can explain the code to me along with the pictures.

      (For others who have trouble debugging, I find drawing pictures help a lot).

Comments are closed.