Assignment instructions are confusing causing problems with online format class #53
Comments
|
This is an interesting issue, which I thank you for raising. The issue you
raise is an intended part of students failing to think of cases they need
to handle. This is just one of a host of other issues students should have
tested for and handled in their defensive code.
The point of the defensive assignment is to have students have an easy
assignment where they have to handle these sorts of issues. Many students
overlook this issue and in past experience nearly all overlook at least
some issue. Then in the attack portion other students are generally quite
effective at pointing out the problem. This makes students reason better
about how their defensive code can fail as they write attacks.
However, I also think it may be possible that the specific assignment you
are using has some additional lack of clarity. If so, do you want to talk
about which version of the assignment you are using and in which way you
think it should be clarified?
…On Sat, Jan 19, 2019 at 5:14 AM Kurlee ***@***.***> wrote:
In the last semester, it became apparent that the instructions in
docs/EducationalAssignments/ are often confusing, and do not cover the
scope of what is required during the assignment.
Most common issue was that the instructions don't do a good enough job of
introducing the compatibility requirements that need to be built into the
reference monitor so it will not break the functionality of the entire
environment.
Example: The majority of students failed to build their reference monitor
with support for cases when the file was reopened after closing. This
caused issues during the second assignment, when we could not test the
quality of the security monitor, because they had not understood the whole
scope of the assignment.
Online format classes aggravate this issue significantly. I propose
clarifying the requirements of the assignments for future classes. I will
also volunteer myself to do so if there is no other candidate.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#53>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA0XDxjCGAk3hX71WRzISl2PYOubep_Mks5vEvAFgaJpZM4aJEWK>
.
|
|
The main issue we ran into was that students had a difficult time doing the assignment because they had difficulty grasping the entire scope of the project and the base functionality. The assignment we worked on this past semester was the ABopenfile series. I would characterize the issue the majority of the students had as having a hard time linking the assignment, documentation, and guidance peppered in the class forums by TA's. I was really excited about this assignment and it was disappointing to see that I could not properly asses the majority of reference monitors during assignment two because basic functionality was not there. Upon inspecting the source code, it was apparent that the students had implemented features to protect against basic invalid write attacks. I think that the functionality of the openfile method should be clarified in the original assignment as this was the most common issue. Sort of a "What you should be able to do" and "What you shouldnt be able to do" kind of thing |
|
Okay, I think with that context, this makes sense. Do you want to send a
PR?
…On Sat, Jan 19, 2019 at 8:07 PM Kurlee ***@***.***> wrote:
The main issue we ran into was that students had a difficult time doing
the assignment because they had difficulty grasping the entire scope of the
project and the base functionality. The assignment we worked on this past
semester was the ABopenfile series.
I would characterize the issue the majority of the students had as having
a hard time linking the assignment, documentation, and guidance peppered in
the class forums by TA's. I was really excited about this assignment and it
was disappointing to see that I could not properly asses the majority of
reference monitors during assignment two because basic functionality was
not there. Upon inspecting the source code, it was apparent that the
students had implemented features to protect against basic invalid write
attacks.
I think that the functionality of the openfile method should be clarified
in the original assignment as this was the most common issue. Sort of a
"What you should be able to do" and "What you shouldnt be able to do" kind
of thing
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#53 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA0XD7_ekhOZcuWett9VI7cTCgCVU2V6ks5vE8E1gaJpZM4aJEWK>
.
|
|
Yes, I will go ahead and send a PR in the next couple days Thank you, Professor! |
|
Hi, I wanted to piggyback off this mainly because issue is similar in that I believe there is an issue with instructions, and I didn't know a way to phrase the title without giving anything away for students because I also have a very specific issue with instructions as well. Apologies for wall of text in advance. Please let me know what you think Here's a TLDR:
1 Instructions cause what I believe to be a vulnerability.I noticed in the ABopenfile series, in the reference monitor part one it states:
Implying that you keep the a or b file. Keeping the readonly a file, can cause issues if someone tries to open it. I can email you with what issues it might cause that I think is just an inherant vulnerability. 2 Instructions unclear. Specifications and scope are too broadThis being said, the other issue is that it is still confusing as to what the reference monitor is suppose to do to operate correctly. While it is suppose to be open ended and is subject to interpretation, I feel like it does become confusing when there aren't examples of expected behavior. We have an example where file should not have been written to before closing, but not what files we should end up with after closing. It's hard to assess students when students have different interpretations of what is expected. This causes stress as students are unsure what is and is not valid behavior and if something is a thing students should allow. This also causes problems when students write attack code and believe they have found a vulnerability where the defender believes that what attacker did was valid behavior and depending on interpretation both can be correct. While some of my examples seem intuitive such as updating the original file, because why would you have the newer file be a The language is also a little unclear, while I found "the older one is discarded" to obviously mean the valid backup as we only want the valid written to file, it left students concerned with timing. I don't feel like you would be giving anything away by stating that we would keep the valid write only b file if both files are valid. |
|
Okay, thanks for pointing these out. Would you like to submit a pull
request with proposed changes to the assignment text? I'd prefer the
instructions are clearer even if they give things away, but I can be
convinced otherwise.
…On Sat, Sep 21, 2019 at 4:42 AM Victoria Zhong ***@***.***> wrote:
Hi, I wanted to piggyback off this mainly because issue is similar in that
I believe there is an issue with instructions, and I didn't know a way to
phrase the title without giving anything away for students because I also
have a very specific issue with instructions as well. Apologies for wall of
text in advance. Please let me know what you think
Here's a TLDR:
1. Instructions create what I believe to be a vulnerability.
2. Instructions unclear. Need unambiguous expectations of what
resource monitor allows to occur.or clear well defined goals.
Specifications and scope are too broad
1 Instructions cause what I believe to be a vulnerability.
I noticed in the ABopenfile series, in the reference monitor part one it
states:
When close() is called on the file, if a file is not valid, it is
discarded. If both files are valid, the older one is discarded.
Implying that you keep the a or b file. Keeping the readonly a file, can
cause issues if someone tries to open it. I can email you with what issues
it might cause that I think is just an inherant vulnerability.
The other issue is that the assignment is presented as simulating keeping
a backup in case something goes wrong you have a backup, but it is never
specified that the original file be updated, either with the new valid file
or the backup. Also I am not sure why you would keep either of the files
after updating the original file.
------------------------------
2 Instructions unclear. Specifications and scope are too broad
This being said, the other issue is that it is still confusing as to what
the reference monitor is suppose to do to operate correctly. While it is
suppose to be open ended and is subject to interpretation, I feel like it
does become confusing when there aren't examples of expected behavior. We
have an example where file should not have been written to before closing,
but not what files we should end up with after closing. It's hard to assess
students when students have different interpretations of what is expected.
This causes stress as students are unsure what is and is not valid
behavior and if something is a thing students should allow. This also
causes problems when students write attack code and believe they have found
a vulnerability where the defender believes that what attacker did was
valid behavior and depending on interpretation both can be correct.
------------------------------
While some of my examples seem intuitive such as updating the original
file, because why would you have the newer file be a file.b file what if
you need to update again you would need to update on file.b and create a
file.b.b and so forth however a student might not consider that and take
it at face value (as we are accustomed to math problems where the premise
is grounded in some strange reality where Bob goes to the store bought 500
watermelons, but Amy ate 30 of them how many are left) combined with not
having well defined guidelines expected behavior it becomes ambiguous. I
think to solve this you would have to have well defined guidelines, and
describing how it applies to a real world situation will help greatly +
make the assignment more intuitive. We already have the concept of the file
backup for updating firmware. Perhaps we could outline what that entails
aside from the fact that updates create a backup just in case and tie it
into the assignment.
------------------------------
The language is also a little unclear, while I found "the older one is
discarded" to obviously mean the valid backup as we only want the valid
written to file, it left students concerned with timing. I don't feel like
you would be giving anything away by stating that we would keep the valid
write only b file if both files are valid.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#53?email_source=notifications&email_token=AAGROD6F2DUW5GEAHFJPOHLQKXM67A5CNFSM4GREIWFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7INQDA#issuecomment-533780492>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAGROD26KB3OZDS3N7ZDXQLQKXM67ANCNFSM4GREIWFA>
.
|
|
Sure, I'll start working on it! |
In the last semester, it became apparent that the instructions in docs/EducationalAssignments/ are often confusing, and do not cover the scope of what is required during the assignment.
Most common issue was that the instructions don't do a good enough job of introducing the compatibility requirements that need to be built into the reference monitor so it will not break the functionality of the entire environment.
Example: The majority of students failed to build their reference monitor with support for cases when the file was reopened after closing. This caused issues during the second assignment, when we could not test the quality of the security monitor, because they had not understood the whole scope of the assignment.
Online format classes aggravate this issue significantly. I propose clarifying the requirements of the assignments for future classes. I will also volunteer myself to do so if there is no other candidate.
The text was updated successfully, but these errors were encountered: