Deliverables Summary
This part of the document is provided to help you track where you are in the individual project. This section will be updated in future deliverables.
Deliverable 1
Provided
- edittxt specification (linked in_IPD1)
- Skeleton of the main class for edittxt (linked in_IPD1)
- JUnit 5 example tests and skeleton of test class to submit (linked in_IPD1)
- JUnit 5 library (linked in_IPD1)
Expected
- Part I (Category partition)
- txt: TSL file you created
- txt.tsl: test specifications generated by the TSLCompiler tool when run on your TSL file
- Part II (JUnit tests)
- java: JUnit tests derived from your category partition test frames
Deliverable 2
Provided
- Reference implementation of edittxt utility (through Gradescope)
Expected
- Possibly revised set of tests that (1) pass on the reference implementation, (2) cover a set of scenarios (provided through Gradescope), and (3) have suitable oracles
Deliverable 3
Provided: TBD
Expected: TBD
Deliverable 4
Provided: TBD
Expected: TBD
Deliverable 2: Instructions
In this deliverable, we will assume that someone else in your company implemented the edittxt utility while you were creating test cases for it. For the sake of the assignment, we will also assume that your colleague is an infallible developer and produced a flawless implementation. Consequently, all the test cases you created for Deliverable 1 (D1) are supposed to pass on this implementation.
To complete this deliverable, you will submit your test cases (i.e., class MyMainTest) and make sure that they pass on the reference implementation, which will be hidden from you. Gradescope will run them for you and report information about passing and failing tests. You may have to modify some of your tests, or add or delete tests from the tests submitted in D1. You will not be penalized for this, and you do not need to modify your category-partition files accordingly. Note that we are expecting the test class MyMainTest to be self-contained; that is, your tests should not rely on any external classes or resources.
Grade
Your grade for this deliverable will be the average of three parts:
- The first part of your grade will be proportional to the percentage of your test cases that pass on our (correct) reference implementation. For example, if your test class consists of 50 test cases, and 40 of them pass on the reference implementation, your grade for this part will be
- In addition, Gradescope will check whether your tests contain suitable oracles by computing how many of your tests fail on a buggy implementation of the application (that is, no tests should pass on this buggy implementation). The reason for this check is that having tests with no or inadequate oracles would unfairly boost the first part of your grade, as these tests will always pass on the reference implementation. The third part of your grade will be proportional to the number of your tests that fail on the buggy implementation, so you will get full points if all tests fail, as expected.
- Finally, Gradescope will check to what extent your test cases adequately cover the domain of the application, by measuring the percentage of scenarios (among a set of relevant scenarios that we identified) covered by your tests. To get full points for this part of your grade, your tests must cover at least 90% of the relevant scenarios.
Extra credit: If (1) all of your tests pass on the correct reference implementation, (2) your tests cover all of our representative scenarios, and (3) all of your tests fail on the faulty reference implementation, you will get 10 extra points, for a total of 110 points (for this deliverable). There is no partial credit for this assignment; you must meet all three previous requirements to get the extra credit for this deliverable.
Committing and submitting the deliverable
The tests you submitted for D1 should be able to pass all the checks and be graded, so it may be worth doing an initial submission right away, as a sanity check. Moreover, it could be the case that all of your test cases are already passing on the correct reference implementation, failing on the buggy reference implementation, and covering enough scenarios, in which case you would be done with the mandatory part of the deliverable.
While the autograder provides details on which of your test cases fail on the reference code, it only reports the number of scenarios that your tests do not cover. To improve that part of your score, you will have to consider the program domain again and try to identify cases you may be missing. We cannot provide hints on what scenarios your test suite does not cover.
Following are the steps to commit and submit this deliverable:
- As usual, commit and push your code to your individual, assigned private repository.
- Make sure that all Java files and necessary libraries are committed and pushed. As you did for Deliverable 1, you can check that you committed and pushed all the files you needed by doing the following:
- Clone a fresh copy of your personal repo in another directory.
- Go to directory IndividualProject/edittxt in this fresh clone.
- Compile your code. One way is to run, from a Unix-like shell:
javac -cp lib/\* -d classes src/edu/gatech/seclass/edittxt/*.java test/edu/gatech/seclass/edittxt/*.java[1]
- Run your tests. Again, from a Unix-like shell, you can run:
java -cp classes:lib/\* org.junit.platform.console.ConsoleLauncher –select-class edu.gatech.seclass.edittxt.MyMainTest[2]
- Submit on gradescope a file, called txt that contains, in two separate lines (1) your GT username and (2) the commit ID for your submission. For example, the content of file submission.txt for George P. Burdell could look something like the following:
submission.txt
| gpburdell1 |
| 81b2f59 |
As soon as you submit, Gradescope will grade your submission by:
- Making sure that your files are present and in the correct location.
- Performing the same sanity checks it performed in D1.
- Compiling and running your tests against the correct and faulty reference implementations of edittxt.
- Computing coverage information for your tests.
If any of the above steps fail, you will see a grade of 0 and an error message with some diagnostic information. Please note that, as before, if your submission does not pass the Gradescope checks, it will receive a 0. Conversely, if Gradescope can successfully compile and run your code and tests, you will immediately receive a grade that is your actual grade for this deliverable. Note that you can resubmit as many times as you want before the deadline.
Gradescope Feedback
Unlike other assignments, this deliverable is not available to be submitted on Gradescope until the previous deliverable is due. That is, deliverable 2 will be available on Gradescope 48 hours after this assignment is released on Canvas.
The test results you see in Gradescope tell you whether a given test passed or not. If the test didn’t pass, Gradescope should show the difference between the expected and actual output.
As stated in the Grade section, Gradescope will show your score will be calculated based on
- the number of tests that pass on the correct reference implementation, displayed as:
Student tests passing on CORRECT reference code: X/Y
where X is the number of tests passing on the correct reference code and Y is the total number of tests you designed. All tests must pass on this implementation, so X should be equal to Y.
- the number of tests that pass on the faulty reference implementation, displayed as:
Student tests passing on FAULTY reference code: Z/Y
where Z is the number of tests passing on the faulty reference implementation and Y is the total number of tests you designed. All tests must fail on this implementation (since this implementation is incorrect), so Z should be 0.
- the number of scenarios your tests cover based on instructor relevant scenarios, displayed as:
Coverage achieved: <percentage achieved>% (W/T)
where W is the amount of scenarios your test suite uncovered, T is the total number of coverage scenarios we designed, and percentage achieved is the percentage representation of W/T.
Requests for clarifications
If you need clarifications on a specific test or Gradescope output, please post privately on Ed Discussion (if appropriate, we will make it public) and make sure to add, when applicable:
- a link to the Gradescope results,
- an inlined, complete copy of the test(s) you reference, if yours (i.e., no screenshots or commit IDs), and
- any information that may be relevant.
The bottom line is that, to make the interaction efficient, you should make your posts as self-contained and easy-to-check as possible. The faster we can respond to the posts, the more students we can help. Although we tested the autograder, it may still handle some corner cases incorrectly. If you receive feedback that seems to be incorrect, please contact us on Ed Discussion using the approach we describe above.
FAQ
- Can options “consume” other flags? For example, is “-k”, “-i” valid?
Answer: Yes, this shouldn’t result in an error and does not change any behavior. This will keep the lines containing the given substring “-i”. This happens because we parse first, so “-i” becomes the parameter for “-k”, just like any other string.
- Can I update the catpart to increase coverage?
Answer: Yes, you can update the catpart, although it will not be graded.
- Will the coverage scenarios be revealed after this deliverable?
Answer: We cannot reveal any of the scenarios tested against.
[1] On some platforms, you may need to first create directory “classes”.
[2] If using a Windows-based system, you may need to run
java -cp “classes;lib/*” org.junit.platform.console.ConsoleLauncher
–select-class edu.gatech.seclass.edittxt.MyMainTest






