For Mac & Linux
Step 1: Set up Defects4J
Refer https://github.com/rjust/defects4j
Clone the repository
Follow documentation steps
Error:Can't Locate DBI.pm
Resolution for Mac: perl -MCPAN -e 'install DBI'
Install Postgres if required.
Resolution for Linux: sudo apt install libdbi-perl
Step 2: Clone the repo https://bitbucket.org/rjust/fault-localization-data/overview
Step 3: Download and install JDK 1.6 and JDK 1.8.
Step 4: Set evnironment variables
The path of defects4j installed in step 1
export D4J_HOME=/Users/{username}/Downloads/defects4j
The path to this root directory of the cloned repo fault-localization-data and append 'gzoltar/gzoltar.jar'
export GZOLTAR_JAR=/Users/{username}/Downloads/fault-localization-data/gzoltar/gzoltar.jar
Set JAVA_HOME to point to JDK1.6 Home if you have a different Java default version
export JAVA_HOME=/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Add to PATH variable
export PATH=$PATH:$D4J_HOME/framework/bin
Step 6: Replace the run_gzoltar.sh provided in this repository in fault-localization-data/gzoltar/gzoltar
Step 5: Test if set up works
Test defects4j
defects4j info -p Lang
Test Gzoltar
`bash run_gzoltar.sh Lang 37 . developer`
Step 7: Install sloccount
For Mac: brew install sloccount
For Mac: export SLOC_HOME=/usr/local/bin/sloccount
For Ubuntu: sudo apt-get install sloccount
For Ubuntu: export SLOC_HOME=/usr/bin/sloccount
Step 8: Run the script to get the buggy code
bash get_fixed_lines.sh Lang 37 .
This creates a file `Lang-37.fixed.lines` which contains the line fixed in the human patch for the corresponding bug.
This can be used to evaluate the suspiciousness score generated by the Fault localization technique.
To manually run junit tests in the defects4j projects
Step 1: checkout defects4j project(example: Lang)
defects4j checkout -p Lang -v 37b -w /tmp/Lang37
Step 2: Download JUNIT Jar
Download the junit jar from https://github.com/downloads/junit-team/junit/junit-4.10.jar
Step 3: Downgrade Maven to mvn 3.2 so as to run with Java 1.6
Check maven version by running.
mvn --version
If this is 3.2 or below then skip to Step 4.
If not follow these steps to downgrade maven
brew install [email protected]
brew unline maven
brew link --force --overwrite [email protected]
Step 4: Compile Lang Project
Go to the checked out Lang project folder in two session on Terminal
Session 1:
export JAVA_HOME=/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
mvn compile (This will fail)
Session 2:
mvn compile
rm -rf target/*
Session 1:
mvn compile (This should succeed)
Step 5: Run all of the Junit tests in Lang Project using maven
Session 1:
mvn -Dmaven.test.failure.ignore=true install (This will fail)
Session 2:
mvn -Dmaven.test.failure.ignore=true install
rm -rf target/commons-lang-3.0-SNAPSHOT.jar
Session 1:
mvn -Dmaven.test.failure.ignore=true install (This should succeed)
mvn test
Step 6: Run one of the Junit tests in Lang Project from command line without maven
cd target
cp {Path to junit jar}/junit-4.10.jar .
java -cp .:/tmp/Lang37/target/test-classes/:junit-4.10.jar:commons-lang-3.0-SNAPSHOT.jar org.junit.runner.JUnitCore org.apache.commons.lang3.ArrayUtilsAddTest
Step 7a: Setting up javaslicer Clone the repository for javaslicer (https://github.com/hammacher/javaslicer) cd javaslicer ./assemble.sh (Note: There might be failures when you run this the first time. Run ./assemble.sh to fix it.)
Step 7b: Run one of the junit test classes with tracer as a javaagent attached.
Session 2:
java -cp .:/tmp/Lang37/target/test-classes/:junit-4.10.jar:commons-lang-3.0-SNAPSHOT.jar -javaagent:/Users/jithinjohn/Downloads/CSC591/javaslicer/assembly/tracer.jar=tracefile:test.trace org.junit.runner.JUnitCore org.apache.commons.lang3.ArrayUtilsAddTest
Step 8: Run slicer to produce slicing results
Session 2:
java -Xmx2g -jar /Users/jithinjohn/Downloads/CSC591/javaslicer/assembly/slicer.jar -p test.trace org.apache.commons.lang3.ArrayUtils.addAll:2962:* > output.txt
Here are sample commands on how to run the slicer, tracer along with some utilities developed for the automation manually. These are performed by the automation script as specified below.
Change directory to automation/src/
COMPILE:
javac -cp .:/tmp/Lang37/target/:/tmp/Lang37/target/test-classes/:/tmp/Lang37/target/junit-4.10.jar:/tmp/Lang37/target/commons-lang-3.0-SNAPSHOT.jar InvokeTests.java
RUN:
- Running a test file
java -cp .:/tmp/Lang37/target/:/tmp/Lang37/target/test-classes/:/tmp/Lang37/target/junit-4.10.jar:/tmp/Lang37/target/commons-lang-3.0-SNAPSHOT.jar InvokeTests /tmp/Lang37 runTestFile org.apache.commons.lang3.ArrayUtilsAddTest
- Running a test specific test case
java -cp .:/tmp/Lang37/target/:/tmp/Lang37/target/test-classes/:/tmp/Lang37/target/junit-4.10.jar:/tmp/Lang37/target/commons-lang-3.0-SNAPSHOT.jar InvokeTests /tmp/Lang37 runTestCase org.apache.commons.lang3.ArrayUtilsAddTest testJira567
- Getting test cases of a test file
java -cp .:/tmp/Lang37/target/:/tmp/Lang37/target/test-classes/:/tmp/Lang37/target/junit-4.10.jar:/tmp/Lang37/target/commons-lang-3.0-SNAPSHOT.jar InvokeTests /tmp/Lang37 getTestCases org.apache.commons.lang3.ArrayUtilsAddTest
- Getting line numbers of assert statements in a test case
java -cp .:/tmp/Lang37/target/:/tmp/Lang37/target/test-classes/:/tmp/Lang37/target/junit-4.10.jar:/tmp/Lang37/target/commons-lang-3.0-SNAPSHOT.jar InvokeTests /tmp/Lang37 getAssertLines org.apache.commons.lang3.ArrayUtilsAddTest testJira567
The experiments evaluate various fault localization techniques on artificial faults and on real faults.
At a high level, here's how it all works:
- The real and artificial faults come from the Defects4J Project.
- For each D4J fault, the scripts in
d4j_integration/determine which lines are faulty. The resultant files are "buggy-lines" files, and live inanalysis/pipeline-scripts/buggy-lines/. - Many fault localization techniques require coverage information. We use GZoltar and Tacoco to gather coverage information. The resultant files are called "matrix", "spectra" and {defectId}-cov-matrix.json.
- Our scripts enable you to compute all the suspiciousness score and coverage information, but doing so takes a great deal of computation.
-
To run Test Case Purification and Rank refinement, use
automation/automate.py.Example invocation:
python3 automate.py \ Lang 37 \ /tmp/Lang-37Creates the files
Lang-37.killmap.csv.gzandLang-37.mutants.log. -
To run GZoltar seperately, use
gzoltar/run_gzoltar.sh.Example invocation:
bash run_gzoltar.sh Lang 37 . developerCreates the files
./matrixand./spectra.
-
automation/: Scripts and tools required to run analysis on the bugs in defects4j. -
d4j_integration/: Scripts that build upon or extend Defects4J to populate or query its database. -
gzoltar/: Scripts for running the GZoltar tool to collect line coverage information.
=======
If the automate.py script was invoked with /tmp/LangTest as work directory for example say with Project as Lang and Bug id 37 then the results in the word directory would look like this.
`tmp/LangTest`: Root Working directory all results are under this main directory.
│
├──`Lang/`: Any bugs from Lang Project would checked out using defects4j under this folder.
│ │
│ ├──`37/`: Bug ID - 37 is checked out here, then compiled and packaged to the Jar.
│
├──`gzoltars/`: All results generated by gzoltar is this directory.
│ │
│ ├──`Lang/`: Results of gzoltar for any bugs from Lang Project would created here.
│ │ │
│ │ ├──`37/`: Gzoltar results for Bug Id 37 would be present here including the coverage matrix, spectra.
│
├──`tacocos/`: Results of the tacoco tool and the final results of the tool would be here.
│ │
│ ├──`Lang/`: Results for all bugs in Lang Project
│
├──`traces/`: Any bugs from Lang Project would checked out using defects4j under this folder.
│ │
│ ├──`Lang/`: Results tracer and slicer for all bugs in Lang Project
│ │ │
│ │ ├──`37/`: Tracer and slicer output files for bug id 37 in Lang Project .